Original Research

Electronic consultation (e-consult) is designed to increase access to specialty care by facilitating communication between primary care and specialty clinicians without the need for outpatient face-to-face encounters.^1–4 In 2011, the US Department of Veterans Affairs (VA) implemented an e-consult program as a component of its overall strategy to increase access to specialty services, reduce costs of care, and reduce appointment travel burden on patients.

E-consult has substantially increased within the VA since its implementation.^5,6 Consistent with limited evaluations from other health care systems, evaluations of the VA e-consult program demonstrated reduced costs, reduced travel time for patients, and improved access to specialty care.^2,5–11 However, there is wide variation in e-consult use across VA specialties, facilities, and regions.^5,6,12,13 For example, hematology, preoperative evaluation, neurosurgery, endocrinology, and infectious diseases use e-consults more frequently when compared with in-person consults in the VA.⁶ Reasons for this variation or specific barriers and facilitators of using e-consults have not been described.

Prior qualitative studies report that primary care practitioners (PCPs) describe e-consults as convenient, educational, beneficial for patient care, and useful for improving patient access to specialty care.^8,14,15 One study identified limited PCP knowledge of e-consults as a barrier to use.¹⁶ Specialists have reported that e-consult improves clinical communication, but increases their workload.^1,14,17,18 These studies did not assess perspectives from both clinicians who initiate e-consults and those who respond to them. This is the first qualitative study to assess e-consult perceptions from perspectives of both PCPs and specialists among a large, national sample of VA clinicians who use e-consults. The objective of this study was to understand perspectives of e-consults between PCPs and specialists that may be relevant to increasing adoption in the VA.

Methods

The team (CL, ML, PG, 2 analysts under the guidance of GS and JS and support from RRK, and a biostatistician) conducted semistructured interviews with PCPs, specialists, and specialty division leaders who were employed by VA in 2016 and 2017. Specialties of interest were identified by the VA Office of Specialty Care and included cardiology, endocrinology, gastroenterology, and hematology.

E-Consult Procedures

Within the VA, the specific procedures used to initiate, triage and manage e-consults are coordinated at VA medical centers (VAMCs) and at the Veterans Integrated Service Network (VISN) regional level. E-consult can be requested by any clinician. Generally, e-consults are initiated by PCPs through standardized, specialty-specific templates. Recipients, typically specialists, respond by answering questions, suggesting additional testing and evaluation, or requesting an in-person visit. Communication is documented in the patient’s electronic health record (EHR). Specialists receive different levels of workload credit for responding to e-consults similar to a relative value unit reimbursement model. Training in the use of e-consults is available to practitioners but may vary at local and regional levels.

Recruitment

Our sample included PCPs, specialists, and specialty care division leaders. We first quantified e-consult rates (e-consults per 100 patient visits) between July 2016 and June 2017 at VA facilities within primary care and the 4 priority specialties and identified the 30 sites with the highest e-consult rates and 30 sites with the lowest e-consult rates. Sites with < 500 total visits, < 3 specialties, or without any e-consult visit during the study period were excluded. E-consult rates at community-based outpatient clinics were included with associated VAMCs. We then stratified PCPs by whether they were high or low users of e-consults (determined by the top and bottom users within each site) and credentials (MD vs nurse practitioner [NP] or physician assistant [PA]). Specialists were sampled based on their rate of use relative to colleagues within their site and the use rate of their division. We sampled division chiefs and individuals who had > 300 total visits and 1 e-consult during the study period. To recruit participants, the primary investigator sent an initial email and 2 reminder emails. The team followed up with respondents to schedule an interview.

Interview guides were designed to elicit rich descriptions of barriers and facilitators to e-consult use (eAppendix available at doi:10.12788/fp.0214). The team used the Practical Robust Implementation and Sustainability Model (PRISM), which considers factors along 6 domains for intervention planning, implementation, and sustainment.¹⁹ Telephone interviews lasted about 20 minutes and were conducted between September 2017 and March 2018. Interviews were recorded and transcribed verbatim.

Analysis

The team used an iterative, team-based, inductive/deductive approach to conventional content analysis.^20,21 Initial code categories were created so that we could identify e-consult best practices—facilitators of e-consult that were recommended by both PCPs and specialists. Inductive codes or labels applied to identify meaningful quotations, phrases, or key terms were used to identify emergent ideas and were added throughout coding after discussion among team members. Consensus was reached using a team-based approach.²¹ Four analysts independently coded the same 3 transcripts and met to discuss points of divergence and convergence. Analyses continued with emergent themes, categories, and conclusions. Atlas.ti. v.7 was used for coding and data management.²²

Results

We conducted 34 interviews with clinicians (Table 1) from 13 VISNs. Four best-practice themes emerged among both PCPs and specialists, including that e-consults (1) are best suited for certain clinical questions and patients; (2) require relevant background information from requesting clinicians and clear recommendations from responding clinicians; (3) are a novel opportunity to provide efficient, transparent care; and (4) may not be fully adopted due to low awareness. Supporting quotations for the following findings are provided in Table 2.

Specific Clinical Questions and Patients

PCPs described specific patients and questions for which they most frequently used e-consults, such as for medication changes (Q1), determining treatment steps (Q2,3), and or clarifying laboratory or imaging findings. PCPs frequently used e-consults for patients who did not require a physical examination or when specialists could make recommendations without seeing patients face-to-face (Q3). An important use of e-consults described by PCPs was for treating conditions they could manage within primary care if additional guidance were available (Q4). Several PCPs and specialists also noted that e-consults were particularly useful for patients who were unable to travel or did not want face-to-face appointments (Q5). Notably, PCPs and specialists mentioned situations for which e-consults were inappropriate, including when a detailed history or physical examination was needed, or if a complex condition was suspected (Q6).

Background Data and Clear Recommendations

Participants described necessary data that should be included in high-quality e-consults. Specialists voiced frustration in time-consuming chart reviews that were often necessary when these data were not provided by the requestor. In some cases, specialists were unable to access necessary EHR data, which delayed responses (Q7). PCPs noted that the most useful responses carefully considered the question, used current patient information to determine treatments, provided clear recommendations, and defined who was responsible for next steps (Q8). PCPs and specialists stated that e-consult templates that required relevant information facilitated high-quality e-consults. Neither wanted to waste the other clinician's time (Q8).

A Novel Opportunity

Many PCPs felt that e-consults improved communication (eg, efficiency, response time), established new communication between clinicians, and reduced patients’ appointment burden (Q10, Q11). Many specialists felt that e-consults improved documentation of communication between clinicians and increased transparency of clinical decisions (Q12). Additionally, many specialists mentioned that e-consults capture previously informal curbside consults, enabling them to receive workload credit (Q13).

Lack of Awareness

Some noted that the biggest barrier to e-consults was not being aware of them generally, or which specialties offer e-consults (Q14). One PCP described e-consults as the best kept secret and found value in sharing the utility of e-consults with colleagues (Q15). All participants, including those who did not frequently use e-consults, felt that e-consults improved the quality of care by providing more timely care or better answers to clinical questions (Q16). Several practitioners also felt that e-consults increased access to specialty care. For example, specialists reported that e-consults enabled them to better manage patient load by using e-consults to answer relatively simple questions, reserving face-to-face consults for more complex patients (Q17).

Discussion

The objective of this study was to identify potential best practices for e-consults that may help increase their quality and use within the VA. We built on prior studies that offered insights on PCP and specialists’ overall satisfaction with e-consult by identifying several themes relevant to the further adoption of e-consults in the VA and elsewhere without a face-to-face visit.^{8,13,14,16–18} Future work may be beneficial in identifying whether the study themes identified can explain variation in e-consult use or whether addressing these factors might lead to increased or higher quality e-consult use. We are unaware of any qualitative study of comparable scale in a different health care system. Further, this is the first study to assess perspectives on e-consults among those who initiate and respond to them within the same health care system. Perhaps the most important finding from this study is that e-consults are generally viewed favorably, which is a necessary leverage point to increase their adoption within the system.

Clinicians reported several benefits to e-consults, including timely responses to clinical questions, efficient communication, allow for documentation of specialist recommendations, and help capture workload. These benefits are consistent with prior literature that indicates both PCPs and specialists in the VA and other health care systems feel that e-consults improves communication, decreases unnecessary visits, and improves quality of care.^1,14,17,18 In particular, clinicians reported that e-consults improve their practice efficiency and efficacy. This is of critical importance given the pressures of providing timely access to primary and specialty care within the VA. Interestingly, many VA practitioners were unaware which specialties offered e-consults within their facilities, reflecting previous work showing that PCPs are often unaware of e-consult options.¹⁶ This may partially explain variation in e-consult use. Increasing awareness and educating clinicians on the benefits of e-consults may help promote use among non- and low users.

A common theme reported by both groups was the importance of providing necessary information within e-consult questions and responses. Specialists felt there was a need to ensure that PCPs provide relevant and patient-specific information that would enable them to efficiently and accurately answer questions without the need for extensive EHR review. This reflects previous work showing that specialists are often unable to respond to e-consult requests because they do not contain sufficient information.²² PCPs described a need to ensure that specialists’ responses included information that was detailed enough to make clinical decisions without the need for a reconsult. This highlights a common challenge to medical consultation, in that necessary or relevant information may not be apparent to all clinicians. To address this, there may be a role in developing enhanced, flexible templating that elicits necessary patient-specific information. Such a template may automatically pull relevant data from the EHR and prompt clinicians to provide important information. We did not assess how perspectives of templates varied, and further work could help define precisely what constitutes an effective template, including how it should capture appropriate patient data and how this impacts acceptability or use of e-consults generally. Collaboratively developed service agreements and e-consult templates could help guide PCPs and specialists to engage in efficient communication.

Another theme among both groups was that e-consult is most appropriate within specific clinical scenarios. Examples included review of laboratory results, questions about medication changes, or for patients who were reluctant to travel to appointments. Identifying and promoting specific opportunities for e-consults may help increase their use and align e-consult practices with scenarios that are likely to provide the most benefit to patients. For example, it could be helpful to understand the distance patients must travel for specialty care. Providing that information during clinical encounters could trigger clinicians to consider e-consults as an option. Future work might aim to identify clinical scenarios that clinicians feel are not well suited for e-consults and determine how to adapt them for those scenarios.

Limitations

Generalizability of these findings is limited given the qualitative study design. Participants’ descriptions of experiences with e-consults reflect the experiences of clinicians in the VA and may not reflect clinicians in other settings. We also interviewed a sample of clinicians who were already using e-consults. Important information could be learned from future work with those who have not yet adopted e-consult procedures or adopted and abandoned them.

Conclusions

E-consult is perceived as beneficial by VA PCPs and specialists. Participants suggested using e-consults for appropriate questions or patients and including necessary information and next steps in both the initial e-consult and response. Finding ways to facilitate e-consults with these suggestions in mind may increase delivery of high-quality e-consults. Future work could compare the findings of this work to similar work assessing clinicians perceptions of e-consults outside of the VA.

Electronic consultation (e-consult) is designed to increase access to specialty care by facilitating communication between primary care and specialty clinicians without the need for outpatient face-to-face encounters.^1–4 In 2011, the US Department of Veterans Affairs (VA) implemented an e-consult program as a component of its overall strategy to increase access to specialty services, reduce costs of care, and reduce appointment travel burden on patients.

E-consult has substantially increased within the VA since its implementation.^5,6 Consistent with limited evaluations from other health care systems, evaluations of the VA e-consult program demonstrated reduced costs, reduced travel time for patients, and improved access to specialty care.^2,5–11 However, there is wide variation in e-consult use across VA specialties, facilities, and regions.^5,6,12,13 For example, hematology, preoperative evaluation, neurosurgery, endocrinology, and infectious diseases use e-consults more frequently when compared with in-person consults in the VA.⁶ Reasons for this variation or specific barriers and facilitators of using e-consults have not been described.

Prior qualitative studies report that primary care practitioners (PCPs) describe e-consults as convenient, educational, beneficial for patient care, and useful for improving patient access to specialty care.^8,14,15 One study identified limited PCP knowledge of e-consults as a barrier to use.¹⁶ Specialists have reported that e-consult improves clinical communication, but increases their workload.^1,14,17,18 These studies did not assess perspectives from both clinicians who initiate e-consults and those who respond to them. This is the first qualitative study to assess e-consult perceptions from perspectives of both PCPs and specialists among a large, national sample of VA clinicians who use e-consults. The objective of this study was to understand perspectives of e-consults between PCPs and specialists that may be relevant to increasing adoption in the VA.

Methods

The team (CL, ML, PG, 2 analysts under the guidance of GS and JS and support from RRK, and a biostatistician) conducted semistructured interviews with PCPs, specialists, and specialty division leaders who were employed by VA in 2016 and 2017. Specialties of interest were identified by the VA Office of Specialty Care and included cardiology, endocrinology, gastroenterology, and hematology.

E-Consult Procedures

Within the VA, the specific procedures used to initiate, triage and manage e-consults are coordinated at VA medical centers (VAMCs) and at the Veterans Integrated Service Network (VISN) regional level. E-consult can be requested by any clinician. Generally, e-consults are initiated by PCPs through standardized, specialty-specific templates. Recipients, typically specialists, respond by answering questions, suggesting additional testing and evaluation, or requesting an in-person visit. Communication is documented in the patient’s electronic health record (EHR). Specialists receive different levels of workload credit for responding to e-consults similar to a relative value unit reimbursement model. Training in the use of e-consults is available to practitioners but may vary at local and regional levels.

Recruitment

Our sample included PCPs, specialists, and specialty care division leaders. We first quantified e-consult rates (e-consults per 100 patient visits) between July 2016 and June 2017 at VA facilities within primary care and the 4 priority specialties and identified the 30 sites with the highest e-consult rates and 30 sites with the lowest e-consult rates. Sites with < 500 total visits, < 3 specialties, or without any e-consult visit during the study period were excluded. E-consult rates at community-based outpatient clinics were included with associated VAMCs. We then stratified PCPs by whether they were high or low users of e-consults (determined by the top and bottom users within each site) and credentials (MD vs nurse practitioner [NP] or physician assistant [PA]). Specialists were sampled based on their rate of use relative to colleagues within their site and the use rate of their division. We sampled division chiefs and individuals who had > 300 total visits and 1 e-consult during the study period. To recruit participants, the primary investigator sent an initial email and 2 reminder emails. The team followed up with respondents to schedule an interview.

Interview guides were designed to elicit rich descriptions of barriers and facilitators to e-consult use (eAppendix available at doi:10.12788/fp.0214). The team used the Practical Robust Implementation and Sustainability Model (PRISM), which considers factors along 6 domains for intervention planning, implementation, and sustainment.¹⁹ Telephone interviews lasted about 20 minutes and were conducted between September 2017 and March 2018. Interviews were recorded and transcribed verbatim.

Analysis

The team used an iterative, team-based, inductive/deductive approach to conventional content analysis.^20,21 Initial code categories were created so that we could identify e-consult best practices—facilitators of e-consult that were recommended by both PCPs and specialists. Inductive codes or labels applied to identify meaningful quotations, phrases, or key terms were used to identify emergent ideas and were added throughout coding after discussion among team members. Consensus was reached using a team-based approach.²¹ Four analysts independently coded the same 3 transcripts and met to discuss points of divergence and convergence. Analyses continued with emergent themes, categories, and conclusions. Atlas.ti. v.7 was used for coding and data management.²²

Results

We conducted 34 interviews with clinicians (Table 1) from 13 VISNs. Four best-practice themes emerged among both PCPs and specialists, including that e-consults (1) are best suited for certain clinical questions and patients; (2) require relevant background information from requesting clinicians and clear recommendations from responding clinicians; (3) are a novel opportunity to provide efficient, transparent care; and (4) may not be fully adopted due to low awareness. Supporting quotations for the following findings are provided in Table 2.

Specific Clinical Questions and Patients

PCPs described specific patients and questions for which they most frequently used e-consults, such as for medication changes (Q1), determining treatment steps (Q2,3), and or clarifying laboratory or imaging findings. PCPs frequently used e-consults for patients who did not require a physical examination or when specialists could make recommendations without seeing patients face-to-face (Q3). An important use of e-consults described by PCPs was for treating conditions they could manage within primary care if additional guidance were available (Q4). Several PCPs and specialists also noted that e-consults were particularly useful for patients who were unable to travel or did not want face-to-face appointments (Q5). Notably, PCPs and specialists mentioned situations for which e-consults were inappropriate, including when a detailed history or physical examination was needed, or if a complex condition was suspected (Q6).

Background Data and Clear Recommendations

Participants described necessary data that should be included in high-quality e-consults. Specialists voiced frustration in time-consuming chart reviews that were often necessary when these data were not provided by the requestor. In some cases, specialists were unable to access necessary EHR data, which delayed responses (Q7). PCPs noted that the most useful responses carefully considered the question, used current patient information to determine treatments, provided clear recommendations, and defined who was responsible for next steps (Q8). PCPs and specialists stated that e-consult templates that required relevant information facilitated high-quality e-consults. Neither wanted to waste the other clinician's time (Q8).

A Novel Opportunity

Many PCPs felt that e-consults improved communication (eg, efficiency, response time), established new communication between clinicians, and reduced patients’ appointment burden (Q10, Q11). Many specialists felt that e-consults improved documentation of communication between clinicians and increased transparency of clinical decisions (Q12). Additionally, many specialists mentioned that e-consults capture previously informal curbside consults, enabling them to receive workload credit (Q13).

Lack of Awareness

Some noted that the biggest barrier to e-consults was not being aware of them generally, or which specialties offer e-consults (Q14). One PCP described e-consults as the best kept secret and found value in sharing the utility of e-consults with colleagues (Q15). All participants, including those who did not frequently use e-consults, felt that e-consults improved the quality of care by providing more timely care or better answers to clinical questions (Q16). Several practitioners also felt that e-consults increased access to specialty care. For example, specialists reported that e-consults enabled them to better manage patient load by using e-consults to answer relatively simple questions, reserving face-to-face consults for more complex patients (Q17).

Discussion

The objective of this study was to identify potential best practices for e-consults that may help increase their quality and use within the VA. We built on prior studies that offered insights on PCP and specialists’ overall satisfaction with e-consult by identifying several themes relevant to the further adoption of e-consults in the VA and elsewhere without a face-to-face visit.^{8,13,14,16–18} Future work may be beneficial in identifying whether the study themes identified can explain variation in e-consult use or whether addressing these factors might lead to increased or higher quality e-consult use. We are unaware of any qualitative study of comparable scale in a different health care system. Further, this is the first study to assess perspectives on e-consults among those who initiate and respond to them within the same health care system. Perhaps the most important finding from this study is that e-consults are generally viewed favorably, which is a necessary leverage point to increase their adoption within the system.

Clinicians reported several benefits to e-consults, including timely responses to clinical questions, efficient communication, allow for documentation of specialist recommendations, and help capture workload. These benefits are consistent with prior literature that indicates both PCPs and specialists in the VA and other health care systems feel that e-consults improves communication, decreases unnecessary visits, and improves quality of care.^1,14,17,18 In particular, clinicians reported that e-consults improve their practice efficiency and efficacy. This is of critical importance given the pressures of providing timely access to primary and specialty care within the VA. Interestingly, many VA practitioners were unaware which specialties offered e-consults within their facilities, reflecting previous work showing that PCPs are often unaware of e-consult options.¹⁶ This may partially explain variation in e-consult use. Increasing awareness and educating clinicians on the benefits of e-consults may help promote use among non- and low users.

A common theme reported by both groups was the importance of providing necessary information within e-consult questions and responses. Specialists felt there was a need to ensure that PCPs provide relevant and patient-specific information that would enable them to efficiently and accurately answer questions without the need for extensive EHR review. This reflects previous work showing that specialists are often unable to respond to e-consult requests because they do not contain sufficient information.²² PCPs described a need to ensure that specialists’ responses included information that was detailed enough to make clinical decisions without the need for a reconsult. This highlights a common challenge to medical consultation, in that necessary or relevant information may not be apparent to all clinicians. To address this, there may be a role in developing enhanced, flexible templating that elicits necessary patient-specific information. Such a template may automatically pull relevant data from the EHR and prompt clinicians to provide important information. We did not assess how perspectives of templates varied, and further work could help define precisely what constitutes an effective template, including how it should capture appropriate patient data and how this impacts acceptability or use of e-consults generally. Collaboratively developed service agreements and e-consult templates could help guide PCPs and specialists to engage in efficient communication.

Another theme among both groups was that e-consult is most appropriate within specific clinical scenarios. Examples included review of laboratory results, questions about medication changes, or for patients who were reluctant to travel to appointments. Identifying and promoting specific opportunities for e-consults may help increase their use and align e-consult practices with scenarios that are likely to provide the most benefit to patients. For example, it could be helpful to understand the distance patients must travel for specialty care. Providing that information during clinical encounters could trigger clinicians to consider e-consults as an option. Future work might aim to identify clinical scenarios that clinicians feel are not well suited for e-consults and determine how to adapt them for those scenarios.

Limitations

Generalizability of these findings is limited given the qualitative study design. Participants’ descriptions of experiences with e-consults reflect the experiences of clinicians in the VA and may not reflect clinicians in other settings. We also interviewed a sample of clinicians who were already using e-consults. Important information could be learned from future work with those who have not yet adopted e-consult procedures or adopted and abandoned them.

Conclusions

E-consult is perceived as beneficial by VA PCPs and specialists. Participants suggested using e-consults for appropriate questions or patients and including necessary information and next steps in both the initial e-consult and response. Finding ways to facilitate e-consults with these suggestions in mind may increase delivery of high-quality e-consults. Future work could compare the findings of this work to similar work assessing clinicians perceptions of e-consults outside of the VA.

Electronic consultation (e-consult) is designed to increase access to specialty care by facilitating communication between primary care and specialty clinicians without the need for outpatient face-to-face encounters.^1–4 In 2011, the US Department of Veterans Affairs (VA) implemented an e-consult program as a component of its overall strategy to increase access to specialty services, reduce costs of care, and reduce appointment travel burden on patients.

E-consult has substantially increased within the VA since its implementation.^5,6 Consistent with limited evaluations from other health care systems, evaluations of the VA e-consult program demonstrated reduced costs, reduced travel time for patients, and improved access to specialty care.^2,5–11 However, there is wide variation in e-consult use across VA specialties, facilities, and regions.^5,6,12,13 For example, hematology, preoperative evaluation, neurosurgery, endocrinology, and infectious diseases use e-consults more frequently when compared with in-person consults in the VA.⁶ Reasons for this variation or specific barriers and facilitators of using e-consults have not been described.

Prior qualitative studies report that primary care practitioners (PCPs) describe e-consults as convenient, educational, beneficial for patient care, and useful for improving patient access to specialty care.^8,14,15 One study identified limited PCP knowledge of e-consults as a barrier to use.¹⁶ Specialists have reported that e-consult improves clinical communication, but increases their workload.^1,14,17,18 These studies did not assess perspectives from both clinicians who initiate e-consults and those who respond to them. This is the first qualitative study to assess e-consult perceptions from perspectives of both PCPs and specialists among a large, national sample of VA clinicians who use e-consults. The objective of this study was to understand perspectives of e-consults between PCPs and specialists that may be relevant to increasing adoption in the VA.

Methods

The team (CL, ML, PG, 2 analysts under the guidance of GS and JS and support from RRK, and a biostatistician) conducted semistructured interviews with PCPs, specialists, and specialty division leaders who were employed by VA in 2016 and 2017. Specialties of interest were identified by the VA Office of Specialty Care and included cardiology, endocrinology, gastroenterology, and hematology.

E-Consult Procedures

Within the VA, the specific procedures used to initiate, triage and manage e-consults are coordinated at VA medical centers (VAMCs) and at the Veterans Integrated Service Network (VISN) regional level. E-consult can be requested by any clinician. Generally, e-consults are initiated by PCPs through standardized, specialty-specific templates. Recipients, typically specialists, respond by answering questions, suggesting additional testing and evaluation, or requesting an in-person visit. Communication is documented in the patient’s electronic health record (EHR). Specialists receive different levels of workload credit for responding to e-consults similar to a relative value unit reimbursement model. Training in the use of e-consults is available to practitioners but may vary at local and regional levels.

Recruitment

Our sample included PCPs, specialists, and specialty care division leaders. We first quantified e-consult rates (e-consults per 100 patient visits) between July 2016 and June 2017 at VA facilities within primary care and the 4 priority specialties and identified the 30 sites with the highest e-consult rates and 30 sites with the lowest e-consult rates. Sites with < 500 total visits, < 3 specialties, or without any e-consult visit during the study period were excluded. E-consult rates at community-based outpatient clinics were included with associated VAMCs. We then stratified PCPs by whether they were high or low users of e-consults (determined by the top and bottom users within each site) and credentials (MD vs nurse practitioner [NP] or physician assistant [PA]). Specialists were sampled based on their rate of use relative to colleagues within their site and the use rate of their division. We sampled division chiefs and individuals who had > 300 total visits and 1 e-consult during the study period. To recruit participants, the primary investigator sent an initial email and 2 reminder emails. The team followed up with respondents to schedule an interview.

Interview guides were designed to elicit rich descriptions of barriers and facilitators to e-consult use (eAppendix available at doi:10.12788/fp.0214). The team used the Practical Robust Implementation and Sustainability Model (PRISM), which considers factors along 6 domains for intervention planning, implementation, and sustainment.¹⁹ Telephone interviews lasted about 20 minutes and were conducted between September 2017 and March 2018. Interviews were recorded and transcribed verbatim.

Analysis

The team used an iterative, team-based, inductive/deductive approach to conventional content analysis.^20,21 Initial code categories were created so that we could identify e-consult best practices—facilitators of e-consult that were recommended by both PCPs and specialists. Inductive codes or labels applied to identify meaningful quotations, phrases, or key terms were used to identify emergent ideas and were added throughout coding after discussion among team members. Consensus was reached using a team-based approach.²¹ Four analysts independently coded the same 3 transcripts and met to discuss points of divergence and convergence. Analyses continued with emergent themes, categories, and conclusions. Atlas.ti. v.7 was used for coding and data management.²²

Results

We conducted 34 interviews with clinicians (Table 1) from 13 VISNs. Four best-practice themes emerged among both PCPs and specialists, including that e-consults (1) are best suited for certain clinical questions and patients; (2) require relevant background information from requesting clinicians and clear recommendations from responding clinicians; (3) are a novel opportunity to provide efficient, transparent care; and (4) may not be fully adopted due to low awareness. Supporting quotations for the following findings are provided in Table 2.

Specific Clinical Questions and Patients

PCPs described specific patients and questions for which they most frequently used e-consults, such as for medication changes (Q1), determining treatment steps (Q2,3), and or clarifying laboratory or imaging findings. PCPs frequently used e-consults for patients who did not require a physical examination or when specialists could make recommendations without seeing patients face-to-face (Q3). An important use of e-consults described by PCPs was for treating conditions they could manage within primary care if additional guidance were available (Q4). Several PCPs and specialists also noted that e-consults were particularly useful for patients who were unable to travel or did not want face-to-face appointments (Q5). Notably, PCPs and specialists mentioned situations for which e-consults were inappropriate, including when a detailed history or physical examination was needed, or if a complex condition was suspected (Q6).

Background Data and Clear Recommendations

Participants described necessary data that should be included in high-quality e-consults. Specialists voiced frustration in time-consuming chart reviews that were often necessary when these data were not provided by the requestor. In some cases, specialists were unable to access necessary EHR data, which delayed responses (Q7). PCPs noted that the most useful responses carefully considered the question, used current patient information to determine treatments, provided clear recommendations, and defined who was responsible for next steps (Q8). PCPs and specialists stated that e-consult templates that required relevant information facilitated high-quality e-consults. Neither wanted to waste the other clinician's time (Q8).

A Novel Opportunity

Many PCPs felt that e-consults improved communication (eg, efficiency, response time), established new communication between clinicians, and reduced patients’ appointment burden (Q10, Q11). Many specialists felt that e-consults improved documentation of communication between clinicians and increased transparency of clinical decisions (Q12). Additionally, many specialists mentioned that e-consults capture previously informal curbside consults, enabling them to receive workload credit (Q13).

Lack of Awareness

Some noted that the biggest barrier to e-consults was not being aware of them generally, or which specialties offer e-consults (Q14). One PCP described e-consults as the best kept secret and found value in sharing the utility of e-consults with colleagues (Q15). All participants, including those who did not frequently use e-consults, felt that e-consults improved the quality of care by providing more timely care or better answers to clinical questions (Q16). Several practitioners also felt that e-consults increased access to specialty care. For example, specialists reported that e-consults enabled them to better manage patient load by using e-consults to answer relatively simple questions, reserving face-to-face consults for more complex patients (Q17).

Discussion

The objective of this study was to identify potential best practices for e-consults that may help increase their quality and use within the VA. We built on prior studies that offered insights on PCP and specialists’ overall satisfaction with e-consult by identifying several themes relevant to the further adoption of e-consults in the VA and elsewhere without a face-to-face visit.^{8,13,14,16–18} Future work may be beneficial in identifying whether the study themes identified can explain variation in e-consult use or whether addressing these factors might lead to increased or higher quality e-consult use. We are unaware of any qualitative study of comparable scale in a different health care system. Further, this is the first study to assess perspectives on e-consults among those who initiate and respond to them within the same health care system. Perhaps the most important finding from this study is that e-consults are generally viewed favorably, which is a necessary leverage point to increase their adoption within the system.

Clinicians reported several benefits to e-consults, including timely responses to clinical questions, efficient communication, allow for documentation of specialist recommendations, and help capture workload. These benefits are consistent with prior literature that indicates both PCPs and specialists in the VA and other health care systems feel that e-consults improves communication, decreases unnecessary visits, and improves quality of care.^1,14,17,18 In particular, clinicians reported that e-consults improve their practice efficiency and efficacy. This is of critical importance given the pressures of providing timely access to primary and specialty care within the VA. Interestingly, many VA practitioners were unaware which specialties offered e-consults within their facilities, reflecting previous work showing that PCPs are often unaware of e-consult options.¹⁶ This may partially explain variation in e-consult use. Increasing awareness and educating clinicians on the benefits of e-consults may help promote use among non- and low users.

A common theme reported by both groups was the importance of providing necessary information within e-consult questions and responses. Specialists felt there was a need to ensure that PCPs provide relevant and patient-specific information that would enable them to efficiently and accurately answer questions without the need for extensive EHR review. This reflects previous work showing that specialists are often unable to respond to e-consult requests because they do not contain sufficient information.²² PCPs described a need to ensure that specialists’ responses included information that was detailed enough to make clinical decisions without the need for a reconsult. This highlights a common challenge to medical consultation, in that necessary or relevant information may not be apparent to all clinicians. To address this, there may be a role in developing enhanced, flexible templating that elicits necessary patient-specific information. Such a template may automatically pull relevant data from the EHR and prompt clinicians to provide important information. We did not assess how perspectives of templates varied, and further work could help define precisely what constitutes an effective template, including how it should capture appropriate patient data and how this impacts acceptability or use of e-consults generally. Collaboratively developed service agreements and e-consult templates could help guide PCPs and specialists to engage in efficient communication.

Another theme among both groups was that e-consult is most appropriate within specific clinical scenarios. Examples included review of laboratory results, questions about medication changes, or for patients who were reluctant to travel to appointments. Identifying and promoting specific opportunities for e-consults may help increase their use and align e-consult practices with scenarios that are likely to provide the most benefit to patients. For example, it could be helpful to understand the distance patients must travel for specialty care. Providing that information during clinical encounters could trigger clinicians to consider e-consults as an option. Future work might aim to identify clinical scenarios that clinicians feel are not well suited for e-consults and determine how to adapt them for those scenarios.

Limitations

Generalizability of these findings is limited given the qualitative study design. Participants’ descriptions of experiences with e-consults reflect the experiences of clinicians in the VA and may not reflect clinicians in other settings. We also interviewed a sample of clinicians who were already using e-consults. Important information could be learned from future work with those who have not yet adopted e-consult procedures or adopted and abandoned them.

Conclusions

E-consult is perceived as beneficial by VA PCPs and specialists. Participants suggested using e-consults for appropriate questions or patients and including necessary information and next steps in both the initial e-consult and response. Finding ways to facilitate e-consults with these suggestions in mind may increase delivery of high-quality e-consults. Future work could compare the findings of this work to similar work assessing clinicians perceptions of e-consults outside of the VA.

Vitiligo is a chronic dermatologic condition that negatively affects quality of life (QOL), with substantial burden on the psychosocial well-being of patients.¹ There is no cure, and current treatment modalities are aimed at controlling the chronic relapsing condition.^1-3 Despite topical and cosmetic treatments for stabilization and repigmentation, vitiligo remains unpredictable.³

All genders, races, ethnicities, and socioeconomic classes are equally affected.⁴ The underlying etiology of vitiligo remains unknown to a great extent and is more poorly understood by the general public compared with other skin diseases (eg, acne).⁵ Patients with vitiligo experience social withdrawal, decreased sense of self-esteem, anxiety, depression, and suicidal ideation.^5,6 Stigmatization has the greatest impact on QOL, with strong correlations between avoidance behaviors and lesion concealment.^6-8 Although the condition is especially disfiguring for darker skin types, lighter skin types also are substantially affected, with similar overall self-reported stress.^6,7

Individuals with chronic illnesses such as vitiligo turn to online communities for health information and social support, commiserating with others who have the same condition.^9,10 Online forums are platforms for asynchronous peer-to-peer exchange of disease-related information for better management of long-term disease.¹¹ Moreover, of all available internet resources, online forum posts are the most commonly accessed source of information (91%) for patients following visits with their doctors.¹²

Qualitative research involving chronic skin conditions and the information exchanged in online forums has been conducted for patients with acne, psoriasis, and atopic dermatitis, but not for patients with vitiligo.^13-16 Although online questionnaires have been administered to patients with vitiligo, the content within online forums is not well characterized.^2,17

The purpose of this qualitative study was to evaluate the online content exchanged by individuals with vitiligo to better understand the general attitudes and help-seeking behaviors in online forums.

Methods

Study Design—This qualitative study sought to investigate health beliefs and messages about vitiligo posted by users in US-based online discussion forums. An interpretive research paradigm was utilized so that all content collected in online forums were the views expressed by individuals.^18-20 An integrated approach was used in the development of the coding manual, with pre-established major themes and subthemes as a guiding framework.^16,21,22 We adhered to an inductive grounded method by means of de novo line-by-line coding, such that we had flexibility for new subthemes to emerge throughout the duration of the entire coding process.²³

Individual posts and subsequent replies embedded within public online forums were used as the collected data source. Google was utilized as the primary search engine to identify forums pertaining to vitiligo, as 80% of US adults with chronic disease report that their inquiries for health information start with Google, Bing, or Yahoo.²⁴ The institutional review board at the Wake Forest School of Medicine (Winston-Salem, North Carolina) granted approval of the study (IRB00063073). Online forums were considered “property” of the public domain and were accessible to all, eliminating the need for written informed consent.^24-26

Search Criteria—We conducted our forum search in February 2020 with a systematic approach using predetermined phrases—online forum vitiligo support, vitiligo online message board, and vitiligo forums—which yielded more than 358,171 total results (eTable 1). Threads were identified in chronological order (from newest to oldest) based on how they appeared during each internet search, and all Google results for the respective search phrases were reviewed. Dates of selected threads ranged from 2005 to 2020. Only sites with US domains were included. Posts that either included views and understandings of vitiligo or belonged to a thread that contained a vitiligo discussion were deemed relevant for inclusion. Forums were excluded if registration or means of payment was required to view posts, if there were fewer than 2 user replies to a thread, if threads contained patient photographs, or if no posts had been made in the last 2 years (rendering the thread inactive). No social media platforms, such as Facebook, or formal online platforms, such as MyVitiligoTeam, were included in the search. A no-fee-for-access was chosen for this study, as the majority of those with a chronic condition who encounter a required paywall find the information elsewhere.²⁵

Data Analysis—A total of 39 online forums were deemed relevant to the topic of vitiligo; 9 of them met inclusion criteria (eTable 2). The messages within the forums were copied verbatim into a password-encrypted text document, and usernames in the threads were de-identified, ensuring user confidentiality.

An inductive thematic analysis was utilized to explore the views and beliefs of online forum users discussing vitiligo. One author (M.B.G.) read the extracted message threads, developed an initial codebook, and established a finalized version with the agreement of another author (A.M.B.)(eTable 3). The forums were independently coded (M.B.G. and A.M.B.) in a line-by-line manner according to the codebook. Discrepancies were documented and resolved. Data saturation was adequately achieved, such that no new themes emerged during the iterative coding process. NVivo was used for qualitative analysis.

Results

Nine forums met inclusion criteria, comprising 105 pages of text. There were 61 total discussion threads, with 382 anonymous contributing users. Most users initiated a thread by posting either a question, an advice statement, or a request for help. The psychosocial impact of the disease permeated multiple domains,including personal relationships and daily life. Several threads discussed treatment, including effective camouflage and makeup, as well as peer validation of physician-prescribed treatments, along with threads dedicated to “cures” or homeopathy regimens. In several instances, commercial product endorsement, testimonials, and marketing links were reposted by the same user multiple times.

Inductive thematic analysis highlighted diverse themes and subthemes related to the beliefs and perspectives of users with vitiligo or with relatives or friends with vitiligo: psychosocial impact, disease management and camouflage/concealment, alternative medicine/homeopathy/cures, interactions with the public and health care providers, and skin tone and race. Quotes from individuals were included to demonstrate themes and subthemes.

Psychosocial Impact: QOL, Sources of Support, and Coping—There was a broad range of comments on how patients cope with and view their vitiligo. Some individuals felt vitiligo made them special, and others were at peace with and accepted their condition. In contrast, others reported the disease had devastated them and interfered with relationships. Individuals shared their stories of grief and hardships through childhood and adulthood and their concerns, especially on affected visible areas or the potential for disease progression. Users were vocal about how vitiligo affected their daily routines and lives, sharing how they felt uncomfortable outside the home, no longer engaged in swimming or exposing their legs, and preferred to stay inside instead. Some users adopted a “tough love” approach to coping, sharing how they have learned to either embrace their vitiligo or “live with it.” Some examples include:

“My best advice is go with the flow, vitiligo is not the worst thing that can happen.”

“I hate my life with vitiligo yet really I feel so selfish that there is much worse suffering in the world than a few white patches.”

Other advice was very practical:

“I hope it isn’t vanity that is tearing you apart because that is only skin deep. Make a fashion statement with hats.”

Some users acknowledged and adopted the mantra that vitiligo is not a somatic condition or “physical ailment,” while others emphasized its pervasive psychological burden:

“I still deal with this psychologically . . . You must keep a positive attitude and frame of mind . . . Vitiligo will not kill you, but you do need to stay strong and keep your head up emotionally.”

“I am just really thankful that I have a disease that will not kill me or that has [not] affected me physically at all. I consider myself lucky.”

Disease Management: Treatment, Vitiligo Course, Advice-Seeking, Camouflage—The range of information discussed for treatment was highly variable. There were many accounts in which users advised others to seek professional help, namely that of a dermatologist, for a formal assessment. Many expressed frustrations with treatments and their ineffectiveness, to which the majority of users said to consult with a professional and to remain patient and hopeful/optimistic:

“The best thing to do would be to take an appointment with a dermatologist and have the discoloration checked out. That’s the only way to know whether it is vitiligo or not.”

“My way of dealing with it is to gain control by camouflage.”

“The calming effect of being in control of my vitiligo, whether with concealers, self-tan or anything else, has stopped my feelings of despair.”

Beliefs on Alternative Medicine: Homeopathy and Alternative Regimens—Although some threads started with a post asking for the best treatments, others initiated a discussion by posting “best herbal treatments for cure” or “how to cure my vitiligo,” emphasizing the beliefs and wishes for a cure for vitiligo. Alternative therapies that users endorsed included apple cider vinegar, toothpaste, vitamins, and Ayurvedic treatment, among others. Dietary plans were popular, with users claiming success with dietary alterations in stopping and preventing lesion progression. For example, individuals felt that avoidance of sugar, meat, dairy, and citrus fruits or drinks and consumption of only filtered water were crucial to preventing further lesion spread and resulted in their “cure”:

“Don’t eat chocolate, wine (made of grapes), coffee, or tea if you don’t want to have vitiligo or let it get worse. Take Vitamin B, biotin, and nuts for Vitamin E.”

Other dangerous messages pitted treatments by health professionals against beliefs in homeopathy:

“I feel that vitiligo treatment is all in your diet and vitamins. All that medicine and UV lights is a no-no . . .w ith every medicine there is a side effect. The doctors could be healing your vitiligo and severely damaging you inside and out, and you won’t know until years later.”

There was a minor presence of users advising against homeopathy and the associated misinformation and inaccurate claims on curing vitiligo, though this group was small in comparison to the number of users posting outlandish claims on cure:

“There is no cure . . . It’s where your immune system attacks your skin cells causing loss of pigmentation. The skin that has lost the pigmentation can’t be reversed.”

Interactions With the Public and Health Care Providers—Those with vitiligo encounter unique situations in public and in their daily lives. Many of the accounts shared anecdotal stories on how patients have handled the stigma and discrimination faced:

“I have had to face discrimination at school, public places, college, functions, and every new person I have met has asked me this: ‘how did this happen?’”

Those with vitiligo even stated how they wished others would deal with their condition out in public, hoping that others would directly ask what the lesions were instead of the more hurtful staring. There were many stories in which users said others feel vitiligo was contagious or “dirty” and stressed that the condition is not infectious:

“I refer to myself as ‘camo-man’ and reassure people I come into contact with that it is not contagious.”

“Once I was eating at a restaurant . . . and a little girl said to her mom, ‘Look, Mom, that lady doesn’t wash her arms, look how dirty they are.’ That just broke my heart.”

Skin Tone and Implications—The belief that vitiligo lesions are less dramatic or less anxiety provoking for individuals with lighter skin was noted by users themselves and by health care providers in certain cases. Skin tone and its impact on QOL was confusing and contentious. Some users with fair skin stated their vitiligo was “less of an annoyance” or “less obvious” compared with individuals with darker complexions. Conversely, other accounts of self-reported White users vehemently stressed the anxieties felt by depigmented lesions, despite being “already white at baseline.”

“Was told by my dermatologist (upon diagnosis) that ‘You’re lucky you’re not African American—it shows up on them much worse. You’re so fair, it doesn’t really matter.’ ”

“You didn’t say what race you are. I could imagine it has a bigger impact if you are anything other than White.”

Comment

Patients Looking for Cures—The general attitude within the forums was uplifting and encouraging, with users detailing how they respond to others in public and sharing their personal perspectives. We found a mix of information regarding disease management and treatment of vitiligo. Overall, there was uncertainty about treatments, with individuals expressing concern that their treatments were ineffective or had failed or that better alternatives would be more suitable for their condition. We found many anecdotal endorsements of homeopathic remedies for vitiligo, with users boasting that their disease had not only been cured but had never returned. Some users completely denounced these statements, while other threads seemed to revolve completely around “cure” discussions with no dissenting voices. The number of discussions related to homeopathy was concerning. Furthermore, there often were no moderators within threads to remove cure-related content, whether commercially endorsed or anecdotal. It is plausible that supplements and vitamins recommended by some physicians may be incorrectly interpreted as a “cure” in online discussions. Our findings are consistent with prior reports that forums are a platform to express dissatisfaction with treatment and the need for additional treatment options.^15,22

Concern Expressed by Health Care Providers—Prior qualitative research has described how patients with chronic dermatologic conditions believe that health care providers minimize patients’ psychological distress.^27,28 We found several accounts in which an individual had explicitly stated their provider had “belittled” the extent and impact of vitiligo when comparing skin phototypes. This suggests either that physicians underestimate the impact of vitiligo on their patients or that physicians are not expressing enough empathic concern about the impact the condition has on those affected.

Cosmetic Aspects of Vitiligo—Few clinical trials have investigated QOL and cosmetic acceptability of treatments as outcome measures.²⁹ We found several instances in which users with vitiligo had reported being dismissed as having a “cosmetic disease,” consistent with other work demonstrating the negative impact on such dismissals.²² Moreover, concealment and camouflage techniques frequently were discussed, demonstrating the relevance of cosmetic management as an important research topic.

Trustworthy Sources of Health Information—Patients still view physicians as trustworthy and a key source of health care information and advice.^30-32 Patients with vitiligo who have been directed to reliable information sources often express gratitude²² and want health professionals to remain an important source in their health information-seeking.³¹ Given the range in information discussed online, it may be valuable to invite patients to share what information they have encountered online.

Our study highlights the conflicting health information and advice shared by users in online forums, complicating an already psychologically burdensome condition. Guiding patients to credible, moderated sites and resources that are accurate, understandable, and easy to access may help dispel the conflicting messages and stories discussed in the online community.

Study Strengths and Limitations—Limitations included reporting bias and reliance on self-reported information on the diagnosis and extent of individuals’ vitiligo. Excluding social media websites and platforms from the data collection is a limitation to comprehensively assessing the topic of internet users with vitiligo. Many social media platforms direct patients and their family members to support groups and therefore may have excluded these particular individuals. Social media platforms were excluded from our research owing to the prerequisite of creating user accounts or registering as an online member. Our inclusion criteria were specific to forums that did not require registering or creating an account and were therefore freely accessible to all internet viewers. There is an inherent lack of context present in online forums, preventing data collection on individuals’ demographics and socioeconomic backgrounds. However, anonymity may have allowed individuals to express their thoughts more freely.

An integrated approach, along with our sampling method of online forums not requiring registration, allows for greater transferability and understanding of the health needs of the general public with vitiligo.

Conclusion

Individuals with vitiligo continue to seek peer psychosocial support for the physical and emotional management of their disease. Counseling those with vitiligo about cosmetic concealment options, homeopathy, and treatment scams remains paramount. Directing patients to evidence-based resources, along with providing structured sources of support, may help to improve the psychosocial burden and QOL experienced by patients with vitiligo. Connecting patients with local and national support groups moderated by physicians, such as the Global Vitiligo Foundation (https://globalvitiligofoundation.org/), may provide benefit to patients with vitiligo.

Vitiligo is a chronic dermatologic condition that negatively affects quality of life (QOL), with substantial burden on the psychosocial well-being of patients.¹ There is no cure, and current treatment modalities are aimed at controlling the chronic relapsing condition.^1-3 Despite topical and cosmetic treatments for stabilization and repigmentation, vitiligo remains unpredictable.³

All genders, races, ethnicities, and socioeconomic classes are equally affected.⁴ The underlying etiology of vitiligo remains unknown to a great extent and is more poorly understood by the general public compared with other skin diseases (eg, acne).⁵ Patients with vitiligo experience social withdrawal, decreased sense of self-esteem, anxiety, depression, and suicidal ideation.^5,6 Stigmatization has the greatest impact on QOL, with strong correlations between avoidance behaviors and lesion concealment.^6-8 Although the condition is especially disfiguring for darker skin types, lighter skin types also are substantially affected, with similar overall self-reported stress.^6,7

Individuals with chronic illnesses such as vitiligo turn to online communities for health information and social support, commiserating with others who have the same condition.^9,10 Online forums are platforms for asynchronous peer-to-peer exchange of disease-related information for better management of long-term disease.¹¹ Moreover, of all available internet resources, online forum posts are the most commonly accessed source of information (91%) for patients following visits with their doctors.¹²

Qualitative research involving chronic skin conditions and the information exchanged in online forums has been conducted for patients with acne, psoriasis, and atopic dermatitis, but not for patients with vitiligo.^13-16 Although online questionnaires have been administered to patients with vitiligo, the content within online forums is not well characterized.^2,17

The purpose of this qualitative study was to evaluate the online content exchanged by individuals with vitiligo to better understand the general attitudes and help-seeking behaviors in online forums.

Methods

Study Design—This qualitative study sought to investigate health beliefs and messages about vitiligo posted by users in US-based online discussion forums. An interpretive research paradigm was utilized so that all content collected in online forums were the views expressed by individuals.^18-20 An integrated approach was used in the development of the coding manual, with pre-established major themes and subthemes as a guiding framework.^16,21,22 We adhered to an inductive grounded method by means of de novo line-by-line coding, such that we had flexibility for new subthemes to emerge throughout the duration of the entire coding process.²³

Individual posts and subsequent replies embedded within public online forums were used as the collected data source. Google was utilized as the primary search engine to identify forums pertaining to vitiligo, as 80% of US adults with chronic disease report that their inquiries for health information start with Google, Bing, or Yahoo.²⁴ The institutional review board at the Wake Forest School of Medicine (Winston-Salem, North Carolina) granted approval of the study (IRB00063073). Online forums were considered “property” of the public domain and were accessible to all, eliminating the need for written informed consent.^24-26

Search Criteria—We conducted our forum search in February 2020 with a systematic approach using predetermined phrases—online forum vitiligo support, vitiligo online message board, and vitiligo forums—which yielded more than 358,171 total results (eTable 1). Threads were identified in chronological order (from newest to oldest) based on how they appeared during each internet search, and all Google results for the respective search phrases were reviewed. Dates of selected threads ranged from 2005 to 2020. Only sites with US domains were included. Posts that either included views and understandings of vitiligo or belonged to a thread that contained a vitiligo discussion were deemed relevant for inclusion. Forums were excluded if registration or means of payment was required to view posts, if there were fewer than 2 user replies to a thread, if threads contained patient photographs, or if no posts had been made in the last 2 years (rendering the thread inactive). No social media platforms, such as Facebook, or formal online platforms, such as MyVitiligoTeam, were included in the search. A no-fee-for-access was chosen for this study, as the majority of those with a chronic condition who encounter a required paywall find the information elsewhere.²⁵

Data Analysis—A total of 39 online forums were deemed relevant to the topic of vitiligo; 9 of them met inclusion criteria (eTable 2). The messages within the forums were copied verbatim into a password-encrypted text document, and usernames in the threads were de-identified, ensuring user confidentiality.

An inductive thematic analysis was utilized to explore the views and beliefs of online forum users discussing vitiligo. One author (M.B.G.) read the extracted message threads, developed an initial codebook, and established a finalized version with the agreement of another author (A.M.B.)(eTable 3). The forums were independently coded (M.B.G. and A.M.B.) in a line-by-line manner according to the codebook. Discrepancies were documented and resolved. Data saturation was adequately achieved, such that no new themes emerged during the iterative coding process. NVivo was used for qualitative analysis.

Results

Nine forums met inclusion criteria, comprising 105 pages of text. There were 61 total discussion threads, with 382 anonymous contributing users. Most users initiated a thread by posting either a question, an advice statement, or a request for help. The psychosocial impact of the disease permeated multiple domains,including personal relationships and daily life. Several threads discussed treatment, including effective camouflage and makeup, as well as peer validation of physician-prescribed treatments, along with threads dedicated to “cures” or homeopathy regimens. In several instances, commercial product endorsement, testimonials, and marketing links were reposted by the same user multiple times.

Inductive thematic analysis highlighted diverse themes and subthemes related to the beliefs and perspectives of users with vitiligo or with relatives or friends with vitiligo: psychosocial impact, disease management and camouflage/concealment, alternative medicine/homeopathy/cures, interactions with the public and health care providers, and skin tone and race. Quotes from individuals were included to demonstrate themes and subthemes.

Psychosocial Impact: QOL, Sources of Support, and Coping—There was a broad range of comments on how patients cope with and view their vitiligo. Some individuals felt vitiligo made them special, and others were at peace with and accepted their condition. In contrast, others reported the disease had devastated them and interfered with relationships. Individuals shared their stories of grief and hardships through childhood and adulthood and their concerns, especially on affected visible areas or the potential for disease progression. Users were vocal about how vitiligo affected their daily routines and lives, sharing how they felt uncomfortable outside the home, no longer engaged in swimming or exposing their legs, and preferred to stay inside instead. Some users adopted a “tough love” approach to coping, sharing how they have learned to either embrace their vitiligo or “live with it.” Some examples include:

“My best advice is go with the flow, vitiligo is not the worst thing that can happen.”

“I hate my life with vitiligo yet really I feel so selfish that there is much worse suffering in the world than a few white patches.”

Other advice was very practical:

“I hope it isn’t vanity that is tearing you apart because that is only skin deep. Make a fashion statement with hats.”

Some users acknowledged and adopted the mantra that vitiligo is not a somatic condition or “physical ailment,” while others emphasized its pervasive psychological burden:

“I still deal with this psychologically . . . You must keep a positive attitude and frame of mind . . . Vitiligo will not kill you, but you do need to stay strong and keep your head up emotionally.”

“I am just really thankful that I have a disease that will not kill me or that has [not] affected me physically at all. I consider myself lucky.”

Disease Management: Treatment, Vitiligo Course, Advice-Seeking, Camouflage—The range of information discussed for treatment was highly variable. There were many accounts in which users advised others to seek professional help, namely that of a dermatologist, for a formal assessment. Many expressed frustrations with treatments and their ineffectiveness, to which the majority of users said to consult with a professional and to remain patient and hopeful/optimistic:

“The best thing to do would be to take an appointment with a dermatologist and have the discoloration checked out. That’s the only way to know whether it is vitiligo or not.”

“My way of dealing with it is to gain control by camouflage.”

“The calming effect of being in control of my vitiligo, whether with concealers, self-tan or anything else, has stopped my feelings of despair.”

Beliefs on Alternative Medicine: Homeopathy and Alternative Regimens—Although some threads started with a post asking for the best treatments, others initiated a discussion by posting “best herbal treatments for cure” or “how to cure my vitiligo,” emphasizing the beliefs and wishes for a cure for vitiligo. Alternative therapies that users endorsed included apple cider vinegar, toothpaste, vitamins, and Ayurvedic treatment, among others. Dietary plans were popular, with users claiming success with dietary alterations in stopping and preventing lesion progression. For example, individuals felt that avoidance of sugar, meat, dairy, and citrus fruits or drinks and consumption of only filtered water were crucial to preventing further lesion spread and resulted in their “cure”:

“Don’t eat chocolate, wine (made of grapes), coffee, or tea if you don’t want to have vitiligo or let it get worse. Take Vitamin B, biotin, and nuts for Vitamin E.”

Other dangerous messages pitted treatments by health professionals against beliefs in homeopathy:

“I feel that vitiligo treatment is all in your diet and vitamins. All that medicine and UV lights is a no-no . . .w ith every medicine there is a side effect. The doctors could be healing your vitiligo and severely damaging you inside and out, and you won’t know until years later.”

There was a minor presence of users advising against homeopathy and the associated misinformation and inaccurate claims on curing vitiligo, though this group was small in comparison to the number of users posting outlandish claims on cure:

“There is no cure . . . It’s where your immune system attacks your skin cells causing loss of pigmentation. The skin that has lost the pigmentation can’t be reversed.”

Interactions With the Public and Health Care Providers—Those with vitiligo encounter unique situations in public and in their daily lives. Many of the accounts shared anecdotal stories on how patients have handled the stigma and discrimination faced:

“I have had to face discrimination at school, public places, college, functions, and every new person I have met has asked me this: ‘how did this happen?’”

Those with vitiligo even stated how they wished others would deal with their condition out in public, hoping that others would directly ask what the lesions were instead of the more hurtful staring. There were many stories in which users said others feel vitiligo was contagious or “dirty” and stressed that the condition is not infectious:

“I refer to myself as ‘camo-man’ and reassure people I come into contact with that it is not contagious.”

“Once I was eating at a restaurant . . . and a little girl said to her mom, ‘Look, Mom, that lady doesn’t wash her arms, look how dirty they are.’ That just broke my heart.”

Skin Tone and Implications—The belief that vitiligo lesions are less dramatic or less anxiety provoking for individuals with lighter skin was noted by users themselves and by health care providers in certain cases. Skin tone and its impact on QOL was confusing and contentious. Some users with fair skin stated their vitiligo was “less of an annoyance” or “less obvious” compared with individuals with darker complexions. Conversely, other accounts of self-reported White users vehemently stressed the anxieties felt by depigmented lesions, despite being “already white at baseline.”

“Was told by my dermatologist (upon diagnosis) that ‘You’re lucky you’re not African American—it shows up on them much worse. You’re so fair, it doesn’t really matter.’ ”

“You didn’t say what race you are. I could imagine it has a bigger impact if you are anything other than White.”

Comment

Patients Looking for Cures—The general attitude within the forums was uplifting and encouraging, with users detailing how they respond to others in public and sharing their personal perspectives. We found a mix of information regarding disease management and treatment of vitiligo. Overall, there was uncertainty about treatments, with individuals expressing concern that their treatments were ineffective or had failed or that better alternatives would be more suitable for their condition. We found many anecdotal endorsements of homeopathic remedies for vitiligo, with users boasting that their disease had not only been cured but had never returned. Some users completely denounced these statements, while other threads seemed to revolve completely around “cure” discussions with no dissenting voices. The number of discussions related to homeopathy was concerning. Furthermore, there often were no moderators within threads to remove cure-related content, whether commercially endorsed or anecdotal. It is plausible that supplements and vitamins recommended by some physicians may be incorrectly interpreted as a “cure” in online discussions. Our findings are consistent with prior reports that forums are a platform to express dissatisfaction with treatment and the need for additional treatment options.^15,22

Concern Expressed by Health Care Providers—Prior qualitative research has described how patients with chronic dermatologic conditions believe that health care providers minimize patients’ psychological distress.^27,28 We found several accounts in which an individual had explicitly stated their provider had “belittled” the extent and impact of vitiligo when comparing skin phototypes. This suggests either that physicians underestimate the impact of vitiligo on their patients or that physicians are not expressing enough empathic concern about the impact the condition has on those affected.

Cosmetic Aspects of Vitiligo—Few clinical trials have investigated QOL and cosmetic acceptability of treatments as outcome measures.²⁹ We found several instances in which users with vitiligo had reported being dismissed as having a “cosmetic disease,” consistent with other work demonstrating the negative impact on such dismissals.²² Moreover, concealment and camouflage techniques frequently were discussed, demonstrating the relevance of cosmetic management as an important research topic.

Trustworthy Sources of Health Information—Patients still view physicians as trustworthy and a key source of health care information and advice.^30-32 Patients with vitiligo who have been directed to reliable information sources often express gratitude²² and want health professionals to remain an important source in their health information-seeking.³¹ Given the range in information discussed online, it may be valuable to invite patients to share what information they have encountered online.

Our study highlights the conflicting health information and advice shared by users in online forums, complicating an already psychologically burdensome condition. Guiding patients to credible, moderated sites and resources that are accurate, understandable, and easy to access may help dispel the conflicting messages and stories discussed in the online community.

Study Strengths and Limitations—Limitations included reporting bias and reliance on self-reported information on the diagnosis and extent of individuals’ vitiligo. Excluding social media websites and platforms from the data collection is a limitation to comprehensively assessing the topic of internet users with vitiligo. Many social media platforms direct patients and their family members to support groups and therefore may have excluded these particular individuals. Social media platforms were excluded from our research owing to the prerequisite of creating user accounts or registering as an online member. Our inclusion criteria were specific to forums that did not require registering or creating an account and were therefore freely accessible to all internet viewers. There is an inherent lack of context present in online forums, preventing data collection on individuals’ demographics and socioeconomic backgrounds. However, anonymity may have allowed individuals to express their thoughts more freely.

An integrated approach, along with our sampling method of online forums not requiring registration, allows for greater transferability and understanding of the health needs of the general public with vitiligo.

Conclusion

Individuals with vitiligo continue to seek peer psychosocial support for the physical and emotional management of their disease. Counseling those with vitiligo about cosmetic concealment options, homeopathy, and treatment scams remains paramount. Directing patients to evidence-based resources, along with providing structured sources of support, may help to improve the psychosocial burden and QOL experienced by patients with vitiligo. Connecting patients with local and national support groups moderated by physicians, such as the Global Vitiligo Foundation (https://globalvitiligofoundation.org/), may provide benefit to patients with vitiligo.

Vitiligo is a chronic dermatologic condition that negatively affects quality of life (QOL), with substantial burden on the psychosocial well-being of patients.¹ There is no cure, and current treatment modalities are aimed at controlling the chronic relapsing condition.^1-3 Despite topical and cosmetic treatments for stabilization and repigmentation, vitiligo remains unpredictable.³

All genders, races, ethnicities, and socioeconomic classes are equally affected.⁴ The underlying etiology of vitiligo remains unknown to a great extent and is more poorly understood by the general public compared with other skin diseases (eg, acne).⁵ Patients with vitiligo experience social withdrawal, decreased sense of self-esteem, anxiety, depression, and suicidal ideation.^5,6 Stigmatization has the greatest impact on QOL, with strong correlations between avoidance behaviors and lesion concealment.^6-8 Although the condition is especially disfiguring for darker skin types, lighter skin types also are substantially affected, with similar overall self-reported stress.^6,7

Individuals with chronic illnesses such as vitiligo turn to online communities for health information and social support, commiserating with others who have the same condition.^9,10 Online forums are platforms for asynchronous peer-to-peer exchange of disease-related information for better management of long-term disease.¹¹ Moreover, of all available internet resources, online forum posts are the most commonly accessed source of information (91%) for patients following visits with their doctors.¹²

Qualitative research involving chronic skin conditions and the information exchanged in online forums has been conducted for patients with acne, psoriasis, and atopic dermatitis, but not for patients with vitiligo.^13-16 Although online questionnaires have been administered to patients with vitiligo, the content within online forums is not well characterized.^2,17

The purpose of this qualitative study was to evaluate the online content exchanged by individuals with vitiligo to better understand the general attitudes and help-seeking behaviors in online forums.

Methods

Study Design—This qualitative study sought to investigate health beliefs and messages about vitiligo posted by users in US-based online discussion forums. An interpretive research paradigm was utilized so that all content collected in online forums were the views expressed by individuals.^18-20 An integrated approach was used in the development of the coding manual, with pre-established major themes and subthemes as a guiding framework.^16,21,22 We adhered to an inductive grounded method by means of de novo line-by-line coding, such that we had flexibility for new subthemes to emerge throughout the duration of the entire coding process.²³

Individual posts and subsequent replies embedded within public online forums were used as the collected data source. Google was utilized as the primary search engine to identify forums pertaining to vitiligo, as 80% of US adults with chronic disease report that their inquiries for health information start with Google, Bing, or Yahoo.²⁴ The institutional review board at the Wake Forest School of Medicine (Winston-Salem, North Carolina) granted approval of the study (IRB00063073). Online forums were considered “property” of the public domain and were accessible to all, eliminating the need for written informed consent.^24-26

Search Criteria—We conducted our forum search in February 2020 with a systematic approach using predetermined phrases—online forum vitiligo support, vitiligo online message board, and vitiligo forums—which yielded more than 358,171 total results (eTable 1). Threads were identified in chronological order (from newest to oldest) based on how they appeared during each internet search, and all Google results for the respective search phrases were reviewed. Dates of selected threads ranged from 2005 to 2020. Only sites with US domains were included. Posts that either included views and understandings of vitiligo or belonged to a thread that contained a vitiligo discussion were deemed relevant for inclusion. Forums were excluded if registration or means of payment was required to view posts, if there were fewer than 2 user replies to a thread, if threads contained patient photographs, or if no posts had been made in the last 2 years (rendering the thread inactive). No social media platforms, such as Facebook, or formal online platforms, such as MyVitiligoTeam, were included in the search. A no-fee-for-access was chosen for this study, as the majority of those with a chronic condition who encounter a required paywall find the information elsewhere.²⁵

Data Analysis—A total of 39 online forums were deemed relevant to the topic of vitiligo; 9 of them met inclusion criteria (eTable 2). The messages within the forums were copied verbatim into a password-encrypted text document, and usernames in the threads were de-identified, ensuring user confidentiality.

An inductive thematic analysis was utilized to explore the views and beliefs of online forum users discussing vitiligo. One author (M.B.G.) read the extracted message threads, developed an initial codebook, and established a finalized version with the agreement of another author (A.M.B.)(eTable 3). The forums were independently coded (M.B.G. and A.M.B.) in a line-by-line manner according to the codebook. Discrepancies were documented and resolved. Data saturation was adequately achieved, such that no new themes emerged during the iterative coding process. NVivo was used for qualitative analysis.

Results

Nine forums met inclusion criteria, comprising 105 pages of text. There were 61 total discussion threads, with 382 anonymous contributing users. Most users initiated a thread by posting either a question, an advice statement, or a request for help. The psychosocial impact of the disease permeated multiple domains,including personal relationships and daily life. Several threads discussed treatment, including effective camouflage and makeup, as well as peer validation of physician-prescribed treatments, along with threads dedicated to “cures” or homeopathy regimens. In several instances, commercial product endorsement, testimonials, and marketing links were reposted by the same user multiple times.

Inductive thematic analysis highlighted diverse themes and subthemes related to the beliefs and perspectives of users with vitiligo or with relatives or friends with vitiligo: psychosocial impact, disease management and camouflage/concealment, alternative medicine/homeopathy/cures, interactions with the public and health care providers, and skin tone and race. Quotes from individuals were included to demonstrate themes and subthemes.

Psychosocial Impact: QOL, Sources of Support, and Coping—There was a broad range of comments on how patients cope with and view their vitiligo. Some individuals felt vitiligo made them special, and others were at peace with and accepted their condition. In contrast, others reported the disease had devastated them and interfered with relationships. Individuals shared their stories of grief and hardships through childhood and adulthood and their concerns, especially on affected visible areas or the potential for disease progression. Users were vocal about how vitiligo affected their daily routines and lives, sharing how they felt uncomfortable outside the home, no longer engaged in swimming or exposing their legs, and preferred to stay inside instead. Some users adopted a “tough love” approach to coping, sharing how they have learned to either embrace their vitiligo or “live with it.” Some examples include:

“My best advice is go with the flow, vitiligo is not the worst thing that can happen.”

“I hate my life with vitiligo yet really I feel so selfish that there is much worse suffering in the world than a few white patches.”

Other advice was very practical:

“I hope it isn’t vanity that is tearing you apart because that is only skin deep. Make a fashion statement with hats.”

Some users acknowledged and adopted the mantra that vitiligo is not a somatic condition or “physical ailment,” while others emphasized its pervasive psychological burden:

“I still deal with this psychologically . . . You must keep a positive attitude and frame of mind . . . Vitiligo will not kill you, but you do need to stay strong and keep your head up emotionally.”

“I am just really thankful that I have a disease that will not kill me or that has [not] affected me physically at all. I consider myself lucky.”

Disease Management: Treatment, Vitiligo Course, Advice-Seeking, Camouflage—The range of information discussed for treatment was highly variable. There were many accounts in which users advised others to seek professional help, namely that of a dermatologist, for a formal assessment. Many expressed frustrations with treatments and their ineffectiveness, to which the majority of users said to consult with a professional and to remain patient and hopeful/optimistic:

“The best thing to do would be to take an appointment with a dermatologist and have the discoloration checked out. That’s the only way to know whether it is vitiligo or not.”

“My way of dealing with it is to gain control by camouflage.”

“The calming effect of being in control of my vitiligo, whether with concealers, self-tan or anything else, has stopped my feelings of despair.”

Beliefs on Alternative Medicine: Homeopathy and Alternative Regimens—Although some threads started with a post asking for the best treatments, others initiated a discussion by posting “best herbal treatments for cure” or “how to cure my vitiligo,” emphasizing the beliefs and wishes for a cure for vitiligo. Alternative therapies that users endorsed included apple cider vinegar, toothpaste, vitamins, and Ayurvedic treatment, among others. Dietary plans were popular, with users claiming success with dietary alterations in stopping and preventing lesion progression. For example, individuals felt that avoidance of sugar, meat, dairy, and citrus fruits or drinks and consumption of only filtered water were crucial to preventing further lesion spread and resulted in their “cure”:

“Don’t eat chocolate, wine (made of grapes), coffee, or tea if you don’t want to have vitiligo or let it get worse. Take Vitamin B, biotin, and nuts for Vitamin E.”

Other dangerous messages pitted treatments by health professionals against beliefs in homeopathy:

“I feel that vitiligo treatment is all in your diet and vitamins. All that medicine and UV lights is a no-no . . .w ith every medicine there is a side effect. The doctors could be healing your vitiligo and severely damaging you inside and out, and you won’t know until years later.”

There was a minor presence of users advising against homeopathy and the associated misinformation and inaccurate claims on curing vitiligo, though this group was small in comparison to the number of users posting outlandish claims on cure:

“There is no cure . . . It’s where your immune system attacks your skin cells causing loss of pigmentation. The skin that has lost the pigmentation can’t be reversed.”

Interactions With the Public and Health Care Providers—Those with vitiligo encounter unique situations in public and in their daily lives. Many of the accounts shared anecdotal stories on how patients have handled the stigma and discrimination faced:

“I have had to face discrimination at school, public places, college, functions, and every new person I have met has asked me this: ‘how did this happen?’”

Those with vitiligo even stated how they wished others would deal with their condition out in public, hoping that others would directly ask what the lesions were instead of the more hurtful staring. There were many stories in which users said others feel vitiligo was contagious or “dirty” and stressed that the condition is not infectious:

“I refer to myself as ‘camo-man’ and reassure people I come into contact with that it is not contagious.”

“Once I was eating at a restaurant . . . and a little girl said to her mom, ‘Look, Mom, that lady doesn’t wash her arms, look how dirty they are.’ That just broke my heart.”

Skin Tone and Implications—The belief that vitiligo lesions are less dramatic or less anxiety provoking for individuals with lighter skin was noted by users themselves and by health care providers in certain cases. Skin tone and its impact on QOL was confusing and contentious. Some users with fair skin stated their vitiligo was “less of an annoyance” or “less obvious” compared with individuals with darker complexions. Conversely, other accounts of self-reported White users vehemently stressed the anxieties felt by depigmented lesions, despite being “already white at baseline.”

“Was told by my dermatologist (upon diagnosis) that ‘You’re lucky you’re not African American—it shows up on them much worse. You’re so fair, it doesn’t really matter.’ ”

“You didn’t say what race you are. I could imagine it has a bigger impact if you are anything other than White.”

Comment

Patients Looking for Cures—The general attitude within the forums was uplifting and encouraging, with users detailing how they respond to others in public and sharing their personal perspectives. We found a mix of information regarding disease management and treatment of vitiligo. Overall, there was uncertainty about treatments, with individuals expressing concern that their treatments were ineffective or had failed or that better alternatives would be more suitable for their condition. We found many anecdotal endorsements of homeopathic remedies for vitiligo, with users boasting that their disease had not only been cured but had never returned. Some users completely denounced these statements, while other threads seemed to revolve completely around “cure” discussions with no dissenting voices. The number of discussions related to homeopathy was concerning. Furthermore, there often were no moderators within threads to remove cure-related content, whether commercially endorsed or anecdotal. It is plausible that supplements and vitamins recommended by some physicians may be incorrectly interpreted as a “cure” in online discussions. Our findings are consistent with prior reports that forums are a platform to express dissatisfaction with treatment and the need for additional treatment options.^15,22

Concern Expressed by Health Care Providers—Prior qualitative research has described how patients with chronic dermatologic conditions believe that health care providers minimize patients’ psychological distress.^27,28 We found several accounts in which an individual had explicitly stated their provider had “belittled” the extent and impact of vitiligo when comparing skin phototypes. This suggests either that physicians underestimate the impact of vitiligo on their patients or that physicians are not expressing enough empathic concern about the impact the condition has on those affected.

Cosmetic Aspects of Vitiligo—Few clinical trials have investigated QOL and cosmetic acceptability of treatments as outcome measures.²⁹ We found several instances in which users with vitiligo had reported being dismissed as having a “cosmetic disease,” consistent with other work demonstrating the negative impact on such dismissals.²² Moreover, concealment and camouflage techniques frequently were discussed, demonstrating the relevance of cosmetic management as an important research topic.

Trustworthy Sources of Health Information—Patients still view physicians as trustworthy and a key source of health care information and advice.^30-32 Patients with vitiligo who have been directed to reliable information sources often express gratitude²² and want health professionals to remain an important source in their health information-seeking.³¹ Given the range in information discussed online, it may be valuable to invite patients to share what information they have encountered online.

Our study highlights the conflicting health information and advice shared by users in online forums, complicating an already psychologically burdensome condition. Guiding patients to credible, moderated sites and resources that are accurate, understandable, and easy to access may help dispel the conflicting messages and stories discussed in the online community.

Study Strengths and Limitations—Limitations included reporting bias and reliance on self-reported information on the diagnosis and extent of individuals’ vitiligo. Excluding social media websites and platforms from the data collection is a limitation to comprehensively assessing the topic of internet users with vitiligo. Many social media platforms direct patients and their family members to support groups and therefore may have excluded these particular individuals. Social media platforms were excluded from our research owing to the prerequisite of creating user accounts or registering as an online member. Our inclusion criteria were specific to forums that did not require registering or creating an account and were therefore freely accessible to all internet viewers. There is an inherent lack of context present in online forums, preventing data collection on individuals’ demographics and socioeconomic backgrounds. However, anonymity may have allowed individuals to express their thoughts more freely.

An integrated approach, along with our sampling method of online forums not requiring registration, allows for greater transferability and understanding of the health needs of the general public with vitiligo.

Conclusion

Individuals with vitiligo continue to seek peer psychosocial support for the physical and emotional management of their disease. Counseling those with vitiligo about cosmetic concealment options, homeopathy, and treatment scams remains paramount. Directing patients to evidence-based resources, along with providing structured sources of support, may help to improve the psychosocial burden and QOL experienced by patients with vitiligo. Connecting patients with local and national support groups moderated by physicians, such as the Global Vitiligo Foundation (https://globalvitiligofoundation.org/), may provide benefit to patients with vitiligo.

Medication nonadherence is common with oral antipsychotic formulations, resulting in relapse, increased morbidity, and more frequent psychiatric hospitalization.^1-7 Psychiatric hospitalization and illness decompensation is costly to health care systems and leads to reduced quality of life for veterans and families.^6,7 Long-acting injectable antipsychotics (LAIAs) were developed to enhance antipsychotic adherence and improve patient outcomes, including reduced psychiatric hospitalization.^8-12

Little outcomes data exist comparing LAIAs, including biweekly risperidone microspheres and monthly paliperidone palmitate.^10-13 Risperidone microspheres require a 3-week oral crossover and are administered every 2 weeks, whereas paliperidone palmitate does not require an oral crossover and is administered every 4 weeks. The paliperidone palmitate loading regimen replaces an oral crossover.

The primary objective of this study was to compare the number of psychiatric hospitalizations between veterans administered risperidone microspheres and those on paliperidone palmitate pre- and post-LAIA initiation. Secondary objectives were to assess rehospitalization rates between patients taking risperidone microspheres and paliperidone palmitate, reduction in pre- and posthospitalization rates with LAIAs, and medication adherence.

Methods

This observational study with a retrospective cohort design was conducted at the Veterans Affairs Loma Linda Healthcare System (VALLHS) in California. We examined veterans who were initiated on LAIAs risperidone microspheres or paliperidone palmitate from January 01, 2016 through December 31, 2018. Veterans who were aged ≥ 18 years and received ≥ 2 injections of either risperidone microspheres or paliperidone palmitate during the study period were included. Veterans were excluded if they had received < 2 doses of either LAIA, received the LAIA outside of the review period, were nonadherent to risperidone crossover if they received risperidone microspheres, or transferred their care to another facility. At VALLHS, LAIA injections are administered by a nurse, and veterans must travel to the facility to receive the injections.

Extracted patient chart elements included participant demographics; diagnoses; comorbid alcohol, nicotine, opioid, or other substance use; duration on LAIA; psychiatric hospitalizations pre- and postinitiation of the LAIA; medication adherence; and medication discontinuation based on clinician documentation and clinic orders (Table 1).

Data Analysis

Patient demographics were analyzed using descriptive statistics and experimental comparisons between the risperidone microspheres and paliperidone palmitate groups to assess baseline differences between groups. Psychiatric hospitalizations pre- and post-LAIA were analyzed with parallel group (between veterans–independent groups) and pre-post (within veterans–dependent groups) designs. Index hospitalizations were examined for a period equivalent to the length of time veterans were on the LAIA. Psychiatric rehospitalization rates were analyzed for patients who had index hospitalizations and were rehospitalized for any period when they were receiving the LAIA. Incidences of pre- and post-LAIA hospitalizations were calculated in 100 person-years.

Parallel-group analysis was analyzed using the χ² and Mann-Whitney U tests. Pre-post analyses were analyzed using the Wilcoxon rank sum test. P was set at < .05 for statistical significance.

Results

We screened 111 veterans, and 97 were included in this study (risperidone microspheres, 44; paliperidone palmitate, 53). Mean (SD) age was 46 (13.8) years, 92% were male, 38% were White, 94% were diagnosed with schizophrenia or schizoaffective disorder, and 11% were homeless. Substance use was documented as 52% for nicotine products, 40% for alcohol, 31% for cannabis, 27% for methamphetamine, 7% for cocaine, and 3% for opioids. Cannabis, methamphetamine, cocaine, and opioid use were based on clinician documentation and listed as active diagnoses at the time of LAIA initiation. Statistical significance was found in index hospitalizations P = .009) and history of cocaine use disorder (6.8% vs 7.5%, P < .001).

Veterans administered risperidone microspheres had fewer mean (SD) post-LAIA hospitalizations (0.4 [1.0] vs 0.9 [1.5]; P = .02) and were less likely to be rehospitalized (22.7% vs 47.2%, P = .01) compared with paliperidone palmitate. However, veterans taking risperidone microspheres had a shorter mean (SD) treatment duration (41.6 [40.2] vs 58.2 [45.7] weeks, P = .04) compared with paliperidone palmitate, mainly because patients switched to a different LAIA or oral antipsychotic. No differences were detected in nonadherence and discontinuation between risperidone microspheres and paliperidone palmitate. All veterans in the risperidone microspheres group adhered to oral risperidone crossover with an average 87.8% days covered (Table 2).

Discussion

The main finding of this study was that initiation of LAIAs significantly reduced hospitalizations. Veterans taking risperidone microspheres had higher index hospitalizations and lower posttreatment hospitalizations compared with paliperidone palmitate. We found that patients initiated on risperidone microspheres had more hospitalizations before use of a LAIA than those initiated on paliperidone palmitate. Risperidone microspheres reduced the number of hospitalization post-LAIA significantly more than paliperidone palmitate. We also found that veterans taking risperidone microspheres were on the medication for less mean (SD) time than those on paliperidone palmitate (41.6 [40.2] vs 58.2 [45.7] weeks; P = .04).

To our knowledge, this is one of the few studies that compared outcomes of psychiatric hospitalizations, medication adherence, and treatment discontinuation between risperidone microspheres and paliperidone palmitate, specifically in a veteran population.^16-19 Limosin and colleagues aimed to compare length of stay during the initial hospitalization, rehospitalization risk, and treatment duration between risperidone microspheres and paliperidone palmitate in patients with schizophrenia.¹⁶ These researchers detected no differences in initial hospitalization duration and time to rehospitalization between risperidone microspheres and paliperidone palmitate.¹⁶ The study revealed a more favorable trend in time to discontinuation for paliperidone palmitate, but switching between LAIAs might have confounded the data.¹⁶ The authors note that their study lacked power, and patients on paliperidone palmitate had significantly more nonpsychiatric comorbidities.¹⁶ Joshi and colleagues looked at adherence, medication discontinuation, hospitalization rates, emergency department visits, and hospitalization costs between risperidone microspheres and paliperidone palmitate in patients identified in Truven MarketScan Commercial, Medicare Supplemental, and Medicaid Multi-State insurance databases.¹⁷ The authors found paliperidone palmitate to be superior in all objectives with better adherence, lower discontinuation rates, less likelihood of hospitalization, fewer emergency department visits, and lower hospitalization costs compared with risperidone microspheres.¹⁷ Korell and colleagues aimed to establish reference ranges for plasma concentrations of risperidone and paliperidone among adherent patients.¹⁸

The results in our study are somewhat unexpected in part because of the close relationship between risperidone and paliperidone. Risperidone is converted to paliperidone (9-OH-risperidone) via hepatic cytochrome P450 2D6. Although the molecules do not have identical pharmacologic profiles, it is accepted that they are similar enough that risperidone can establish oral tolerability when transitioning therapy to paliperidone palmitate and vice versa.²⁴ Although the active moiety in risperidone microspheres and paliperidone palmitate is similar, the dosing interval for risperidone microspheres is 2 weeks compared with 4 weeks with paliperidone palmitate. One potential explanation as to why veterans started on risperidone microspheres experienced better outcomes is because they had twice as many office visits with the health care team. Facility procedures dictate veterans receive the LAIA at an on-site clinic. During the visits, a licensed vocational nurse administers the injection and monitors the patient for 15 to 30 minutes afterward.

Despite new LAIAs coming to market, high-quality data examining potential differences in treatment outcomes among agents are limited. This is problematic for clinicians who want to optimize care by understanding how administration schedules or other aspects of LAIA use could modify treatment outcomes. Our results suggest that an advantage might exist in selecting an agent with a more frequent administration schedule, at least initially. This could allow for close monitoring and regular therapeutic contact, which could improve short-term outcomes. This conclusion is supported by meta-analyses, randomized controlled trials, and conceptual articles conducted by Wehring and colleagues, Berwaerts and colleagues, and Parellada and colleagues, respectively, who examined patients on different LAIAs and contact with health care professionals as part of their research.^26-28 These researchers concluded that patients who had regular contact with a health care professional had better outcomes when initiated on a LAIA.^26-28

Limitations

There are several limitations in this study. Retrospective and observational methods introduce risks of bias and confounding variables. Sample size might have limited statistical power to detect differences. Veterans might have had undocumented pre- or posthospitalizations at other institutions, which was not accounted for and lack of rehospitalization is not conclusive of a positive outcome. Institutions could improve on our study and help to fill gaps in comparative data by conducting larger analyses over longer periods and including more LAIA agents.

Conclusions

Although veterans that were administered risperidone microspheres had a shorter treatment duration, they were less likely to be rehospitalized, had a fewer mean number of post-LAIA hospitalizations, and had a larger difference in incidence in 100 person-years compared with veterans on paliperidone palmitate. Nonadherence and discontinuation rates were comparable between risperidone microspheres and paliperidone palmitate. Future studies could aim to further clarify differences in outcomes among agents or administration schedules.

Medication nonadherence is common with oral antipsychotic formulations, resulting in relapse, increased morbidity, and more frequent psychiatric hospitalization.^1-7 Psychiatric hospitalization and illness decompensation is costly to health care systems and leads to reduced quality of life for veterans and families.^6,7 Long-acting injectable antipsychotics (LAIAs) were developed to enhance antipsychotic adherence and improve patient outcomes, including reduced psychiatric hospitalization.^8-12

Little outcomes data exist comparing LAIAs, including biweekly risperidone microspheres and monthly paliperidone palmitate.^10-13 Risperidone microspheres require a 3-week oral crossover and are administered every 2 weeks, whereas paliperidone palmitate does not require an oral crossover and is administered every 4 weeks. The paliperidone palmitate loading regimen replaces an oral crossover.

The primary objective of this study was to compare the number of psychiatric hospitalizations between veterans administered risperidone microspheres and those on paliperidone palmitate pre- and post-LAIA initiation. Secondary objectives were to assess rehospitalization rates between patients taking risperidone microspheres and paliperidone palmitate, reduction in pre- and posthospitalization rates with LAIAs, and medication adherence.

Methods

This observational study with a retrospective cohort design was conducted at the Veterans Affairs Loma Linda Healthcare System (VALLHS) in California. We examined veterans who were initiated on LAIAs risperidone microspheres or paliperidone palmitate from January 01, 2016 through December 31, 2018. Veterans who were aged ≥ 18 years and received ≥ 2 injections of either risperidone microspheres or paliperidone palmitate during the study period were included. Veterans were excluded if they had received < 2 doses of either LAIA, received the LAIA outside of the review period, were nonadherent to risperidone crossover if they received risperidone microspheres, or transferred their care to another facility. At VALLHS, LAIA injections are administered by a nurse, and veterans must travel to the facility to receive the injections.

Extracted patient chart elements included participant demographics; diagnoses; comorbid alcohol, nicotine, opioid, or other substance use; duration on LAIA; psychiatric hospitalizations pre- and postinitiation of the LAIA; medication adherence; and medication discontinuation based on clinician documentation and clinic orders (Table 1).

Data Analysis

Patient demographics were analyzed using descriptive statistics and experimental comparisons between the risperidone microspheres and paliperidone palmitate groups to assess baseline differences between groups. Psychiatric hospitalizations pre- and post-LAIA were analyzed with parallel group (between veterans–independent groups) and pre-post (within veterans–dependent groups) designs. Index hospitalizations were examined for a period equivalent to the length of time veterans were on the LAIA. Psychiatric rehospitalization rates were analyzed for patients who had index hospitalizations and were rehospitalized for any period when they were receiving the LAIA. Incidences of pre- and post-LAIA hospitalizations were calculated in 100 person-years.

Parallel-group analysis was analyzed using the χ² and Mann-Whitney U tests. Pre-post analyses were analyzed using the Wilcoxon rank sum test. P was set at < .05 for statistical significance.

Results

We screened 111 veterans, and 97 were included in this study (risperidone microspheres, 44; paliperidone palmitate, 53). Mean (SD) age was 46 (13.8) years, 92% were male, 38% were White, 94% were diagnosed with schizophrenia or schizoaffective disorder, and 11% were homeless. Substance use was documented as 52% for nicotine products, 40% for alcohol, 31% for cannabis, 27% for methamphetamine, 7% for cocaine, and 3% for opioids. Cannabis, methamphetamine, cocaine, and opioid use were based on clinician documentation and listed as active diagnoses at the time of LAIA initiation. Statistical significance was found in index hospitalizations P = .009) and history of cocaine use disorder (6.8% vs 7.5%, P < .001).

Veterans administered risperidone microspheres had fewer mean (SD) post-LAIA hospitalizations (0.4 [1.0] vs 0.9 [1.5]; P = .02) and were less likely to be rehospitalized (22.7% vs 47.2%, P = .01) compared with paliperidone palmitate. However, veterans taking risperidone microspheres had a shorter mean (SD) treatment duration (41.6 [40.2] vs 58.2 [45.7] weeks, P = .04) compared with paliperidone palmitate, mainly because patients switched to a different LAIA or oral antipsychotic. No differences were detected in nonadherence and discontinuation between risperidone microspheres and paliperidone palmitate. All veterans in the risperidone microspheres group adhered to oral risperidone crossover with an average 87.8% days covered (Table 2).

Discussion

The main finding of this study was that initiation of LAIAs significantly reduced hospitalizations. Veterans taking risperidone microspheres had higher index hospitalizations and lower posttreatment hospitalizations compared with paliperidone palmitate. We found that patients initiated on risperidone microspheres had more hospitalizations before use of a LAIA than those initiated on paliperidone palmitate. Risperidone microspheres reduced the number of hospitalization post-LAIA significantly more than paliperidone palmitate. We also found that veterans taking risperidone microspheres were on the medication for less mean (SD) time than those on paliperidone palmitate (41.6 [40.2] vs 58.2 [45.7] weeks; P = .04).

To our knowledge, this is one of the few studies that compared outcomes of psychiatric hospitalizations, medication adherence, and treatment discontinuation between risperidone microspheres and paliperidone palmitate, specifically in a veteran population.^16-19 Limosin and colleagues aimed to compare length of stay during the initial hospitalization, rehospitalization risk, and treatment duration between risperidone microspheres and paliperidone palmitate in patients with schizophrenia.¹⁶ These researchers detected no differences in initial hospitalization duration and time to rehospitalization between risperidone microspheres and paliperidone palmitate.¹⁶ The study revealed a more favorable trend in time to discontinuation for paliperidone palmitate, but switching between LAIAs might have confounded the data.¹⁶ The authors note that their study lacked power, and patients on paliperidone palmitate had significantly more nonpsychiatric comorbidities.¹⁶ Joshi and colleagues looked at adherence, medication discontinuation, hospitalization rates, emergency department visits, and hospitalization costs between risperidone microspheres and paliperidone palmitate in patients identified in Truven MarketScan Commercial, Medicare Supplemental, and Medicaid Multi-State insurance databases.¹⁷ The authors found paliperidone palmitate to be superior in all objectives with better adherence, lower discontinuation rates, less likelihood of hospitalization, fewer emergency department visits, and lower hospitalization costs compared with risperidone microspheres.¹⁷ Korell and colleagues aimed to establish reference ranges for plasma concentrations of risperidone and paliperidone among adherent patients.¹⁸

The results in our study are somewhat unexpected in part because of the close relationship between risperidone and paliperidone. Risperidone is converted to paliperidone (9-OH-risperidone) via hepatic cytochrome P450 2D6. Although the molecules do not have identical pharmacologic profiles, it is accepted that they are similar enough that risperidone can establish oral tolerability when transitioning therapy to paliperidone palmitate and vice versa.²⁴ Although the active moiety in risperidone microspheres and paliperidone palmitate is similar, the dosing interval for risperidone microspheres is 2 weeks compared with 4 weeks with paliperidone palmitate. One potential explanation as to why veterans started on risperidone microspheres experienced better outcomes is because they had twice as many office visits with the health care team. Facility procedures dictate veterans receive the LAIA at an on-site clinic. During the visits, a licensed vocational nurse administers the injection and monitors the patient for 15 to 30 minutes afterward.

Despite new LAIAs coming to market, high-quality data examining potential differences in treatment outcomes among agents are limited. This is problematic for clinicians who want to optimize care by understanding how administration schedules or other aspects of LAIA use could modify treatment outcomes. Our results suggest that an advantage might exist in selecting an agent with a more frequent administration schedule, at least initially. This could allow for close monitoring and regular therapeutic contact, which could improve short-term outcomes. This conclusion is supported by meta-analyses, randomized controlled trials, and conceptual articles conducted by Wehring and colleagues, Berwaerts and colleagues, and Parellada and colleagues, respectively, who examined patients on different LAIAs and contact with health care professionals as part of their research.^26-28 These researchers concluded that patients who had regular contact with a health care professional had better outcomes when initiated on a LAIA.^26-28

Limitations

There are several limitations in this study. Retrospective and observational methods introduce risks of bias and confounding variables. Sample size might have limited statistical power to detect differences. Veterans might have had undocumented pre- or posthospitalizations at other institutions, which was not accounted for and lack of rehospitalization is not conclusive of a positive outcome. Institutions could improve on our study and help to fill gaps in comparative data by conducting larger analyses over longer periods and including more LAIA agents.

Conclusions

Although veterans that were administered risperidone microspheres had a shorter treatment duration, they were less likely to be rehospitalized, had a fewer mean number of post-LAIA hospitalizations, and had a larger difference in incidence in 100 person-years compared with veterans on paliperidone palmitate. Nonadherence and discontinuation rates were comparable between risperidone microspheres and paliperidone palmitate. Future studies could aim to further clarify differences in outcomes among agents or administration schedules.

Medication nonadherence is common with oral antipsychotic formulations, resulting in relapse, increased morbidity, and more frequent psychiatric hospitalization.^1-7 Psychiatric hospitalization and illness decompensation is costly to health care systems and leads to reduced quality of life for veterans and families.^6,7 Long-acting injectable antipsychotics (LAIAs) were developed to enhance antipsychotic adherence and improve patient outcomes, including reduced psychiatric hospitalization.^8-12

Little outcomes data exist comparing LAIAs, including biweekly risperidone microspheres and monthly paliperidone palmitate.^10-13 Risperidone microspheres require a 3-week oral crossover and are administered every 2 weeks, whereas paliperidone palmitate does not require an oral crossover and is administered every 4 weeks. The paliperidone palmitate loading regimen replaces an oral crossover.

The primary objective of this study was to compare the number of psychiatric hospitalizations between veterans administered risperidone microspheres and those on paliperidone palmitate pre- and post-LAIA initiation. Secondary objectives were to assess rehospitalization rates between patients taking risperidone microspheres and paliperidone palmitate, reduction in pre- and posthospitalization rates with LAIAs, and medication adherence.

Methods

This observational study with a retrospective cohort design was conducted at the Veterans Affairs Loma Linda Healthcare System (VALLHS) in California. We examined veterans who were initiated on LAIAs risperidone microspheres or paliperidone palmitate from January 01, 2016 through December 31, 2018. Veterans who were aged ≥ 18 years and received ≥ 2 injections of either risperidone microspheres or paliperidone palmitate during the study period were included. Veterans were excluded if they had received < 2 doses of either LAIA, received the LAIA outside of the review period, were nonadherent to risperidone crossover if they received risperidone microspheres, or transferred their care to another facility. At VALLHS, LAIA injections are administered by a nurse, and veterans must travel to the facility to receive the injections.

Extracted patient chart elements included participant demographics; diagnoses; comorbid alcohol, nicotine, opioid, or other substance use; duration on LAIA; psychiatric hospitalizations pre- and postinitiation of the LAIA; medication adherence; and medication discontinuation based on clinician documentation and clinic orders (Table 1).

Data Analysis

Patient demographics were analyzed using descriptive statistics and experimental comparisons between the risperidone microspheres and paliperidone palmitate groups to assess baseline differences between groups. Psychiatric hospitalizations pre- and post-LAIA were analyzed with parallel group (between veterans–independent groups) and pre-post (within veterans–dependent groups) designs. Index hospitalizations were examined for a period equivalent to the length of time veterans were on the LAIA. Psychiatric rehospitalization rates were analyzed for patients who had index hospitalizations and were rehospitalized for any period when they were receiving the LAIA. Incidences of pre- and post-LAIA hospitalizations were calculated in 100 person-years.

Parallel-group analysis was analyzed using the χ² and Mann-Whitney U tests. Pre-post analyses were analyzed using the Wilcoxon rank sum test. P was set at < .05 for statistical significance.

Results

We screened 111 veterans, and 97 were included in this study (risperidone microspheres, 44; paliperidone palmitate, 53). Mean (SD) age was 46 (13.8) years, 92% were male, 38% were White, 94% were diagnosed with schizophrenia or schizoaffective disorder, and 11% were homeless. Substance use was documented as 52% for nicotine products, 40% for alcohol, 31% for cannabis, 27% for methamphetamine, 7% for cocaine, and 3% for opioids. Cannabis, methamphetamine, cocaine, and opioid use were based on clinician documentation and listed as active diagnoses at the time of LAIA initiation. Statistical significance was found in index hospitalizations P = .009) and history of cocaine use disorder (6.8% vs 7.5%, P < .001).

Veterans administered risperidone microspheres had fewer mean (SD) post-LAIA hospitalizations (0.4 [1.0] vs 0.9 [1.5]; P = .02) and were less likely to be rehospitalized (22.7% vs 47.2%, P = .01) compared with paliperidone palmitate. However, veterans taking risperidone microspheres had a shorter mean (SD) treatment duration (41.6 [40.2] vs 58.2 [45.7] weeks, P = .04) compared with paliperidone palmitate, mainly because patients switched to a different LAIA or oral antipsychotic. No differences were detected in nonadherence and discontinuation between risperidone microspheres and paliperidone palmitate. All veterans in the risperidone microspheres group adhered to oral risperidone crossover with an average 87.8% days covered (Table 2).

Discussion

The main finding of this study was that initiation of LAIAs significantly reduced hospitalizations. Veterans taking risperidone microspheres had higher index hospitalizations and lower posttreatment hospitalizations compared with paliperidone palmitate. We found that patients initiated on risperidone microspheres had more hospitalizations before use of a LAIA than those initiated on paliperidone palmitate. Risperidone microspheres reduced the number of hospitalization post-LAIA significantly more than paliperidone palmitate. We also found that veterans taking risperidone microspheres were on the medication for less mean (SD) time than those on paliperidone palmitate (41.6 [40.2] vs 58.2 [45.7] weeks; P = .04).

To our knowledge, this is one of the few studies that compared outcomes of psychiatric hospitalizations, medication adherence, and treatment discontinuation between risperidone microspheres and paliperidone palmitate, specifically in a veteran population.^16-19 Limosin and colleagues aimed to compare length of stay during the initial hospitalization, rehospitalization risk, and treatment duration between risperidone microspheres and paliperidone palmitate in patients with schizophrenia.¹⁶ These researchers detected no differences in initial hospitalization duration and time to rehospitalization between risperidone microspheres and paliperidone palmitate.¹⁶ The study revealed a more favorable trend in time to discontinuation for paliperidone palmitate, but switching between LAIAs might have confounded the data.¹⁶ The authors note that their study lacked power, and patients on paliperidone palmitate had significantly more nonpsychiatric comorbidities.¹⁶ Joshi and colleagues looked at adherence, medication discontinuation, hospitalization rates, emergency department visits, and hospitalization costs between risperidone microspheres and paliperidone palmitate in patients identified in Truven MarketScan Commercial, Medicare Supplemental, and Medicaid Multi-State insurance databases.¹⁷ The authors found paliperidone palmitate to be superior in all objectives with better adherence, lower discontinuation rates, less likelihood of hospitalization, fewer emergency department visits, and lower hospitalization costs compared with risperidone microspheres.¹⁷ Korell and colleagues aimed to establish reference ranges for plasma concentrations of risperidone and paliperidone among adherent patients.¹⁸

The results in our study are somewhat unexpected in part because of the close relationship between risperidone and paliperidone. Risperidone is converted to paliperidone (9-OH-risperidone) via hepatic cytochrome P450 2D6. Although the molecules do not have identical pharmacologic profiles, it is accepted that they are similar enough that risperidone can establish oral tolerability when transitioning therapy to paliperidone palmitate and vice versa.²⁴ Although the active moiety in risperidone microspheres and paliperidone palmitate is similar, the dosing interval for risperidone microspheres is 2 weeks compared with 4 weeks with paliperidone palmitate. One potential explanation as to why veterans started on risperidone microspheres experienced better outcomes is because they had twice as many office visits with the health care team. Facility procedures dictate veterans receive the LAIA at an on-site clinic. During the visits, a licensed vocational nurse administers the injection and monitors the patient for 15 to 30 minutes afterward.

Despite new LAIAs coming to market, high-quality data examining potential differences in treatment outcomes among agents are limited. This is problematic for clinicians who want to optimize care by understanding how administration schedules or other aspects of LAIA use could modify treatment outcomes. Our results suggest that an advantage might exist in selecting an agent with a more frequent administration schedule, at least initially. This could allow for close monitoring and regular therapeutic contact, which could improve short-term outcomes. This conclusion is supported by meta-analyses, randomized controlled trials, and conceptual articles conducted by Wehring and colleagues, Berwaerts and colleagues, and Parellada and colleagues, respectively, who examined patients on different LAIAs and contact with health care professionals as part of their research.^26-28 These researchers concluded that patients who had regular contact with a health care professional had better outcomes when initiated on a LAIA.^26-28

Limitations

There are several limitations in this study. Retrospective and observational methods introduce risks of bias and confounding variables. Sample size might have limited statistical power to detect differences. Veterans might have had undocumented pre- or posthospitalizations at other institutions, which was not accounted for and lack of rehospitalization is not conclusive of a positive outcome. Institutions could improve on our study and help to fill gaps in comparative data by conducting larger analyses over longer periods and including more LAIA agents.

Conclusions

Although veterans that were administered risperidone microspheres had a shorter treatment duration, they were less likely to be rehospitalized, had a fewer mean number of post-LAIA hospitalizations, and had a larger difference in incidence in 100 person-years compared with veterans on paliperidone palmitate. Nonadherence and discontinuation rates were comparable between risperidone microspheres and paliperidone palmitate. Future studies could aim to further clarify differences in outcomes among agents or administration schedules.

Ease of access to opioids in the perioperative period is a risk factor for opioid misuse and has been identified as a strong risk factor for heroin use.^1,2 Three-quarters of today’s heroin users were introduced to opioids through prescription medications.² The United States accounts for about 80% of the global opioid supply consumption, and deaths from opioid overdose are increasing: 70,630 deaths in 2019 alone.^3,4

The Centers for Disease Control and Prevention (CDC) has called for changes in opioid prescribing. The American Academy of Orthopaedic Surgeons (AAOS) also has published an information statement with strategies to decrease opioid misuse and abuse.^5,6 Arthroplasty surgeons have recently focused on decreasing use of opioids in total knee arthroplasty (TKA), a procedure traditionally associated with high levels of opioid consumption and historical reliance on opioid monotherapy for postoperative analgesia.^7,8 From a clinical perspective, prolonged postoperative opioid use contributes to poorer surgical outcomes due to increased risk of complications, including stiffness, infection, and revision TKA.⁹

Multimodal pain regimens are increasingly being used to control postoperative pain as data supports their efficacy.^10,11 Previous studies have found that simultaneous modulation of multiple pain pathways decreases narcotics consumption and improves patient outcomes.^12,13 Along with other adjuvant therapies, peripheral nerve blocks, such as adductor canal block (ACB) and femoral nerve block (FNB), have been used to decrease postoperative pain.¹⁴ Studies have shown that ACB has fewer complications and shorter functional recovery times compared with FNB.^15,16 The distribution of the ACB excludes the femoral nerve, thus preserving greater quadriceps strength while providing equivalent levels of analgesia compared with FNB.^15,17,18 The ACB has shown decreased near-fall events and improved balance scores in the immediate postoperative period.¹⁹

Our study analyzed opioid consumption patterns of TKA patients from a US Department of Veterans Affairs (VA) medical center before and after the institution of a multimodal analgesic protocol using ACB. The primary purpose of this study was to determine whether a protocol that included intraoperative spinal anesthesia with a postoperative multimodal analgesic regimen and ACB was associated with a decreased postoperative opioid requirement when compared with patients who received intraoperative general anesthesia and a traditional opioid regimen. Secondary outcomes included the effect of opioid consumption on range of motion on postoperative day (POD) 1 and number of opioid prescriptions written at the first postoperative clinic visit.

Methods

Approval for the study was obtained from the institutional review board at the Dayton Veterans Affairs Medical Center (DVAMC) in Ohio. A retrospective chart review was performed to collect data from all patients undergoing TKA at DVAMC from June 1, 2011, through December 31, 2015. Exclusion criteria included multiple surgeries in the study time frame, documented chronic pain, allergy to local anesthetics, daily preoperative use of opioids, and incomplete data in the health record.

All surgeries were performed by 2 staff arthroplasty surgeons at a single VAMC. All patients attended a preoperative visit where a history, physical, and anesthesia evaluation were performed, and watched an educational video detailing surgical indications and postoperative rehabilitation. All surgeries were performed with tourniquets and a periarticular injection was performed at the conclusion of each case. Surgeon 1 treatment of choice was 10 mL 0.5% bupivacaine, whereas surgeon 2 performed a posterior capsular injection of 30 mL 0.25% bupivacaine and a periarticular injection of 30 mg ketorolac in 10 mL 0.25% bupivacaine with epinephrine.

Prior to August 2014, general endotracheal anesthesia was used intraoperatively. A patient-controlled analgesia (PCA) pump of morphine or hydromorphone and additional oral oxycodone or hydrocodone was used for postoperative pain. PCA pumps were patient dependent. In the control group, 245 patients received the morphine PCA while 61 received the hydromorphone PCA. Morphine PCA dosing consisted of 1-mg doses every 10 minutes with potential baseline infusion rates of 0.5 to 1.0 mg/h and a 4-hour limit of 20 mg. Hydromorphone PCA dosing consisted of 0.2 to 0.4-mg doses with a potential continuous dose of 0.2 to 0.4 mg/h and a 4-hour limit of 4 mg.

Results

During the study period from June 1, 2011, through December 31, 2015, 533 eligible TKAs were performed, 306 in the control group and 227 in the protocol group. The groups had similar sex distribution; body mass index; knee range of motion; diagnoses of diabetes mellitus, coronary artery disease, and chronic kidney disease; and history of deep vein thrombosis (DVT) or pulmonary embolism (P ≥ .05). The protocol group was significantly older (P = .04) and had a significantly higher rate of chronic obstructive pulmonary disease (COPD) (P = .002). There were no significant differences between number of procedures performed by surgeon (P = .48) or total tourniquet time (P = .13) (Table 2). Mean (SD) length of stay was significantly greater in the control group compared with the protocol group (2.5 [1.3] vs 1.4 [0.7] days, P < .001).

Figure 1 shows the distributions of each type of opioid used. Compared with the control group, the protocol group had a significantly lower mean (SD) IV opioid use: 178.2 (98.0) MED vs 12.0 (24.6) MED (P < .001; d = 2.19; Δ = 0.94) and mean (SD) total opioid use: 241.7 (120.1) MED vs 74.8 (42.7) MED (P < .001; d = 1.76; Δ = 0.89). Mean (SD) oral opioid use did not differ between groups (control, 63.6 [45.4] MED; protocol, 62.9 [31.4] MED; P = .85; d = 0.02; Δ = 0.51). A significantly lower percentage of patients in the protocol group received additional opioids at the 3-week follow-up when compared to the control group: 46.7% vs 61.3%, respectively (P < .001; RR, 0.76; 95% CI, 0.65-0.90).

Discussion

Coordinated efforts with major medical organizations are being made to decrease opioid prescriptions and exposure.^5,6 To our knowledge, no study has quantified a decrease in opioid requirement in a VA population after implementation of a protocol that includes intraoperative spinal anesthesia and a postoperative multimodal analgesic regimen including ACB after TKA. The analgesic protocol described in this study aligns with recommendations from both the CDC and the AAOS to decrease opioid use and misuse by maximizing nonopioid medications and limiting the size and number of opioid prescriptions. However, public and medical opinion of opioids as well as prescribing practices have changed over time with a trend toward lower opioid use. The interventions, as part of the described protocol, are a result of these changes and attempt to minimize opioid use while maximizing postoperative analgesia.

Our data showed a significant decrease in total opioid use through POD 1, IV opioid use, and opioid prescriptions provided at the first postoperative visit. The protocol group used only 6.7% of the IV opioids and 30.9% of the total opioids that were used by the control group. The substantial difference in IV opioid requirement, 166.2 MED, is equivalent to 8 mg of IV hydromorphone or 55 mg of IV morphine. The difference in total opioid requirement was similar at 166.9 MED, equivalent to 111 mg of oral oxycodone.

Decreasing opioid use has the additional benefit of improving outcomes, as higher doses of opioids have been associated with increased length of stay, greater rates of DVT, and postoperative infection.²³ These complications occurred in a stepwise manner, suggesting a dose-response gradient that makes the sizable decrease noted in our data of greater relevance.²³ While the adverse effects (AEs) of opioids are well known, there are limited data on opioid dosing and its effect on perioperative outcomes.²³

A significant decrease in the percentage of patients receiving an opioid prescription at the first postoperative visit suggests a decrease in the number of patients on prolonged opioids after TKA with implementation of modern analgesic modalities. The duration of postoperative opioid use has been found to be the strongest predictor of misuse, and each postoperative refill increases the probability of misuse by 44%.²⁴ In addition, opioid use for > 3 months after TKA is associated with increased risk of periprosthetic infection, increased overall revision rate, and stiffness at 1 year postoperatively.⁹ While not entirely under the control of the surgeon, measures to decrease the number of postoperative opioid refills may lead to a decrease in opioid misuse.

Limitations

A number of limitations of this study should be noted. One was a protocol change regarding length of stay, which occurred during the study period and resulted in a significantly shorter length of stay in the protocol group. As a result, opioid use data were analyzed only through midnight at the end of POD 1. Patients who were discharged on POD 1 did not have opioid use data available for the full duration of the first POD, which may exaggerate the decrease in opioid requirements, as opioids used after discharge but prior to midnight on POD 1 were not recorded. However, opioids taken at home are oral with a low MME compared with IV opioids received by hospitalized patients in the control group. In addition, if taken as prescribed, patients at home would only have enough time to take a few doses of opioids prior to the midnight cutoff. We do not believe this difference in time of opioid use meaningfully affected the data. An additional limitation includes the variability between periarticular injections between surgeons. While the percentage of patients that received injections from surgeon 1 vs surgeon 2 were similar, it cannot be ruled out as a potential confounding factor. Other limitations include a lack of pain scores to compare subjective pain ratings, the retrospective nature of the study, and a largely homogenous male VA population.

Conclusions

Ease of access to opioids is a risk factor for opioid abuse, which itself is a risk factor for subsequent heroin use.^1,2 The CDC and AAOS have thus published recommendations regarding opioid prescribing practices to decrease opioid use and abuse.^5,6 Our described protocol, which aligns with these recommendations, resulted in a significant decrease in IV opioid requirement, total opioid requirement, and lower rates of opioid prescriptions provided at the first postoperative visit. These promising findings demonstrate a lower percentage of patients on long-term opioids after TKA and a significantly decreased cumulative opioid exposure.

Ease of access to opioids in the perioperative period is a risk factor for opioid misuse and has been identified as a strong risk factor for heroin use.^1,2 Three-quarters of today’s heroin users were introduced to opioids through prescription medications.² The United States accounts for about 80% of the global opioid supply consumption, and deaths from opioid overdose are increasing: 70,630 deaths in 2019 alone.^3,4

The Centers for Disease Control and Prevention (CDC) has called for changes in opioid prescribing. The American Academy of Orthopaedic Surgeons (AAOS) also has published an information statement with strategies to decrease opioid misuse and abuse.^5,6 Arthroplasty surgeons have recently focused on decreasing use of opioids in total knee arthroplasty (TKA), a procedure traditionally associated with high levels of opioid consumption and historical reliance on opioid monotherapy for postoperative analgesia.^7,8 From a clinical perspective, prolonged postoperative opioid use contributes to poorer surgical outcomes due to increased risk of complications, including stiffness, infection, and revision TKA.⁹

Multimodal pain regimens are increasingly being used to control postoperative pain as data supports their efficacy.^10,11 Previous studies have found that simultaneous modulation of multiple pain pathways decreases narcotics consumption and improves patient outcomes.^12,13 Along with other adjuvant therapies, peripheral nerve blocks, such as adductor canal block (ACB) and femoral nerve block (FNB), have been used to decrease postoperative pain.¹⁴ Studies have shown that ACB has fewer complications and shorter functional recovery times compared with FNB.^15,16 The distribution of the ACB excludes the femoral nerve, thus preserving greater quadriceps strength while providing equivalent levels of analgesia compared with FNB.^15,17,18 The ACB has shown decreased near-fall events and improved balance scores in the immediate postoperative period.¹⁹

Our study analyzed opioid consumption patterns of TKA patients from a US Department of Veterans Affairs (VA) medical center before and after the institution of a multimodal analgesic protocol using ACB. The primary purpose of this study was to determine whether a protocol that included intraoperative spinal anesthesia with a postoperative multimodal analgesic regimen and ACB was associated with a decreased postoperative opioid requirement when compared with patients who received intraoperative general anesthesia and a traditional opioid regimen. Secondary outcomes included the effect of opioid consumption on range of motion on postoperative day (POD) 1 and number of opioid prescriptions written at the first postoperative clinic visit.

Methods

Approval for the study was obtained from the institutional review board at the Dayton Veterans Affairs Medical Center (DVAMC) in Ohio. A retrospective chart review was performed to collect data from all patients undergoing TKA at DVAMC from June 1, 2011, through December 31, 2015. Exclusion criteria included multiple surgeries in the study time frame, documented chronic pain, allergy to local anesthetics, daily preoperative use of opioids, and incomplete data in the health record.

All surgeries were performed by 2 staff arthroplasty surgeons at a single VAMC. All patients attended a preoperative visit where a history, physical, and anesthesia evaluation were performed, and watched an educational video detailing surgical indications and postoperative rehabilitation. All surgeries were performed with tourniquets and a periarticular injection was performed at the conclusion of each case. Surgeon 1 treatment of choice was 10 mL 0.5% bupivacaine, whereas surgeon 2 performed a posterior capsular injection of 30 mL 0.25% bupivacaine and a periarticular injection of 30 mg ketorolac in 10 mL 0.25% bupivacaine with epinephrine.

Prior to August 2014, general endotracheal anesthesia was used intraoperatively. A patient-controlled analgesia (PCA) pump of morphine or hydromorphone and additional oral oxycodone or hydrocodone was used for postoperative pain. PCA pumps were patient dependent. In the control group, 245 patients received the morphine PCA while 61 received the hydromorphone PCA. Morphine PCA dosing consisted of 1-mg doses every 10 minutes with potential baseline infusion rates of 0.5 to 1.0 mg/h and a 4-hour limit of 20 mg. Hydromorphone PCA dosing consisted of 0.2 to 0.4-mg doses with a potential continuous dose of 0.2 to 0.4 mg/h and a 4-hour limit of 4 mg.

Results

During the study period from June 1, 2011, through December 31, 2015, 533 eligible TKAs were performed, 306 in the control group and 227 in the protocol group. The groups had similar sex distribution; body mass index; knee range of motion; diagnoses of diabetes mellitus, coronary artery disease, and chronic kidney disease; and history of deep vein thrombosis (DVT) or pulmonary embolism (P ≥ .05). The protocol group was significantly older (P = .04) and had a significantly higher rate of chronic obstructive pulmonary disease (COPD) (P = .002). There were no significant differences between number of procedures performed by surgeon (P = .48) or total tourniquet time (P = .13) (Table 2). Mean (SD) length of stay was significantly greater in the control group compared with the protocol group (2.5 [1.3] vs 1.4 [0.7] days, P < .001).

Figure 1 shows the distributions of each type of opioid used. Compared with the control group, the protocol group had a significantly lower mean (SD) IV opioid use: 178.2 (98.0) MED vs 12.0 (24.6) MED (P < .001; d = 2.19; Δ = 0.94) and mean (SD) total opioid use: 241.7 (120.1) MED vs 74.8 (42.7) MED (P < .001; d = 1.76; Δ = 0.89). Mean (SD) oral opioid use did not differ between groups (control, 63.6 [45.4] MED; protocol, 62.9 [31.4] MED; P = .85; d = 0.02; Δ = 0.51). A significantly lower percentage of patients in the protocol group received additional opioids at the 3-week follow-up when compared to the control group: 46.7% vs 61.3%, respectively (P < .001; RR, 0.76; 95% CI, 0.65-0.90).

Discussion

Coordinated efforts with major medical organizations are being made to decrease opioid prescriptions and exposure.^5,6 To our knowledge, no study has quantified a decrease in opioid requirement in a VA population after implementation of a protocol that includes intraoperative spinal anesthesia and a postoperative multimodal analgesic regimen including ACB after TKA. The analgesic protocol described in this study aligns with recommendations from both the CDC and the AAOS to decrease opioid use and misuse by maximizing nonopioid medications and limiting the size and number of opioid prescriptions. However, public and medical opinion of opioids as well as prescribing practices have changed over time with a trend toward lower opioid use. The interventions, as part of the described protocol, are a result of these changes and attempt to minimize opioid use while maximizing postoperative analgesia.

Our data showed a significant decrease in total opioid use through POD 1, IV opioid use, and opioid prescriptions provided at the first postoperative visit. The protocol group used only 6.7% of the IV opioids and 30.9% of the total opioids that were used by the control group. The substantial difference in IV opioid requirement, 166.2 MED, is equivalent to 8 mg of IV hydromorphone or 55 mg of IV morphine. The difference in total opioid requirement was similar at 166.9 MED, equivalent to 111 mg of oral oxycodone.

Decreasing opioid use has the additional benefit of improving outcomes, as higher doses of opioids have been associated with increased length of stay, greater rates of DVT, and postoperative infection.²³ These complications occurred in a stepwise manner, suggesting a dose-response gradient that makes the sizable decrease noted in our data of greater relevance.²³ While the adverse effects (AEs) of opioids are well known, there are limited data on opioid dosing and its effect on perioperative outcomes.²³

A significant decrease in the percentage of patients receiving an opioid prescription at the first postoperative visit suggests a decrease in the number of patients on prolonged opioids after TKA with implementation of modern analgesic modalities. The duration of postoperative opioid use has been found to be the strongest predictor of misuse, and each postoperative refill increases the probability of misuse by 44%.²⁴ In addition, opioid use for > 3 months after TKA is associated with increased risk of periprosthetic infection, increased overall revision rate, and stiffness at 1 year postoperatively.⁹ While not entirely under the control of the surgeon, measures to decrease the number of postoperative opioid refills may lead to a decrease in opioid misuse.

Limitations

A number of limitations of this study should be noted. One was a protocol change regarding length of stay, which occurred during the study period and resulted in a significantly shorter length of stay in the protocol group. As a result, opioid use data were analyzed only through midnight at the end of POD 1. Patients who were discharged on POD 1 did not have opioid use data available for the full duration of the first POD, which may exaggerate the decrease in opioid requirements, as opioids used after discharge but prior to midnight on POD 1 were not recorded. However, opioids taken at home are oral with a low MME compared with IV opioids received by hospitalized patients in the control group. In addition, if taken as prescribed, patients at home would only have enough time to take a few doses of opioids prior to the midnight cutoff. We do not believe this difference in time of opioid use meaningfully affected the data. An additional limitation includes the variability between periarticular injections between surgeons. While the percentage of patients that received injections from surgeon 1 vs surgeon 2 were similar, it cannot be ruled out as a potential confounding factor. Other limitations include a lack of pain scores to compare subjective pain ratings, the retrospective nature of the study, and a largely homogenous male VA population.

Conclusions

Ease of access to opioids is a risk factor for opioid abuse, which itself is a risk factor for subsequent heroin use.^1,2 The CDC and AAOS have thus published recommendations regarding opioid prescribing practices to decrease opioid use and abuse.^5,6 Our described protocol, which aligns with these recommendations, resulted in a significant decrease in IV opioid requirement, total opioid requirement, and lower rates of opioid prescriptions provided at the first postoperative visit. These promising findings demonstrate a lower percentage of patients on long-term opioids after TKA and a significantly decreased cumulative opioid exposure.

Ease of access to opioids in the perioperative period is a risk factor for opioid misuse and has been identified as a strong risk factor for heroin use.^1,2 Three-quarters of today’s heroin users were introduced to opioids through prescription medications.² The United States accounts for about 80% of the global opioid supply consumption, and deaths from opioid overdose are increasing: 70,630 deaths in 2019 alone.^3,4

The Centers for Disease Control and Prevention (CDC) has called for changes in opioid prescribing. The American Academy of Orthopaedic Surgeons (AAOS) also has published an information statement with strategies to decrease opioid misuse and abuse.^5,6 Arthroplasty surgeons have recently focused on decreasing use of opioids in total knee arthroplasty (TKA), a procedure traditionally associated with high levels of opioid consumption and historical reliance on opioid monotherapy for postoperative analgesia.^7,8 From a clinical perspective, prolonged postoperative opioid use contributes to poorer surgical outcomes due to increased risk of complications, including stiffness, infection, and revision TKA.⁹

Multimodal pain regimens are increasingly being used to control postoperative pain as data supports their efficacy.^10,11 Previous studies have found that simultaneous modulation of multiple pain pathways decreases narcotics consumption and improves patient outcomes.^12,13 Along with other adjuvant therapies, peripheral nerve blocks, such as adductor canal block (ACB) and femoral nerve block (FNB), have been used to decrease postoperative pain.¹⁴ Studies have shown that ACB has fewer complications and shorter functional recovery times compared with FNB.^15,16 The distribution of the ACB excludes the femoral nerve, thus preserving greater quadriceps strength while providing equivalent levels of analgesia compared with FNB.^15,17,18 The ACB has shown decreased near-fall events and improved balance scores in the immediate postoperative period.¹⁹

Our study analyzed opioid consumption patterns of TKA patients from a US Department of Veterans Affairs (VA) medical center before and after the institution of a multimodal analgesic protocol using ACB. The primary purpose of this study was to determine whether a protocol that included intraoperative spinal anesthesia with a postoperative multimodal analgesic regimen and ACB was associated with a decreased postoperative opioid requirement when compared with patients who received intraoperative general anesthesia and a traditional opioid regimen. Secondary outcomes included the effect of opioid consumption on range of motion on postoperative day (POD) 1 and number of opioid prescriptions written at the first postoperative clinic visit.

Methods

Approval for the study was obtained from the institutional review board at the Dayton Veterans Affairs Medical Center (DVAMC) in Ohio. A retrospective chart review was performed to collect data from all patients undergoing TKA at DVAMC from June 1, 2011, through December 31, 2015. Exclusion criteria included multiple surgeries in the study time frame, documented chronic pain, allergy to local anesthetics, daily preoperative use of opioids, and incomplete data in the health record.

All surgeries were performed by 2 staff arthroplasty surgeons at a single VAMC. All patients attended a preoperative visit where a history, physical, and anesthesia evaluation were performed, and watched an educational video detailing surgical indications and postoperative rehabilitation. All surgeries were performed with tourniquets and a periarticular injection was performed at the conclusion of each case. Surgeon 1 treatment of choice was 10 mL 0.5% bupivacaine, whereas surgeon 2 performed a posterior capsular injection of 30 mL 0.25% bupivacaine and a periarticular injection of 30 mg ketorolac in 10 mL 0.25% bupivacaine with epinephrine.

Prior to August 2014, general endotracheal anesthesia was used intraoperatively. A patient-controlled analgesia (PCA) pump of morphine or hydromorphone and additional oral oxycodone or hydrocodone was used for postoperative pain. PCA pumps were patient dependent. In the control group, 245 patients received the morphine PCA while 61 received the hydromorphone PCA. Morphine PCA dosing consisted of 1-mg doses every 10 minutes with potential baseline infusion rates of 0.5 to 1.0 mg/h and a 4-hour limit of 20 mg. Hydromorphone PCA dosing consisted of 0.2 to 0.4-mg doses with a potential continuous dose of 0.2 to 0.4 mg/h and a 4-hour limit of 4 mg.

Results

During the study period from June 1, 2011, through December 31, 2015, 533 eligible TKAs were performed, 306 in the control group and 227 in the protocol group. The groups had similar sex distribution; body mass index; knee range of motion; diagnoses of diabetes mellitus, coronary artery disease, and chronic kidney disease; and history of deep vein thrombosis (DVT) or pulmonary embolism (P ≥ .05). The protocol group was significantly older (P = .04) and had a significantly higher rate of chronic obstructive pulmonary disease (COPD) (P = .002). There were no significant differences between number of procedures performed by surgeon (P = .48) or total tourniquet time (P = .13) (Table 2). Mean (SD) length of stay was significantly greater in the control group compared with the protocol group (2.5 [1.3] vs 1.4 [0.7] days, P < .001).

Figure 1 shows the distributions of each type of opioid used. Compared with the control group, the protocol group had a significantly lower mean (SD) IV opioid use: 178.2 (98.0) MED vs 12.0 (24.6) MED (P < .001; d = 2.19; Δ = 0.94) and mean (SD) total opioid use: 241.7 (120.1) MED vs 74.8 (42.7) MED (P < .001; d = 1.76; Δ = 0.89). Mean (SD) oral opioid use did not differ between groups (control, 63.6 [45.4] MED; protocol, 62.9 [31.4] MED; P = .85; d = 0.02; Δ = 0.51). A significantly lower percentage of patients in the protocol group received additional opioids at the 3-week follow-up when compared to the control group: 46.7% vs 61.3%, respectively (P < .001; RR, 0.76; 95% CI, 0.65-0.90).

Discussion

Coordinated efforts with major medical organizations are being made to decrease opioid prescriptions and exposure.^5,6 To our knowledge, no study has quantified a decrease in opioid requirement in a VA population after implementation of a protocol that includes intraoperative spinal anesthesia and a postoperative multimodal analgesic regimen including ACB after TKA. The analgesic protocol described in this study aligns with recommendations from both the CDC and the AAOS to decrease opioid use and misuse by maximizing nonopioid medications and limiting the size and number of opioid prescriptions. However, public and medical opinion of opioids as well as prescribing practices have changed over time with a trend toward lower opioid use. The interventions, as part of the described protocol, are a result of these changes and attempt to minimize opioid use while maximizing postoperative analgesia.

Our data showed a significant decrease in total opioid use through POD 1, IV opioid use, and opioid prescriptions provided at the first postoperative visit. The protocol group used only 6.7% of the IV opioids and 30.9% of the total opioids that were used by the control group. The substantial difference in IV opioid requirement, 166.2 MED, is equivalent to 8 mg of IV hydromorphone or 55 mg of IV morphine. The difference in total opioid requirement was similar at 166.9 MED, equivalent to 111 mg of oral oxycodone.

Decreasing opioid use has the additional benefit of improving outcomes, as higher doses of opioids have been associated with increased length of stay, greater rates of DVT, and postoperative infection.²³ These complications occurred in a stepwise manner, suggesting a dose-response gradient that makes the sizable decrease noted in our data of greater relevance.²³ While the adverse effects (AEs) of opioids are well known, there are limited data on opioid dosing and its effect on perioperative outcomes.²³

A significant decrease in the percentage of patients receiving an opioid prescription at the first postoperative visit suggests a decrease in the number of patients on prolonged opioids after TKA with implementation of modern analgesic modalities. The duration of postoperative opioid use has been found to be the strongest predictor of misuse, and each postoperative refill increases the probability of misuse by 44%.²⁴ In addition, opioid use for > 3 months after TKA is associated with increased risk of periprosthetic infection, increased overall revision rate, and stiffness at 1 year postoperatively.⁹ While not entirely under the control of the surgeon, measures to decrease the number of postoperative opioid refills may lead to a decrease in opioid misuse.

Limitations

A number of limitations of this study should be noted. One was a protocol change regarding length of stay, which occurred during the study period and resulted in a significantly shorter length of stay in the protocol group. As a result, opioid use data were analyzed only through midnight at the end of POD 1. Patients who were discharged on POD 1 did not have opioid use data available for the full duration of the first POD, which may exaggerate the decrease in opioid requirements, as opioids used after discharge but prior to midnight on POD 1 were not recorded. However, opioids taken at home are oral with a low MME compared with IV opioids received by hospitalized patients in the control group. In addition, if taken as prescribed, patients at home would only have enough time to take a few doses of opioids prior to the midnight cutoff. We do not believe this difference in time of opioid use meaningfully affected the data. An additional limitation includes the variability between periarticular injections between surgeons. While the percentage of patients that received injections from surgeon 1 vs surgeon 2 were similar, it cannot be ruled out as a potential confounding factor. Other limitations include a lack of pain scores to compare subjective pain ratings, the retrospective nature of the study, and a largely homogenous male VA population.

Conclusions

Ease of access to opioids is a risk factor for opioid abuse, which itself is a risk factor for subsequent heroin use.^1,2 The CDC and AAOS have thus published recommendations regarding opioid prescribing practices to decrease opioid use and abuse.^5,6 Our described protocol, which aligns with these recommendations, resulted in a significant decrease in IV opioid requirement, total opioid requirement, and lower rates of opioid prescriptions provided at the first postoperative visit. These promising findings demonstrate a lower percentage of patients on long-term opioids after TKA and a significantly decreased cumulative opioid exposure.

To the Editor:

Approved medical devices on the market are substantial capital investments for practitioners. E-commerce websites, such as Alibaba.com (https://www.alibaba.com/) and DHgate.com (https://www.dhgate.com/), sell sham medical devices at a fraction of the cost of authentic products, with sellers often echoing the same treatment claims as legitimate devices that have been cleared by the US Food and Drug Administration (FDA).

In dermatology, devices claiming to perform cryolipolysis, laser skin resurfacing, radiofrequency skin tightening, and more exist on e-commerce websites. These counterfeit medical devices might differ from legitimate devices in ways that affect patient safety and treatment efficacy.^1,2 The degree of difference between counterfeit and legitimate devices remains unknown, and potential harm from so-called knockoff devices needs to be critically examined by providers.

In this exploratory study, we characterize counterfeit listings of devices commonly used in dermatology. Using the trademark name of devices as the key terms, we searched Alibaba.com and DHgate.com for listings of counterfeit products. We recorded the total number of listings; the listing name, catalog number, and unit price; and claims of FDA certification. Characteristics of counterfeit listings were summarized using standard descriptive statistics in Microsoft Excel. Continuous variables were summarized with means and ranges.

Six medical devices that had been cleared by the FDA between 2002 and 2012 for use in dermatology were explored, including systems for picosecond and fractionated lasers, monopolar and bipolar radiofrequency skin tightening, cryolipolysis, and nonablative radiofrequency skin resurfacing. Our search of these 6 representative dermatologic devices revealed 47,055 counterfeit product listings on Alibaba.com and DHgate.com. Upon searching these popular e-commerce websites using the device name as the search term, the number of listings varied considerably between the 2 e-commerce websites for the same device and from device to device on the same e-commerce website. On Alibaba.com, the greatest number of listings resulted for picosecond laser (23,622 listings), fractionated laser (15,269), and radiofrequency skin tightening devices (3555); cryolipolysis and nonablative radiofrequency resurfacing devices had notably fewer listings (35 and 38, respectively). On DHGate.com, a similar trend was noted with the most numerous listings for picosecond and fractionated laser systems (2429 and 1345, respectively).

Among the first 10 listings of products on Alibaba.com and DHgate.com for these 6 devices, 10.7% (11 of 103) had advertised claims of FDA clearance on the listing page. Of 103 counterfeit products, China was the country of origin for 100; South Korea for 2; and Thailand for 1. Unit pricing was heterogeneous between the 2 e-commerce websites for the counterfeit listings; pricing for duplicate fractionated laser systems was particularly dissimilar, with an average price on Alibab.com of US $8105.80 and an average price on DHgate.com of US $3409.14. Even on the same e-commerce website, the range of unit pricing differed greatly for dermatologic devices. For example, among the first 10 listings on Alibaba.com for a fractionated laser system, the price ranged from US $2300 to US $32,000.

Counterfeit medical devices are on the rise in dermatology.^1,3 Although devices such as radiofrequency and laser systems had thousands of knockoff listings on 2 e-commerce websites, other devices, such as cryolipolysis and body contouring systems, had fewer listings, suggesting heterogeneity in the prevalence of different counterfeit dermatologic devices on the market.

The varied pricing of the top 10 listings for each product and spurious claims of FDA clearance for some listings highlight the lack of regulatory authority over consistent product information on e-commerce websites. Furthermore, differences between characteristics of counterfeit device listings can impede efforts to trace suppliers and increase the opacity of counterfeit purchasing.

Three criteria have been proposed for a device to be considered counterfeit³:

• The device has no proven safety or efficacy among consumers. For example, the substantial threat of copycat devices in dermatology has been demonstrated by reports of burns caused by fake cryolipolysis devices.²

• The device violates patent rights or copy trademarks. Due to the regional nature of intellectual property rights, country-specific filings of patents and trademarks are required if protections are sought internationally. In this study, counterfeit devices originated in China, South Korea, and Thailand, where patent and trademark protections for the original devices do not extend.

• The device is falsely claimed to have been cleared by the FDA or other clinical regulatory authorities. Legitimate medical devices are subject to rounds of safety and compatibility testing using standards set by regulatory bodies, such as the FDA’s Center for Devices and Radiological Health, the International Organization of Standardization, and the International Electrotechnical Commission. Compliance with these safety standards is lost, however, among unregulated internet sales of medical devices. Our search revealed that 10.7% of the top 10 counterfeit device listings for each product explicitly mentioned FDA clearance in the product description. Among the thousands of listings on e-commerce sites, even a fraction that make spurious FDA-clearance claims can mislead consumers.

The issue of counterfeit medical devices has not gone unrecognized globally. In 2013, the World Health Organization created the Global Surveillance and Monitoring System to unify international efforts for reporting substandard, unlicensed, or falsified medical products.⁴ Although universal monitoring systems can improve detection of counterfeit products, we highlight the alarming continuing ease of purchasing counterfeit dermatologic devices through e-commerce websites. Due to the widespread nature of counterfeiting across all domains of medicine, the onus of curbing counterfeit dermatologic devices might be on dermatology providers to recognize and report such occurrences.

This exploration of counterfeit dermatologic devices revealed a lack of consistency throughout product listings on 2 popular e-commerce websites, Alibaba.com and DHgate.com. Given the alarming availability of these devices on the internet, practitioners should approach the purchase of any device with concern about counterfeiting. Future avenues of study might explore the prevalence of counterfeit devices used in dermatology practices and offer insight on regulation and consumer safety efforts.

To the Editor:

Approved medical devices on the market are substantial capital investments for practitioners. E-commerce websites, such as Alibaba.com (https://www.alibaba.com/) and DHgate.com (https://www.dhgate.com/), sell sham medical devices at a fraction of the cost of authentic products, with sellers often echoing the same treatment claims as legitimate devices that have been cleared by the US Food and Drug Administration (FDA).

In dermatology, devices claiming to perform cryolipolysis, laser skin resurfacing, radiofrequency skin tightening, and more exist on e-commerce websites. These counterfeit medical devices might differ from legitimate devices in ways that affect patient safety and treatment efficacy.^1,2 The degree of difference between counterfeit and legitimate devices remains unknown, and potential harm from so-called knockoff devices needs to be critically examined by providers.

In this exploratory study, we characterize counterfeit listings of devices commonly used in dermatology. Using the trademark name of devices as the key terms, we searched Alibaba.com and DHgate.com for listings of counterfeit products. We recorded the total number of listings; the listing name, catalog number, and unit price; and claims of FDA certification. Characteristics of counterfeit listings were summarized using standard descriptive statistics in Microsoft Excel. Continuous variables were summarized with means and ranges.

Six medical devices that had been cleared by the FDA between 2002 and 2012 for use in dermatology were explored, including systems for picosecond and fractionated lasers, monopolar and bipolar radiofrequency skin tightening, cryolipolysis, and nonablative radiofrequency skin resurfacing. Our search of these 6 representative dermatologic devices revealed 47,055 counterfeit product listings on Alibaba.com and DHgate.com. Upon searching these popular e-commerce websites using the device name as the search term, the number of listings varied considerably between the 2 e-commerce websites for the same device and from device to device on the same e-commerce website. On Alibaba.com, the greatest number of listings resulted for picosecond laser (23,622 listings), fractionated laser (15,269), and radiofrequency skin tightening devices (3555); cryolipolysis and nonablative radiofrequency resurfacing devices had notably fewer listings (35 and 38, respectively). On DHGate.com, a similar trend was noted with the most numerous listings for picosecond and fractionated laser systems (2429 and 1345, respectively).

Among the first 10 listings of products on Alibaba.com and DHgate.com for these 6 devices, 10.7% (11 of 103) had advertised claims of FDA clearance on the listing page. Of 103 counterfeit products, China was the country of origin for 100; South Korea for 2; and Thailand for 1. Unit pricing was heterogeneous between the 2 e-commerce websites for the counterfeit listings; pricing for duplicate fractionated laser systems was particularly dissimilar, with an average price on Alibab.com of US $8105.80 and an average price on DHgate.com of US $3409.14. Even on the same e-commerce website, the range of unit pricing differed greatly for dermatologic devices. For example, among the first 10 listings on Alibaba.com for a fractionated laser system, the price ranged from US $2300 to US $32,000.

Counterfeit medical devices are on the rise in dermatology.^1,3 Although devices such as radiofrequency and laser systems had thousands of knockoff listings on 2 e-commerce websites, other devices, such as cryolipolysis and body contouring systems, had fewer listings, suggesting heterogeneity in the prevalence of different counterfeit dermatologic devices on the market.

The varied pricing of the top 10 listings for each product and spurious claims of FDA clearance for some listings highlight the lack of regulatory authority over consistent product information on e-commerce websites. Furthermore, differences between characteristics of counterfeit device listings can impede efforts to trace suppliers and increase the opacity of counterfeit purchasing.

Three criteria have been proposed for a device to be considered counterfeit³:

• The device has no proven safety or efficacy among consumers. For example, the substantial threat of copycat devices in dermatology has been demonstrated by reports of burns caused by fake cryolipolysis devices.²

• The device violates patent rights or copy trademarks. Due to the regional nature of intellectual property rights, country-specific filings of patents and trademarks are required if protections are sought internationally. In this study, counterfeit devices originated in China, South Korea, and Thailand, where patent and trademark protections for the original devices do not extend.

• The device is falsely claimed to have been cleared by the FDA or other clinical regulatory authorities. Legitimate medical devices are subject to rounds of safety and compatibility testing using standards set by regulatory bodies, such as the FDA’s Center for Devices and Radiological Health, the International Organization of Standardization, and the International Electrotechnical Commission. Compliance with these safety standards is lost, however, among unregulated internet sales of medical devices. Our search revealed that 10.7% of the top 10 counterfeit device listings for each product explicitly mentioned FDA clearance in the product description. Among the thousands of listings on e-commerce sites, even a fraction that make spurious FDA-clearance claims can mislead consumers.

The issue of counterfeit medical devices has not gone unrecognized globally. In 2013, the World Health Organization created the Global Surveillance and Monitoring System to unify international efforts for reporting substandard, unlicensed, or falsified medical products.⁴ Although universal monitoring systems can improve detection of counterfeit products, we highlight the alarming continuing ease of purchasing counterfeit dermatologic devices through e-commerce websites. Due to the widespread nature of counterfeiting across all domains of medicine, the onus of curbing counterfeit dermatologic devices might be on dermatology providers to recognize and report such occurrences.

This exploration of counterfeit dermatologic devices revealed a lack of consistency throughout product listings on 2 popular e-commerce websites, Alibaba.com and DHgate.com. Given the alarming availability of these devices on the internet, practitioners should approach the purchase of any device with concern about counterfeiting. Future avenues of study might explore the prevalence of counterfeit devices used in dermatology practices and offer insight on regulation and consumer safety efforts.

To the Editor:

Approved medical devices on the market are substantial capital investments for practitioners. E-commerce websites, such as Alibaba.com (https://www.alibaba.com/) and DHgate.com (https://www.dhgate.com/), sell sham medical devices at a fraction of the cost of authentic products, with sellers often echoing the same treatment claims as legitimate devices that have been cleared by the US Food and Drug Administration (FDA).

In dermatology, devices claiming to perform cryolipolysis, laser skin resurfacing, radiofrequency skin tightening, and more exist on e-commerce websites. These counterfeit medical devices might differ from legitimate devices in ways that affect patient safety and treatment efficacy.^1,2 The degree of difference between counterfeit and legitimate devices remains unknown, and potential harm from so-called knockoff devices needs to be critically examined by providers.

In this exploratory study, we characterize counterfeit listings of devices commonly used in dermatology. Using the trademark name of devices as the key terms, we searched Alibaba.com and DHgate.com for listings of counterfeit products. We recorded the total number of listings; the listing name, catalog number, and unit price; and claims of FDA certification. Characteristics of counterfeit listings were summarized using standard descriptive statistics in Microsoft Excel. Continuous variables were summarized with means and ranges.

Six medical devices that had been cleared by the FDA between 2002 and 2012 for use in dermatology were explored, including systems for picosecond and fractionated lasers, monopolar and bipolar radiofrequency skin tightening, cryolipolysis, and nonablative radiofrequency skin resurfacing. Our search of these 6 representative dermatologic devices revealed 47,055 counterfeit product listings on Alibaba.com and DHgate.com. Upon searching these popular e-commerce websites using the device name as the search term, the number of listings varied considerably between the 2 e-commerce websites for the same device and from device to device on the same e-commerce website. On Alibaba.com, the greatest number of listings resulted for picosecond laser (23,622 listings), fractionated laser (15,269), and radiofrequency skin tightening devices (3555); cryolipolysis and nonablative radiofrequency resurfacing devices had notably fewer listings (35 and 38, respectively). On DHGate.com, a similar trend was noted with the most numerous listings for picosecond and fractionated laser systems (2429 and 1345, respectively).

Among the first 10 listings of products on Alibaba.com and DHgate.com for these 6 devices, 10.7% (11 of 103) had advertised claims of FDA clearance on the listing page. Of 103 counterfeit products, China was the country of origin for 100; South Korea for 2; and Thailand for 1. Unit pricing was heterogeneous between the 2 e-commerce websites for the counterfeit listings; pricing for duplicate fractionated laser systems was particularly dissimilar, with an average price on Alibab.com of US $8105.80 and an average price on DHgate.com of US $3409.14. Even on the same e-commerce website, the range of unit pricing differed greatly for dermatologic devices. For example, among the first 10 listings on Alibaba.com for a fractionated laser system, the price ranged from US $2300 to US $32,000.

Counterfeit medical devices are on the rise in dermatology.^1,3 Although devices such as radiofrequency and laser systems had thousands of knockoff listings on 2 e-commerce websites, other devices, such as cryolipolysis and body contouring systems, had fewer listings, suggesting heterogeneity in the prevalence of different counterfeit dermatologic devices on the market.

The varied pricing of the top 10 listings for each product and spurious claims of FDA clearance for some listings highlight the lack of regulatory authority over consistent product information on e-commerce websites. Furthermore, differences between characteristics of counterfeit device listings can impede efforts to trace suppliers and increase the opacity of counterfeit purchasing.

Three criteria have been proposed for a device to be considered counterfeit³:

• The device has no proven safety or efficacy among consumers. For example, the substantial threat of copycat devices in dermatology has been demonstrated by reports of burns caused by fake cryolipolysis devices.²

• The device violates patent rights or copy trademarks. Due to the regional nature of intellectual property rights, country-specific filings of patents and trademarks are required if protections are sought internationally. In this study, counterfeit devices originated in China, South Korea, and Thailand, where patent and trademark protections for the original devices do not extend.

• The device is falsely claimed to have been cleared by the FDA or other clinical regulatory authorities. Legitimate medical devices are subject to rounds of safety and compatibility testing using standards set by regulatory bodies, such as the FDA’s Center for Devices and Radiological Health, the International Organization of Standardization, and the International Electrotechnical Commission. Compliance with these safety standards is lost, however, among unregulated internet sales of medical devices. Our search revealed that 10.7% of the top 10 counterfeit device listings for each product explicitly mentioned FDA clearance in the product description. Among the thousands of listings on e-commerce sites, even a fraction that make spurious FDA-clearance claims can mislead consumers.

The issue of counterfeit medical devices has not gone unrecognized globally. In 2013, the World Health Organization created the Global Surveillance and Monitoring System to unify international efforts for reporting substandard, unlicensed, or falsified medical products.⁴ Although universal monitoring systems can improve detection of counterfeit products, we highlight the alarming continuing ease of purchasing counterfeit dermatologic devices through e-commerce websites. Due to the widespread nature of counterfeiting across all domains of medicine, the onus of curbing counterfeit dermatologic devices might be on dermatology providers to recognize and report such occurrences.

This exploration of counterfeit dermatologic devices revealed a lack of consistency throughout product listings on 2 popular e-commerce websites, Alibaba.com and DHgate.com. Given the alarming availability of these devices on the internet, practitioners should approach the purchase of any device with concern about counterfeiting. Future avenues of study might explore the prevalence of counterfeit devices used in dermatology practices and offer insight on regulation and consumer safety efforts.

Although the number of female graduates from US medical schools has steadily increased,¹ several studies since the 1970s indicate that a disparity exists in salary, academic rank, and promotion among female and male physicians across multiple specialties.^2-8 Proposed explanations include women working fewer hours, having lower productivity rates, undernegotiating compensation, and underbilling for the same services. However, when controlling for variables such as time, experience, specialty, rank, and research activities, this gap unequivocally persists. There are limited data on this topic in dermatology, a field in which women comprise more than half of the working population.^6,7 Most analyses of gender disparities in dermatology are based on data primarily from academic dermatologists, which may not be representative of the larger population of dermatologists.^8,9 The purpose of this study is to determine if an income disparity exists between male and female physicians in dermatology, including those in private practice and those who are specialty trained.

Methods

Population—We performed a cross-sectional self-reported survey to examine compensation of male and female board-certified dermatologists (MDs/DOs). Several populations of dermatologists were surveyed in August and September 2018. Approximately 20% of the members of the American Academy of Dermatology were randomly selected and sent a link to the survey. Additionally, a survey link was emailed to members of the Association of Professors of Dermatology, American College of Mohs Surgery, and American Society for Dermatologic Surgery. A link to the survey also was published on “The Board Certified Dermatologists” Facebook group.

Statistical Analysis—Descriptive statistics were used to summarize the distribution of variables overall and within gender (male or female). Not all respondents completed every section, and duplicates and incomplete responses were removed. Variables were compared between genders using t tests (continuous), the Pearson χ² test (nominal), or the Cochran-Mantel-Haenszel test (ordinal). For categorical variables with small cell counts, an exact χ² test for small samples was used. For continuous variables, t test P values were calculated using either pooled or Satterthwaithe approximation.

To analyze the effect of different variables on total income using multivariate and univariate linear regression, the income variable was transformed into a continuous variable by using midpoints of the categories. Univariate linear regression was used to assess the effect and significance of each variable on total annual income. Variables that were found to have a P value of less than .05 (α=.05) were deemed as significant predictors of total annual income. These variables were added to a multivariate linear regression model to determine their effect on income when adjusting for other significant (and approaching significance) factors. In addition, variables that were found to have a P value of less than .2 (α=.05) were added to the multivariate linear regression model to assess significance of these specific variables when adjusting for other factors. In this way, we tested and accounted for a multitude of variables as potential sources of confounding.

Results

Demographics—Our survey was emailed to 3079 members of the American Academy of Dermatology, and 277 responses were received. Approximately 144 additional responses were obtained collectively from links sent to the directories of the Association of Professors of Dermatology, American College of Mohs Surgery, and American Society for Dermatologic Surgery and from social media. Of these respondents, 53.65% (213/397) were female and 46.35% (184/397) were male. When stratifying by race/ethnicity, 77.33% identified as White; 13.85% identified as Asian; 6.3% identified as Black or African American, Hispanic/Latino, and Native American; and 2.52% chose not to respond. Although most male and female respondents were White, a significantly higher proportion of female respondents identified as Asian or Black/African American/Hispanic/Latino/Native American (P=.0006). We found that race/ethnicity did not significantly impact income (P=.2736). All US Census regions were represented in this study, and geographic distribution as well as population density of practice location (ie, rural, suburban, urban setting) did not differ significantly between males and females (P=.5982 and P=.1007, respectively) and did not significantly impact income (P=.3225 and P=.10663, respectively).

Income—Total annual income was defined as the aggregate sum of all types of financial compensation received in 1 calendar year (eg, salary, bonuses, benefits) and was elicited as an ordinal variable in income brackets of US $100,000. Overall, χ² analysis showed a statistically significant difference in annual total income between male and female dermatologists (P<.0001), with a higher proportion of males in the highest pay bracket (Figure). Gender remained a statistically significant predictor of income on both univariate and multivariate linear regression analyses (P=.0002 and P<.0001, respectively), indicating that gender has a significant impact on compensation, even after controlling for other variables (eTable). Of note, males in this sample were on average older and in practice longer than females (approximately 6 years, P<.0001). However, when univariate linear regression was performed, both age (P=.8281) and number of years since residency or fellowship completion (P=.8743) were not significant predictors of income.

Practice Type—There were no statistically significant differences between men and women in practice type (P=.1489), including academic/university, hospital based, and solo and group private practice; pay structure (P=.1437), including base salary, collection-based salary, or salary plus incentive; holding a supervisory role (P=.0846); or having ownership of a practice (P=.3565)(eTable). Most respondents were in solo or group private practice (58.2%) and had a component of productivity-based compensation (77.5%). In addition, 62% of private practice dermatologists (133/212) had an ownership interest in their practice. As expected, univariate and multivariate regression analyses showed that practice type, pay structure, supervisory roles, and employee vs ownership roles were significant predictors of income (P<.05)(eTable).

Work Productivity—Statistically significant differences were found between men and women in hours worked per week in direct patient care (P<.0001) and in patient visits per week (P=.0052), with a higher percentage of men working more than 40 hours per week and men seeing an average of approximately 22 more patients per week than women. In the subgroup of all dermatologists working more than 40 hours per week, a statistically significant difference in income persisted between males and females (P=.0001). Hours worked per week and patient visits per week were statistically significant predictors of income on both univariate and multivariate regression analyses (P<.05)(Table).

Education and Fellowship Training—No significant difference existed between males and females in type of undergraduate school attended, namely public or private institutions (P=.1090), but a significant difference existed within type of medical school education, with a higher percentage of females attending private medical schools (53.03%) compared to males (38.24%)(P=.0045). However, type of undergraduate or medical school attended had no impact on income (P=.9103). A higher percentage of males (27.32%) completed additional advanced degrees, such as a master of business administration or a master of public health, compared to females (16.9%)(P=.0122). However, the completion of additional advanced degrees had no significant impact on income (P=.2379). No statistical significance existed between males and females in number of residencies completed (P=.3236), and residencies completed had no significant impact on income (P=.4584).

Of 397 respondents, approximately one-third of respondents completed fellowship training (36.5%). Fellowships included dermatopathology, surgery/cosmetics, and other (encompassing complex medical, research, transplant, and pediatric dermatology). Although similar percentages of men and women completed fellowship training, men and women differed significantly by type of fellowship completed (P=.0188). There were similar rates of dermatopathology and surgical fellowship completion between genders but almost 3 times the number of females who completed other fellowships. Type of fellowship training was a statistically significant predictor of income on both univariate and multivariate regression analyses (P<.00001 and P<.0001, respectively).

Work Activity—Respondents were asked to estimate the amount of time devoted to general dermatology, dermatopathology, Mohs micrographic surgery, cosmetics, and dermatologic surgery in their practices (Table). Women devoted a significantly higher average percentage of time to cosmetics (7.89%) compared to men (4.52%)(P=.0097). The number of cosmetic procedures performed per week was not statistically significantly different between men and women (P=.8035) but was a significant factor for income on univariate regression analysis (P=.0002). Time spent performing dermatologic surgery, general dermatology, or Mohs micrographic surgery did not significantly differ between men and women but was found to significantly influence income.

Academic Dermatology—Among the respondents working in academic settings, χ² analysis identified a significant difference in the faculty rank between males and females, with a tendency for lower academic rank in females (P=.0508). Assistant professorship was comprised of 35% of men vs 51% of women, whereas full professorship consisted of 26% of men but only 13% of women. Academic rank was found to be a significant predictor of income, with higher rank associated with higher income (P<.0001 on univariate regression analysis). However, when adjusting for other factors, academic rank was no longer a significant predictor of income (P=.0840 on multivariate regression analysis). No significant difference existed between men and women in funding received from the National Institutes of Health, conduction of clinical trials, or authorship of scientific publications, and these factors were not found to have a significant impact on income.

Work Leave—Male and female dermatologists showed a statistically significant difference in maternity or Family and Medical Leave Act (FMLA) leave taken over their careers, with 56.03% of females reporting leave taken compared to 6.78% of males (P<.0001). Women reported a significantly higher average number of weeks of maternity or FMLA leave taken over their careers (12.92 weeks) compared to men (2.42 weeks) (P<.0001). However, upon univariate regression analysis, whether or not maternity or FMLA leave was taken over their careers (P=.2005), the number of times that maternity or FMLA leave was taken (P=.4350), and weeks of maternity or FMLA leave taken (P=.4057) were all not significant predictors of income.

Comment

This study sought to investigate the relationship between income and gender in dermatology, and our results demonstrated that statistically significant differences in total annual income exist between male and female dermatologists, with male dermatologists earning a significantly higher income, approximately an additional $80,000. Our results are consistent with other studies of US physician income, which have found a gender gap ranging from $13,399 to $82,000 that persists even when controlling for factors such as specialty choice, practice setting, rank and role in practice, work hours, vacation/leave taken, and others.^2-7,10-15

There was a significant difference in rank of male and female academic dermatologists, with fewer females at higher academic ranks. These results are consistent with numerous studies in academic dermatology that show underrepresentation of women at higher academic ranks and leadership positions.^8,9,16-18 Poor negotiation may contribute to differences in both rank and income.^19,20 There are conflicting data on research productivity of academic dermatologists and length of career, first and senior authorship, and quality and academic impact, all of which add complexity to this topic.^{8,9,12,16-18,20-23}Male and female dermatologists reported significant differences in productivity, with male dermatologists working more hours and seeing more patients per week than female dermatologists. These results are consistent with other studies of dermatologists^4,24 and other physicians.¹² Regardless, gender was still found to have a significant impact on income even when controlling for differences in productivity and FMLA leave taken. These results are consistent with numerous studies of US physicians that found a gender gap in income even when controlling for hours worked.^12,23 Although fellowship training as a whole was found to significantly impact income, our results do not characterize whether the impact on income was positive or negative for each type of fellowship. Fellowship training in specialties such as internal medicine or general surgery likewise has variable effects on income.^24,25

A comprehensive survey design and significant data elicited from dermatologists working in private practice for the first time served as the main strengths of this study. Limitations included self-reported design, categorical ranges, and limited sample size in subgroups. Future directions include deeper analysis of subgroups, including fellowship-trained dermatologists, dermatologists working more than 40 hours per week, and female dermatologists by race/ethnicity.

Conclusion

We have demonstrated that self-reported discrepancies in salary between male and female dermatologists exist, with male dermatologists earning a significantly higher annual salary than their female counterparts. This study identified and stratified several career factors that comprise the broad field and practice of dermatology. Even when controlling for these variations, we have demonstrated that gender alone remains a significant predictor of income, indicating that an unexplained income gap between the 2 genders exists in dermatology.

Although the number of female graduates from US medical schools has steadily increased,¹ several studies since the 1970s indicate that a disparity exists in salary, academic rank, and promotion among female and male physicians across multiple specialties.^2-8 Proposed explanations include women working fewer hours, having lower productivity rates, undernegotiating compensation, and underbilling for the same services. However, when controlling for variables such as time, experience, specialty, rank, and research activities, this gap unequivocally persists. There are limited data on this topic in dermatology, a field in which women comprise more than half of the working population.^6,7 Most analyses of gender disparities in dermatology are based on data primarily from academic dermatologists, which may not be representative of the larger population of dermatologists.^8,9 The purpose of this study is to determine if an income disparity exists between male and female physicians in dermatology, including those in private practice and those who are specialty trained.

Methods

Population—We performed a cross-sectional self-reported survey to examine compensation of male and female board-certified dermatologists (MDs/DOs). Several populations of dermatologists were surveyed in August and September 2018. Approximately 20% of the members of the American Academy of Dermatology were randomly selected and sent a link to the survey. Additionally, a survey link was emailed to members of the Association of Professors of Dermatology, American College of Mohs Surgery, and American Society for Dermatologic Surgery. A link to the survey also was published on “The Board Certified Dermatologists” Facebook group.

Statistical Analysis—Descriptive statistics were used to summarize the distribution of variables overall and within gender (male or female). Not all respondents completed every section, and duplicates and incomplete responses were removed. Variables were compared between genders using t tests (continuous), the Pearson χ² test (nominal), or the Cochran-Mantel-Haenszel test (ordinal). For categorical variables with small cell counts, an exact χ² test for small samples was used. For continuous variables, t test P values were calculated using either pooled or Satterthwaithe approximation.

To analyze the effect of different variables on total income using multivariate and univariate linear regression, the income variable was transformed into a continuous variable by using midpoints of the categories. Univariate linear regression was used to assess the effect and significance of each variable on total annual income. Variables that were found to have a P value of less than .05 (α=.05) were deemed as significant predictors of total annual income. These variables were added to a multivariate linear regression model to determine their effect on income when adjusting for other significant (and approaching significance) factors. In addition, variables that were found to have a P value of less than .2 (α=.05) were added to the multivariate linear regression model to assess significance of these specific variables when adjusting for other factors. In this way, we tested and accounted for a multitude of variables as potential sources of confounding.

Results

Demographics—Our survey was emailed to 3079 members of the American Academy of Dermatology, and 277 responses were received. Approximately 144 additional responses were obtained collectively from links sent to the directories of the Association of Professors of Dermatology, American College of Mohs Surgery, and American Society for Dermatologic Surgery and from social media. Of these respondents, 53.65% (213/397) were female and 46.35% (184/397) were male. When stratifying by race/ethnicity, 77.33% identified as White; 13.85% identified as Asian; 6.3% identified as Black or African American, Hispanic/Latino, and Native American; and 2.52% chose not to respond. Although most male and female respondents were White, a significantly higher proportion of female respondents identified as Asian or Black/African American/Hispanic/Latino/Native American (P=.0006). We found that race/ethnicity did not significantly impact income (P=.2736). All US Census regions were represented in this study, and geographic distribution as well as population density of practice location (ie, rural, suburban, urban setting) did not differ significantly between males and females (P=.5982 and P=.1007, respectively) and did not significantly impact income (P=.3225 and P=.10663, respectively).

Income—Total annual income was defined as the aggregate sum of all types of financial compensation received in 1 calendar year (eg, salary, bonuses, benefits) and was elicited as an ordinal variable in income brackets of US $100,000. Overall, χ² analysis showed a statistically significant difference in annual total income between male and female dermatologists (P<.0001), with a higher proportion of males in the highest pay bracket (Figure). Gender remained a statistically significant predictor of income on both univariate and multivariate linear regression analyses (P=.0002 and P<.0001, respectively), indicating that gender has a significant impact on compensation, even after controlling for other variables (eTable). Of note, males in this sample were on average older and in practice longer than females (approximately 6 years, P<.0001). However, when univariate linear regression was performed, both age (P=.8281) and number of years since residency or fellowship completion (P=.8743) were not significant predictors of income.

Practice Type—There were no statistically significant differences between men and women in practice type (P=.1489), including academic/university, hospital based, and solo and group private practice; pay structure (P=.1437), including base salary, collection-based salary, or salary plus incentive; holding a supervisory role (P=.0846); or having ownership of a practice (P=.3565)(eTable). Most respondents were in solo or group private practice (58.2%) and had a component of productivity-based compensation (77.5%). In addition, 62% of private practice dermatologists (133/212) had an ownership interest in their practice. As expected, univariate and multivariate regression analyses showed that practice type, pay structure, supervisory roles, and employee vs ownership roles were significant predictors of income (P<.05)(eTable).

Work Productivity—Statistically significant differences were found between men and women in hours worked per week in direct patient care (P<.0001) and in patient visits per week (P=.0052), with a higher percentage of men working more than 40 hours per week and men seeing an average of approximately 22 more patients per week than women. In the subgroup of all dermatologists working more than 40 hours per week, a statistically significant difference in income persisted between males and females (P=.0001). Hours worked per week and patient visits per week were statistically significant predictors of income on both univariate and multivariate regression analyses (P<.05)(Table).

Education and Fellowship Training—No significant difference existed between males and females in type of undergraduate school attended, namely public or private institutions (P=.1090), but a significant difference existed within type of medical school education, with a higher percentage of females attending private medical schools (53.03%) compared to males (38.24%)(P=.0045). However, type of undergraduate or medical school attended had no impact on income (P=.9103). A higher percentage of males (27.32%) completed additional advanced degrees, such as a master of business administration or a master of public health, compared to females (16.9%)(P=.0122). However, the completion of additional advanced degrees had no significant impact on income (P=.2379). No statistical significance existed between males and females in number of residencies completed (P=.3236), and residencies completed had no significant impact on income (P=.4584).

Of 397 respondents, approximately one-third of respondents completed fellowship training (36.5%). Fellowships included dermatopathology, surgery/cosmetics, and other (encompassing complex medical, research, transplant, and pediatric dermatology). Although similar percentages of men and women completed fellowship training, men and women differed significantly by type of fellowship completed (P=.0188). There were similar rates of dermatopathology and surgical fellowship completion between genders but almost 3 times the number of females who completed other fellowships. Type of fellowship training was a statistically significant predictor of income on both univariate and multivariate regression analyses (P<.00001 and P<.0001, respectively).

Work Activity—Respondents were asked to estimate the amount of time devoted to general dermatology, dermatopathology, Mohs micrographic surgery, cosmetics, and dermatologic surgery in their practices (Table). Women devoted a significantly higher average percentage of time to cosmetics (7.89%) compared to men (4.52%)(P=.0097). The number of cosmetic procedures performed per week was not statistically significantly different between men and women (P=.8035) but was a significant factor for income on univariate regression analysis (P=.0002). Time spent performing dermatologic surgery, general dermatology, or Mohs micrographic surgery did not significantly differ between men and women but was found to significantly influence income.

Academic Dermatology—Among the respondents working in academic settings, χ² analysis identified a significant difference in the faculty rank between males and females, with a tendency for lower academic rank in females (P=.0508). Assistant professorship was comprised of 35% of men vs 51% of women, whereas full professorship consisted of 26% of men but only 13% of women. Academic rank was found to be a significant predictor of income, with higher rank associated with higher income (P<.0001 on univariate regression analysis). However, when adjusting for other factors, academic rank was no longer a significant predictor of income (P=.0840 on multivariate regression analysis). No significant difference existed between men and women in funding received from the National Institutes of Health, conduction of clinical trials, or authorship of scientific publications, and these factors were not found to have a significant impact on income.

Work Leave—Male and female dermatologists showed a statistically significant difference in maternity or Family and Medical Leave Act (FMLA) leave taken over their careers, with 56.03% of females reporting leave taken compared to 6.78% of males (P<.0001). Women reported a significantly higher average number of weeks of maternity or FMLA leave taken over their careers (12.92 weeks) compared to men (2.42 weeks) (P<.0001). However, upon univariate regression analysis, whether or not maternity or FMLA leave was taken over their careers (P=.2005), the number of times that maternity or FMLA leave was taken (P=.4350), and weeks of maternity or FMLA leave taken (P=.4057) were all not significant predictors of income.

Comment

This study sought to investigate the relationship between income and gender in dermatology, and our results demonstrated that statistically significant differences in total annual income exist between male and female dermatologists, with male dermatologists earning a significantly higher income, approximately an additional $80,000. Our results are consistent with other studies of US physician income, which have found a gender gap ranging from $13,399 to $82,000 that persists even when controlling for factors such as specialty choice, practice setting, rank and role in practice, work hours, vacation/leave taken, and others.^2-7,10-15

There was a significant difference in rank of male and female academic dermatologists, with fewer females at higher academic ranks. These results are consistent with numerous studies in academic dermatology that show underrepresentation of women at higher academic ranks and leadership positions.^8,9,16-18 Poor negotiation may contribute to differences in both rank and income.^19,20 There are conflicting data on research productivity of academic dermatologists and length of career, first and senior authorship, and quality and academic impact, all of which add complexity to this topic.^{8,9,12,16-18,20-23}Male and female dermatologists reported significant differences in productivity, with male dermatologists working more hours and seeing more patients per week than female dermatologists. These results are consistent with other studies of dermatologists^4,24 and other physicians.¹² Regardless, gender was still found to have a significant impact on income even when controlling for differences in productivity and FMLA leave taken. These results are consistent with numerous studies of US physicians that found a gender gap in income even when controlling for hours worked.^12,23 Although fellowship training as a whole was found to significantly impact income, our results do not characterize whether the impact on income was positive or negative for each type of fellowship. Fellowship training in specialties such as internal medicine or general surgery likewise has variable effects on income.^24,25

A comprehensive survey design and significant data elicited from dermatologists working in private practice for the first time served as the main strengths of this study. Limitations included self-reported design, categorical ranges, and limited sample size in subgroups. Future directions include deeper analysis of subgroups, including fellowship-trained dermatologists, dermatologists working more than 40 hours per week, and female dermatologists by race/ethnicity.

Conclusion

We have demonstrated that self-reported discrepancies in salary between male and female dermatologists exist, with male dermatologists earning a significantly higher annual salary than their female counterparts. This study identified and stratified several career factors that comprise the broad field and practice of dermatology. Even when controlling for these variations, we have demonstrated that gender alone remains a significant predictor of income, indicating that an unexplained income gap between the 2 genders exists in dermatology.

Although the number of female graduates from US medical schools has steadily increased,¹ several studies since the 1970s indicate that a disparity exists in salary, academic rank, and promotion among female and male physicians across multiple specialties.^2-8 Proposed explanations include women working fewer hours, having lower productivity rates, undernegotiating compensation, and underbilling for the same services. However, when controlling for variables such as time, experience, specialty, rank, and research activities, this gap unequivocally persists. There are limited data on this topic in dermatology, a field in which women comprise more than half of the working population.^6,7 Most analyses of gender disparities in dermatology are based on data primarily from academic dermatologists, which may not be representative of the larger population of dermatologists.^8,9 The purpose of this study is to determine if an income disparity exists between male and female physicians in dermatology, including those in private practice and those who are specialty trained.

Methods

Population—We performed a cross-sectional self-reported survey to examine compensation of male and female board-certified dermatologists (MDs/DOs). Several populations of dermatologists were surveyed in August and September 2018. Approximately 20% of the members of the American Academy of Dermatology were randomly selected and sent a link to the survey. Additionally, a survey link was emailed to members of the Association of Professors of Dermatology, American College of Mohs Surgery, and American Society for Dermatologic Surgery. A link to the survey also was published on “The Board Certified Dermatologists” Facebook group.

Statistical Analysis—Descriptive statistics were used to summarize the distribution of variables overall and within gender (male or female). Not all respondents completed every section, and duplicates and incomplete responses were removed. Variables were compared between genders using t tests (continuous), the Pearson χ² test (nominal), or the Cochran-Mantel-Haenszel test (ordinal). For categorical variables with small cell counts, an exact χ² test for small samples was used. For continuous variables, t test P values were calculated using either pooled or Satterthwaithe approximation.

To analyze the effect of different variables on total income using multivariate and univariate linear regression, the income variable was transformed into a continuous variable by using midpoints of the categories. Univariate linear regression was used to assess the effect and significance of each variable on total annual income. Variables that were found to have a P value of less than .05 (α=.05) were deemed as significant predictors of total annual income. These variables were added to a multivariate linear regression model to determine their effect on income when adjusting for other significant (and approaching significance) factors. In addition, variables that were found to have a P value of less than .2 (α=.05) were added to the multivariate linear regression model to assess significance of these specific variables when adjusting for other factors. In this way, we tested and accounted for a multitude of variables as potential sources of confounding.

Results

Demographics—Our survey was emailed to 3079 members of the American Academy of Dermatology, and 277 responses were received. Approximately 144 additional responses were obtained collectively from links sent to the directories of the Association of Professors of Dermatology, American College of Mohs Surgery, and American Society for Dermatologic Surgery and from social media. Of these respondents, 53.65% (213/397) were female and 46.35% (184/397) were male. When stratifying by race/ethnicity, 77.33% identified as White; 13.85% identified as Asian; 6.3% identified as Black or African American, Hispanic/Latino, and Native American; and 2.52% chose not to respond. Although most male and female respondents were White, a significantly higher proportion of female respondents identified as Asian or Black/African American/Hispanic/Latino/Native American (P=.0006). We found that race/ethnicity did not significantly impact income (P=.2736). All US Census regions were represented in this study, and geographic distribution as well as population density of practice location (ie, rural, suburban, urban setting) did not differ significantly between males and females (P=.5982 and P=.1007, respectively) and did not significantly impact income (P=.3225 and P=.10663, respectively).

Income—Total annual income was defined as the aggregate sum of all types of financial compensation received in 1 calendar year (eg, salary, bonuses, benefits) and was elicited as an ordinal variable in income brackets of US $100,000. Overall, χ² analysis showed a statistically significant difference in annual total income between male and female dermatologists (P<.0001), with a higher proportion of males in the highest pay bracket (Figure). Gender remained a statistically significant predictor of income on both univariate and multivariate linear regression analyses (P=.0002 and P<.0001, respectively), indicating that gender has a significant impact on compensation, even after controlling for other variables (eTable). Of note, males in this sample were on average older and in practice longer than females (approximately 6 years, P<.0001). However, when univariate linear regression was performed, both age (P=.8281) and number of years since residency or fellowship completion (P=.8743) were not significant predictors of income.

Practice Type—There were no statistically significant differences between men and women in practice type (P=.1489), including academic/university, hospital based, and solo and group private practice; pay structure (P=.1437), including base salary, collection-based salary, or salary plus incentive; holding a supervisory role (P=.0846); or having ownership of a practice (P=.3565)(eTable). Most respondents were in solo or group private practice (58.2%) and had a component of productivity-based compensation (77.5%). In addition, 62% of private practice dermatologists (133/212) had an ownership interest in their practice. As expected, univariate and multivariate regression analyses showed that practice type, pay structure, supervisory roles, and employee vs ownership roles were significant predictors of income (P<.05)(eTable).

Work Productivity—Statistically significant differences were found between men and women in hours worked per week in direct patient care (P<.0001) and in patient visits per week (P=.0052), with a higher percentage of men working more than 40 hours per week and men seeing an average of approximately 22 more patients per week than women. In the subgroup of all dermatologists working more than 40 hours per week, a statistically significant difference in income persisted between males and females (P=.0001). Hours worked per week and patient visits per week were statistically significant predictors of income on both univariate and multivariate regression analyses (P<.05)(Table).

Education and Fellowship Training—No significant difference existed between males and females in type of undergraduate school attended, namely public or private institutions (P=.1090), but a significant difference existed within type of medical school education, with a higher percentage of females attending private medical schools (53.03%) compared to males (38.24%)(P=.0045). However, type of undergraduate or medical school attended had no impact on income (P=.9103). A higher percentage of males (27.32%) completed additional advanced degrees, such as a master of business administration or a master of public health, compared to females (16.9%)(P=.0122). However, the completion of additional advanced degrees had no significant impact on income (P=.2379). No statistical significance existed between males and females in number of residencies completed (P=.3236), and residencies completed had no significant impact on income (P=.4584).

Of 397 respondents, approximately one-third of respondents completed fellowship training (36.5%). Fellowships included dermatopathology, surgery/cosmetics, and other (encompassing complex medical, research, transplant, and pediatric dermatology). Although similar percentages of men and women completed fellowship training, men and women differed significantly by type of fellowship completed (P=.0188). There were similar rates of dermatopathology and surgical fellowship completion between genders but almost 3 times the number of females who completed other fellowships. Type of fellowship training was a statistically significant predictor of income on both univariate and multivariate regression analyses (P<.00001 and P<.0001, respectively).

Work Activity—Respondents were asked to estimate the amount of time devoted to general dermatology, dermatopathology, Mohs micrographic surgery, cosmetics, and dermatologic surgery in their practices (Table). Women devoted a significantly higher average percentage of time to cosmetics (7.89%) compared to men (4.52%)(P=.0097). The number of cosmetic procedures performed per week was not statistically significantly different between men and women (P=.8035) but was a significant factor for income on univariate regression analysis (P=.0002). Time spent performing dermatologic surgery, general dermatology, or Mohs micrographic surgery did not significantly differ between men and women but was found to significantly influence income.

Academic Dermatology—Among the respondents working in academic settings, χ² analysis identified a significant difference in the faculty rank between males and females, with a tendency for lower academic rank in females (P=.0508). Assistant professorship was comprised of 35% of men vs 51% of women, whereas full professorship consisted of 26% of men but only 13% of women. Academic rank was found to be a significant predictor of income, with higher rank associated with higher income (P<.0001 on univariate regression analysis). However, when adjusting for other factors, academic rank was no longer a significant predictor of income (P=.0840 on multivariate regression analysis). No significant difference existed between men and women in funding received from the National Institutes of Health, conduction of clinical trials, or authorship of scientific publications, and these factors were not found to have a significant impact on income.

Work Leave—Male and female dermatologists showed a statistically significant difference in maternity or Family and Medical Leave Act (FMLA) leave taken over their careers, with 56.03% of females reporting leave taken compared to 6.78% of males (P<.0001). Women reported a significantly higher average number of weeks of maternity or FMLA leave taken over their careers (12.92 weeks) compared to men (2.42 weeks) (P<.0001). However, upon univariate regression analysis, whether or not maternity or FMLA leave was taken over their careers (P=.2005), the number of times that maternity or FMLA leave was taken (P=.4350), and weeks of maternity or FMLA leave taken (P=.4057) were all not significant predictors of income.

Comment

This study sought to investigate the relationship between income and gender in dermatology, and our results demonstrated that statistically significant differences in total annual income exist between male and female dermatologists, with male dermatologists earning a significantly higher income, approximately an additional $80,000. Our results are consistent with other studies of US physician income, which have found a gender gap ranging from $13,399 to $82,000 that persists even when controlling for factors such as specialty choice, practice setting, rank and role in practice, work hours, vacation/leave taken, and others.^2-7,10-15

There was a significant difference in rank of male and female academic dermatologists, with fewer females at higher academic ranks. These results are consistent with numerous studies in academic dermatology that show underrepresentation of women at higher academic ranks and leadership positions.^8,9,16-18 Poor negotiation may contribute to differences in both rank and income.^19,20 There are conflicting data on research productivity of academic dermatologists and length of career, first and senior authorship, and quality and academic impact, all of which add complexity to this topic.^{8,9,12,16-18,20-23}Male and female dermatologists reported significant differences in productivity, with male dermatologists working more hours and seeing more patients per week than female dermatologists. These results are consistent with other studies of dermatologists^4,24 and other physicians.¹² Regardless, gender was still found to have a significant impact on income even when controlling for differences in productivity and FMLA leave taken. These results are consistent with numerous studies of US physicians that found a gender gap in income even when controlling for hours worked.^12,23 Although fellowship training as a whole was found to significantly impact income, our results do not characterize whether the impact on income was positive or negative for each type of fellowship. Fellowship training in specialties such as internal medicine or general surgery likewise has variable effects on income.^24,25

A comprehensive survey design and significant data elicited from dermatologists working in private practice for the first time served as the main strengths of this study. Limitations included self-reported design, categorical ranges, and limited sample size in subgroups. Future directions include deeper analysis of subgroups, including fellowship-trained dermatologists, dermatologists working more than 40 hours per week, and female dermatologists by race/ethnicity.

Conclusion

We have demonstrated that self-reported discrepancies in salary between male and female dermatologists exist, with male dermatologists earning a significantly higher annual salary than their female counterparts. This study identified and stratified several career factors that comprise the broad field and practice of dermatology. Even when controlling for these variations, we have demonstrated that gender alone remains a significant predictor of income, indicating that an unexplained income gap between the 2 genders exists in dermatology.

Although the universal use of masks by both health care professionals and the general public now appears routine, widely differing recommendations were distributed by different health organizations early in the pandemic. In April 2020, the World Health Organization (WHO) stated that there was no evidence that healthy individuals wearing a medical mask in the community prevented COVID-19 infection.¹ However, these recommendations must be placed in the context of a national shortage of personal protective equipment early in the pandemic. The WHO guidance released on June 5, 2020, recommended continuous use of masks for health care workers in the clinical setting.² Additional recommendations included mask replacement when wet, soiled, or damaged, and when the wearer touched the mask. The WHO also recommended mask usage by those with underlying medical comorbidities and those living in high population–density areas and in settings where physical distancing was not possible.²

The Centers for Disease Control and Prevention (CDC) officially recommended the use of face coverings for the general public to prevent COVID-19 transmission on April 3, 2020.³ The CDC highlighted that masks should not be worn by children younger than 2 years; individuals with respiratory compromise; and patients who are unconscious, incapacitated, or unable to remove a mask without assistance.⁴ Medical masks and respirators were only recommended for health care workers. Importantly, masks with valves/vents were not recommended, as respiratory droplets can be emitted, defeating the purpose of source control.⁴ New York State mandated mask usage in public places starting on April 15, 2020.

These recommendations were based on the hypothesis that COVID-19 transmission occurs primarily via droplets and contact. In reality, SARS-CoV-2 transmission more likely occurs in a continuum from larger droplets to miniscule aerosols expelled from an infected person when talking, coughing, or sneezing.^5,6 It should be noted that there was a formal suggestion of the potential for airborne transmission of SARS-CoV-2 by the CDC in a statement on September 18, 2020, that was subsequently retracted 3 days later.^7,8 The CDC, reversing their prior recommendations, updated their guidance on October 5, 2020, endorsing prior reports that SARS-CoV-2 can be spread through aerosol transmission.⁸

Mask usage helps prevent viral spread by all individuals, especially those who are presymptomatic and asymptomatic. Presymptomatic individuals account for approximately 40% to 60% of transmissions, and asymptomatic individuals account for approximately 4% to 30% of infections by some models, which suggest these individuals are the drivers of the pandemic, more so than symptomatic individuals.^9-15 Additionally, masking also may in effect reduce the amount of SARS-CoV-2 to which individuals are being exposed in the community.¹⁴ Universal masking is a relatively low-cost, low-risk intervention that may provide moderate benefit to the individual but substantial benefit to communities at large.^10-13 Universal masking in other countries also has clearly demonstrated major benefits during the pandemic. Implementation of universal masking in Taiwan resulted in only approximately 440 COVID-19 cases and less than 10 deaths, despite a population of 23 million.¹⁶ South Korea, having experience with Middle East respiratory syndrome, also was able to quickly institute a mask policy for its citizens, resulting in approximately 94% compliance.¹⁷ Moreover, several mathematical models have shown that even imperfect use of masks on a population level can prevent disease transmission and should be instituted.¹⁸

Given the importance and potential benefits of mask usage, we investigated compliance and proper utilization of facial masks in New York City (NYC), once the epicenter of the pandemic in the United States. New York City and the rest of New York State experienced more than 1.13 million and 1.46 million cases of COVID-19, respectively, as of early November 2021.¹⁹ Nationwide, NYC had the greatest absolute death count of more than 34,634 and the greatest rate of death per 100,000 individuals of 412. In contrast, New York State, excluding NYC, had an absolute death count of more than 21,646 and a death rate per 100,000 individuals of 195 as of early November 2021.¹⁹ Now entering 20 months since the first case of COVID-19 in NYC, it continues to be vital for facial mask protocols to be emphasized as part of a comprehensive infection prevention protocol, especially in light of continued vaccine resistance, to help stall continued spread of SARS-CoV-2.²⁰

We seek to show that despite months of policies for universal masking in NYC, there is still considerable mask noncompliance by the general public in health care settings where the use of masks is particularly imperative. We conducted an observational study investigating proper use of face masks of adults entering the main entrance of 4 hospitals located in NYC.

Methods

We observed mask usage in adults entering 4 hospitals in September 2020 (postsurge in NYC and prior to the availability of COVID-19 vaccinations). Hospitals were chosen to represent several types of health care delivery systems available in the United States and included a city, state, federal, and private hospital. Data collection was completed during peak traffic hours (8:00 am to 12:00 pm) on a weekday and continued until a total of 100 unique patients were observed at each site. Each hospital entrance was barricaded, and hospital staff were stationed at these entry points to take each individual’s temperature, screen for symptoms and exposure risk, verify patients’ appointments, and ensure proper mask wearing (in optimal circumstances). Data collectors (J.L. and N.M.) were stationed just past the barricade of each hospital’s entrance and observed those who entered. Individuals were not approached about the study, demographics, or the use and/or views about usage of facial masks. Children and hospital employees were excluded from data collection, with the exception of 1 hospital with a dedicated employee entrance where employees were observed for mask compliance. Except for vented/valved masks or makeshift masks fashioned out of scarfs, bandanas, or similar materials, the type of mask an individual wore was not distinguished (medical masks, cotton masks, or respirator-type masks were not differentiated).

Mask usage was observed and classified into several categories: correctly fitting mask over the nose and mouth, no face mask, mask usage with nose exposed, mask usage with mouth exposed, mask usage with both nose and mouth exposed (ie, mask on the chin/neck area), loosely fitting mask, vented/valved mask, or other form of face covering (eg, bandana, scarf).

Results

We observed a consistent rate of mask compliance between 72% and 85%, with an average of 78% of the 600 individuals observed wearing correctly fitting masks across the 4 hospitals included in this study (Table). The employee entrance included in this study had the highest compliance rate of 85%. An overall low rate of complete mask noncompliance was observed, with only 9 individuals (1.5%) in the entire study not wearing any mask. The federal hospital had the highest rate of mask noncompliance. We also observed a low rate of nose and mouth exposure, with 1.8% of individuals wearing a mask with the nose and mouth exposed (ie, mask tucked under the chin). No individuals were observed with the mouth exposed but with the nose covered by a mask. Additionally, only 3 individuals (0.5%) wore a mask with a vent/valve. The most common way that masks were worn incorrectly was with the nose exposed, accounting for 9.5% of individuals observed. Overall, only 9 individuals (1.5%) wore a nontraditional face covering, with a bandana being the most commonly observed makeshift mask.

Signage regarding the requirement to wear masks and to social distance was universally instituted at all hospital entry points (both inside and outside the hospital) in this study. However, there were no illustrations demonstrating correct and incorrect forms of mask usage. All signage merely displayed a graphic of a facial mask noting the requirement to wear a mask prior to entering the building. Hospital staff also had face masks available for patients who failed to bring a mask or who wore an inappropriate mask (ie, vented/valved masks).

Comment

Mask Effectiveness—Masks reduce the spread of SARS-CoV-2 by preventing both droplets and potentially virus-bearing aerosols.^6,21,22 It has been demonstrated that well-fitted cotton homemade masks and medical masks provide the most effective method of reducing droplet dispersion. Loosely fitted masks as well as bandana-style facial coverings minimally reduce small aerosolized droplets, and an uncovered mouth and nose can disperse particles at a distance much greater than 6 feet.²²

Mask Compliance—We report an overall high compliance rate with mask wearing among individuals visiting a hospital; however, compliance was still imperfect. Overall, 78% of observed individuals wore a correctly fitting mask when entering a hospital, even with hospital staff positioned at entry points to ensure proper mask usage. With all the resources available at health care centers, we anticipated a much higher compliance rate for correctly fitting masks at hospital entrances. We hypothesize that given only 78% of individuals showed proper mask compliance in a setting with enforcement by health care personnel, the mask compliance rate in the larger community is likely much lower. It is imperative to enforce continued mask compliance in medical centers and other public areas given notable vaccine noncompliance in certain parts of the country.

Tools to Prevent Disease Transmission—Mask usage by the general public in NYC helped in its response to the COVID-19 pandemic. Yang et al²³ demonstrated through mathematical modeling that mask usage in NYC was associated with a 6.6% reduction in transmission overall and a 20% decrease in transmission for individuals 65 years and older during the first month of the universal mask policy going into effect. The authors extrapolated these data during the NYC reopening and found that universal masking reduced transmission by approximately 9% to 11%, accounting for the increase in hours spent outside home quarantine. The authors also hypothesized that if universal masking was as effective in its reduction of transmission for everyone in NYC as it was for older adults, the potential reduction in transmission of SARS-CoV-2 could be as high as 28% to 32%.²³

Temperature checks at entrance barricades were standard protocol during the observation period. Although the main purpose of this study was to investigate compliance with and proper use of facial masks in a health care setting, it should be mentioned that, although temperature checks were being done on almost every person entering a hospital, the uniformity and practicality of this intervention has not been backed by substantial evidence. Although many nontouch thermometers are intended to capture a forehead temperature for the most accurate reading, the authors will share that in their observation, medical personnel screening individuals at hospital entrances were observed checking temperatures at any easily accessible body part, such as the forearm, hand, or neck. Furthermore, it has been reported that only approximately 40% of individuals with COVID-19 present with a fever.²⁴ Many hospitals, including the 4 that were included in this investigation, have formal protocols for patients presenting with a fever, especially those presenting to an ambulatory center. Patients are usually instructed to call ahead if they have a fever, and a decision regarding next steps will be discussed with a health care provider. In addition, 1 meta-analysis on the symptoms of COVID-19 suggested that approximately 12% of infected patients are asymptomatic, likely a conservative estimate.²⁵ Although we do not suggest that hospitals stop temperature checks, consistent temperature checks in anatomic locations intended for the specific thermometer used must be employed. Alternatively, a thermographic camera system that could detect heat signatures may be a way to screen faster, only necessitating that those above a threshold be assessed further.

The results of this study suggest that much greater effort is being placed on these temperature checks than on other equally important components of the entrance health assessment. This initial encounter at hospital entrances should serve as an opportunity for education on proper choice and use of masks with clear instructions that masks should not be removed unless directed by a health care provider and in a designated area, such as an examination room. The COVID-19 pandemic in the United States is likely the first time an individual is wearing these types of masks. Reiterating when and how often a mask should be changed (eg, when wet or soiled), how a soiled mask is not an effective mask, how a used mask should be discarded, ways to prevent self-contamination (ie, proper donning and doffing), and the importance of other infection-prevention behaviors—hand hygiene; social distancing; avoidance of touching the eyes, nose, and mouth with unwashed hands; and regular disinfecting of surfaces—should be practiced.^11,26-29 Extended use and reuse of masks also can result in transmission of infection.³⁰

Throughout the pandemic, our personal experience is that some patients often overtly refuse to wear a mask, citing underlying respiratory issues. The implications of patients not wearing a mask in a medical office and endangering other patients and staff are beyond the scope of this analysis. We will, however, comment briefly on the evidence behind this common concern. Matuschek et al³¹ found substantial adverse changes in respiratory rate, oxygen saturation, and CO₂ levels in patients with severe chronic obstructive pulmonary disease who were wearing N95 respirators during a 6-minute walk test. Another study by Chan et al³² showed that nonmedical masks in healthy older adults in the community setting had no impact on oxygen saturation. Ultimately, the most effective mask a patient can wear is a mask that will be worn consistently.³²

Populations With Limited Access to Masks—The COVID-19 pandemic disproportionately impacted disadvantaged populations, both in socioeconomic status and minority status. A disproportionate number of COVID-19 hospitalizations and deaths occurred in lower-income and minority populations.¹⁰ In fact, Lamb et al³³ reported that NYC neighborhoods with a larger proportion of uninsured individuals with limited access to health care and overall lower socioeconomic status had a higher rate of SARS-CoV-2 positivity. A retrospective study in Louisiana showed that Black individuals accounted for 77% of hospitalizations and 71% of deaths due to COVID-19 in a population where only 31% of individuals identified as Black.¹⁰ Chu et al⁶ even asserted that policies should be put into place to address equity issues for populations with limited access to masks. We agree that policies should be put into action to ensure that individuals lacking the means to obtain appropriate masks or unable to obtain an adequate supply of masks be provided this new necessity. It has been calculated that the impact of masks in reducing virus transmission would be greatest if mask availability to disadvantaged populations is ensured.¹⁸ We support a plan for masks to be covered by government-sponsored health plans.

Study Limitations—Several limitations exist in our study that should be discussed. Although the data collectors observed a large number of individuals, each hospital entrance was only observed for 1 half-day morning session. There may be variations in the number of people wearing a mask at different times of day and different days of the week with fluctuations in hospital traffic. Although data were collected at a variety of hospitals representing the diverse health care delivery models available in the United States, the NYC hospitals included in this study may have different resources available for infection-prevention strategies than hospitals across the country, given NYC’s unique population density and demographics.

Study Strengths—The generalizability of the study should be recognized. Data were collected by all major health care delivery models available in the United States—private, state, city, and federal hospital systems. This study can be easily replicated in other health care delivery systems to further investigate potential gaps in mask usage and infection prevention. Repeating this study in areas where a large portion of the population does not believe in the virus also will likely show lower levels of mask use.

Conclusion

As the country grapples with vaccine hesitancy and with the new variants of SARS-CoV-2, continued universal masking is still imperative. The effectiveness of universal masking has been demonstrated, and with the combination of vaccinations, we can be assured that the world will continue to emerge from the pandemic.

Although the universal use of masks by both health care professionals and the general public now appears routine, widely differing recommendations were distributed by different health organizations early in the pandemic. In April 2020, the World Health Organization (WHO) stated that there was no evidence that healthy individuals wearing a medical mask in the community prevented COVID-19 infection.¹ However, these recommendations must be placed in the context of a national shortage of personal protective equipment early in the pandemic. The WHO guidance released on June 5, 2020, recommended continuous use of masks for health care workers in the clinical setting.² Additional recommendations included mask replacement when wet, soiled, or damaged, and when the wearer touched the mask. The WHO also recommended mask usage by those with underlying medical comorbidities and those living in high population–density areas and in settings where physical distancing was not possible.²

The Centers for Disease Control and Prevention (CDC) officially recommended the use of face coverings for the general public to prevent COVID-19 transmission on April 3, 2020.³ The CDC highlighted that masks should not be worn by children younger than 2 years; individuals with respiratory compromise; and patients who are unconscious, incapacitated, or unable to remove a mask without assistance.⁴ Medical masks and respirators were only recommended for health care workers. Importantly, masks with valves/vents were not recommended, as respiratory droplets can be emitted, defeating the purpose of source control.⁴ New York State mandated mask usage in public places starting on April 15, 2020.

These recommendations were based on the hypothesis that COVID-19 transmission occurs primarily via droplets and contact. In reality, SARS-CoV-2 transmission more likely occurs in a continuum from larger droplets to miniscule aerosols expelled from an infected person when talking, coughing, or sneezing.^5,6 It should be noted that there was a formal suggestion of the potential for airborne transmission of SARS-CoV-2 by the CDC in a statement on September 18, 2020, that was subsequently retracted 3 days later.^7,8 The CDC, reversing their prior recommendations, updated their guidance on October 5, 2020, endorsing prior reports that SARS-CoV-2 can be spread through aerosol transmission.⁸

Mask usage helps prevent viral spread by all individuals, especially those who are presymptomatic and asymptomatic. Presymptomatic individuals account for approximately 40% to 60% of transmissions, and asymptomatic individuals account for approximately 4% to 30% of infections by some models, which suggest these individuals are the drivers of the pandemic, more so than symptomatic individuals.^9-15 Additionally, masking also may in effect reduce the amount of SARS-CoV-2 to which individuals are being exposed in the community.¹⁴ Universal masking is a relatively low-cost, low-risk intervention that may provide moderate benefit to the individual but substantial benefit to communities at large.^10-13 Universal masking in other countries also has clearly demonstrated major benefits during the pandemic. Implementation of universal masking in Taiwan resulted in only approximately 440 COVID-19 cases and less than 10 deaths, despite a population of 23 million.¹⁶ South Korea, having experience with Middle East respiratory syndrome, also was able to quickly institute a mask policy for its citizens, resulting in approximately 94% compliance.¹⁷ Moreover, several mathematical models have shown that even imperfect use of masks on a population level can prevent disease transmission and should be instituted.¹⁸

Given the importance and potential benefits of mask usage, we investigated compliance and proper utilization of facial masks in New York City (NYC), once the epicenter of the pandemic in the United States. New York City and the rest of New York State experienced more than 1.13 million and 1.46 million cases of COVID-19, respectively, as of early November 2021.¹⁹ Nationwide, NYC had the greatest absolute death count of more than 34,634 and the greatest rate of death per 100,000 individuals of 412. In contrast, New York State, excluding NYC, had an absolute death count of more than 21,646 and a death rate per 100,000 individuals of 195 as of early November 2021.¹⁹ Now entering 20 months since the first case of COVID-19 in NYC, it continues to be vital for facial mask protocols to be emphasized as part of a comprehensive infection prevention protocol, especially in light of continued vaccine resistance, to help stall continued spread of SARS-CoV-2.²⁰

We seek to show that despite months of policies for universal masking in NYC, there is still considerable mask noncompliance by the general public in health care settings where the use of masks is particularly imperative. We conducted an observational study investigating proper use of face masks of adults entering the main entrance of 4 hospitals located in NYC.

Methods

We observed mask usage in adults entering 4 hospitals in September 2020 (postsurge in NYC and prior to the availability of COVID-19 vaccinations). Hospitals were chosen to represent several types of health care delivery systems available in the United States and included a city, state, federal, and private hospital. Data collection was completed during peak traffic hours (8:00 am to 12:00 pm) on a weekday and continued until a total of 100 unique patients were observed at each site. Each hospital entrance was barricaded, and hospital staff were stationed at these entry points to take each individual’s temperature, screen for symptoms and exposure risk, verify patients’ appointments, and ensure proper mask wearing (in optimal circumstances). Data collectors (J.L. and N.M.) were stationed just past the barricade of each hospital’s entrance and observed those who entered. Individuals were not approached about the study, demographics, or the use and/or views about usage of facial masks. Children and hospital employees were excluded from data collection, with the exception of 1 hospital with a dedicated employee entrance where employees were observed for mask compliance. Except for vented/valved masks or makeshift masks fashioned out of scarfs, bandanas, or similar materials, the type of mask an individual wore was not distinguished (medical masks, cotton masks, or respirator-type masks were not differentiated).

Mask usage was observed and classified into several categories: correctly fitting mask over the nose and mouth, no face mask, mask usage with nose exposed, mask usage with mouth exposed, mask usage with both nose and mouth exposed (ie, mask on the chin/neck area), loosely fitting mask, vented/valved mask, or other form of face covering (eg, bandana, scarf).

Results

We observed a consistent rate of mask compliance between 72% and 85%, with an average of 78% of the 600 individuals observed wearing correctly fitting masks across the 4 hospitals included in this study (Table). The employee entrance included in this study had the highest compliance rate of 85%. An overall low rate of complete mask noncompliance was observed, with only 9 individuals (1.5%) in the entire study not wearing any mask. The federal hospital had the highest rate of mask noncompliance. We also observed a low rate of nose and mouth exposure, with 1.8% of individuals wearing a mask with the nose and mouth exposed (ie, mask tucked under the chin). No individuals were observed with the mouth exposed but with the nose covered by a mask. Additionally, only 3 individuals (0.5%) wore a mask with a vent/valve. The most common way that masks were worn incorrectly was with the nose exposed, accounting for 9.5% of individuals observed. Overall, only 9 individuals (1.5%) wore a nontraditional face covering, with a bandana being the most commonly observed makeshift mask.

Signage regarding the requirement to wear masks and to social distance was universally instituted at all hospital entry points (both inside and outside the hospital) in this study. However, there were no illustrations demonstrating correct and incorrect forms of mask usage. All signage merely displayed a graphic of a facial mask noting the requirement to wear a mask prior to entering the building. Hospital staff also had face masks available for patients who failed to bring a mask or who wore an inappropriate mask (ie, vented/valved masks).

Comment

Mask Effectiveness—Masks reduce the spread of SARS-CoV-2 by preventing both droplets and potentially virus-bearing aerosols.^6,21,22 It has been demonstrated that well-fitted cotton homemade masks and medical masks provide the most effective method of reducing droplet dispersion. Loosely fitted masks as well as bandana-style facial coverings minimally reduce small aerosolized droplets, and an uncovered mouth and nose can disperse particles at a distance much greater than 6 feet.²²

Mask Compliance—We report an overall high compliance rate with mask wearing among individuals visiting a hospital; however, compliance was still imperfect. Overall, 78% of observed individuals wore a correctly fitting mask when entering a hospital, even with hospital staff positioned at entry points to ensure proper mask usage. With all the resources available at health care centers, we anticipated a much higher compliance rate for correctly fitting masks at hospital entrances. We hypothesize that given only 78% of individuals showed proper mask compliance in a setting with enforcement by health care personnel, the mask compliance rate in the larger community is likely much lower. It is imperative to enforce continued mask compliance in medical centers and other public areas given notable vaccine noncompliance in certain parts of the country.

Tools to Prevent Disease Transmission—Mask usage by the general public in NYC helped in its response to the COVID-19 pandemic. Yang et al²³ demonstrated through mathematical modeling that mask usage in NYC was associated with a 6.6% reduction in transmission overall and a 20% decrease in transmission for individuals 65 years and older during the first month of the universal mask policy going into effect. The authors extrapolated these data during the NYC reopening and found that universal masking reduced transmission by approximately 9% to 11%, accounting for the increase in hours spent outside home quarantine. The authors also hypothesized that if universal masking was as effective in its reduction of transmission for everyone in NYC as it was for older adults, the potential reduction in transmission of SARS-CoV-2 could be as high as 28% to 32%.²³

Temperature checks at entrance barricades were standard protocol during the observation period. Although the main purpose of this study was to investigate compliance with and proper use of facial masks in a health care setting, it should be mentioned that, although temperature checks were being done on almost every person entering a hospital, the uniformity and practicality of this intervention has not been backed by substantial evidence. Although many nontouch thermometers are intended to capture a forehead temperature for the most accurate reading, the authors will share that in their observation, medical personnel screening individuals at hospital entrances were observed checking temperatures at any easily accessible body part, such as the forearm, hand, or neck. Furthermore, it has been reported that only approximately 40% of individuals with COVID-19 present with a fever.²⁴ Many hospitals, including the 4 that were included in this investigation, have formal protocols for patients presenting with a fever, especially those presenting to an ambulatory center. Patients are usually instructed to call ahead if they have a fever, and a decision regarding next steps will be discussed with a health care provider. In addition, 1 meta-analysis on the symptoms of COVID-19 suggested that approximately 12% of infected patients are asymptomatic, likely a conservative estimate.²⁵ Although we do not suggest that hospitals stop temperature checks, consistent temperature checks in anatomic locations intended for the specific thermometer used must be employed. Alternatively, a thermographic camera system that could detect heat signatures may be a way to screen faster, only necessitating that those above a threshold be assessed further.

The results of this study suggest that much greater effort is being placed on these temperature checks than on other equally important components of the entrance health assessment. This initial encounter at hospital entrances should serve as an opportunity for education on proper choice and use of masks with clear instructions that masks should not be removed unless directed by a health care provider and in a designated area, such as an examination room. The COVID-19 pandemic in the United States is likely the first time an individual is wearing these types of masks. Reiterating when and how often a mask should be changed (eg, when wet or soiled), how a soiled mask is not an effective mask, how a used mask should be discarded, ways to prevent self-contamination (ie, proper donning and doffing), and the importance of other infection-prevention behaviors—hand hygiene; social distancing; avoidance of touching the eyes, nose, and mouth with unwashed hands; and regular disinfecting of surfaces—should be practiced.^11,26-29 Extended use and reuse of masks also can result in transmission of infection.³⁰

Throughout the pandemic, our personal experience is that some patients often overtly refuse to wear a mask, citing underlying respiratory issues. The implications of patients not wearing a mask in a medical office and endangering other patients and staff are beyond the scope of this analysis. We will, however, comment briefly on the evidence behind this common concern. Matuschek et al³¹ found substantial adverse changes in respiratory rate, oxygen saturation, and CO₂ levels in patients with severe chronic obstructive pulmonary disease who were wearing N95 respirators during a 6-minute walk test. Another study by Chan et al³² showed that nonmedical masks in healthy older adults in the community setting had no impact on oxygen saturation. Ultimately, the most effective mask a patient can wear is a mask that will be worn consistently.³²

Populations With Limited Access to Masks—The COVID-19 pandemic disproportionately impacted disadvantaged populations, both in socioeconomic status and minority status. A disproportionate number of COVID-19 hospitalizations and deaths occurred in lower-income and minority populations.¹⁰ In fact, Lamb et al³³ reported that NYC neighborhoods with a larger proportion of uninsured individuals with limited access to health care and overall lower socioeconomic status had a higher rate of SARS-CoV-2 positivity. A retrospective study in Louisiana showed that Black individuals accounted for 77% of hospitalizations and 71% of deaths due to COVID-19 in a population where only 31% of individuals identified as Black.¹⁰ Chu et al⁶ even asserted that policies should be put into place to address equity issues for populations with limited access to masks. We agree that policies should be put into action to ensure that individuals lacking the means to obtain appropriate masks or unable to obtain an adequate supply of masks be provided this new necessity. It has been calculated that the impact of masks in reducing virus transmission would be greatest if mask availability to disadvantaged populations is ensured.¹⁸ We support a plan for masks to be covered by government-sponsored health plans.

Study Limitations—Several limitations exist in our study that should be discussed. Although the data collectors observed a large number of individuals, each hospital entrance was only observed for 1 half-day morning session. There may be variations in the number of people wearing a mask at different times of day and different days of the week with fluctuations in hospital traffic. Although data were collected at a variety of hospitals representing the diverse health care delivery models available in the United States, the NYC hospitals included in this study may have different resources available for infection-prevention strategies than hospitals across the country, given NYC’s unique population density and demographics.

Study Strengths—The generalizability of the study should be recognized. Data were collected by all major health care delivery models available in the United States—private, state, city, and federal hospital systems. This study can be easily replicated in other health care delivery systems to further investigate potential gaps in mask usage and infection prevention. Repeating this study in areas where a large portion of the population does not believe in the virus also will likely show lower levels of mask use.

Conclusion

As the country grapples with vaccine hesitancy and with the new variants of SARS-CoV-2, continued universal masking is still imperative. The effectiveness of universal masking has been demonstrated, and with the combination of vaccinations, we can be assured that the world will continue to emerge from the pandemic.

Although the universal use of masks by both health care professionals and the general public now appears routine, widely differing recommendations were distributed by different health organizations early in the pandemic. In April 2020, the World Health Organization (WHO) stated that there was no evidence that healthy individuals wearing a medical mask in the community prevented COVID-19 infection.¹ However, these recommendations must be placed in the context of a national shortage of personal protective equipment early in the pandemic. The WHO guidance released on June 5, 2020, recommended continuous use of masks for health care workers in the clinical setting.² Additional recommendations included mask replacement when wet, soiled, or damaged, and when the wearer touched the mask. The WHO also recommended mask usage by those with underlying medical comorbidities and those living in high population–density areas and in settings where physical distancing was not possible.²

The Centers for Disease Control and Prevention (CDC) officially recommended the use of face coverings for the general public to prevent COVID-19 transmission on April 3, 2020.³ The CDC highlighted that masks should not be worn by children younger than 2 years; individuals with respiratory compromise; and patients who are unconscious, incapacitated, or unable to remove a mask without assistance.⁴ Medical masks and respirators were only recommended for health care workers. Importantly, masks with valves/vents were not recommended, as respiratory droplets can be emitted, defeating the purpose of source control.⁴ New York State mandated mask usage in public places starting on April 15, 2020.

These recommendations were based on the hypothesis that COVID-19 transmission occurs primarily via droplets and contact. In reality, SARS-CoV-2 transmission more likely occurs in a continuum from larger droplets to miniscule aerosols expelled from an infected person when talking, coughing, or sneezing.^5,6 It should be noted that there was a formal suggestion of the potential for airborne transmission of SARS-CoV-2 by the CDC in a statement on September 18, 2020, that was subsequently retracted 3 days later.^7,8 The CDC, reversing their prior recommendations, updated their guidance on October 5, 2020, endorsing prior reports that SARS-CoV-2 can be spread through aerosol transmission.⁸

Mask usage helps prevent viral spread by all individuals, especially those who are presymptomatic and asymptomatic. Presymptomatic individuals account for approximately 40% to 60% of transmissions, and asymptomatic individuals account for approximately 4% to 30% of infections by some models, which suggest these individuals are the drivers of the pandemic, more so than symptomatic individuals.^9-15 Additionally, masking also may in effect reduce the amount of SARS-CoV-2 to which individuals are being exposed in the community.¹⁴ Universal masking is a relatively low-cost, low-risk intervention that may provide moderate benefit to the individual but substantial benefit to communities at large.^10-13 Universal masking in other countries also has clearly demonstrated major benefits during the pandemic. Implementation of universal masking in Taiwan resulted in only approximately 440 COVID-19 cases and less than 10 deaths, despite a population of 23 million.¹⁶ South Korea, having experience with Middle East respiratory syndrome, also was able to quickly institute a mask policy for its citizens, resulting in approximately 94% compliance.¹⁷ Moreover, several mathematical models have shown that even imperfect use of masks on a population level can prevent disease transmission and should be instituted.¹⁸

Given the importance and potential benefits of mask usage, we investigated compliance and proper utilization of facial masks in New York City (NYC), once the epicenter of the pandemic in the United States. New York City and the rest of New York State experienced more than 1.13 million and 1.46 million cases of COVID-19, respectively, as of early November 2021.¹⁹ Nationwide, NYC had the greatest absolute death count of more than 34,634 and the greatest rate of death per 100,000 individuals of 412. In contrast, New York State, excluding NYC, had an absolute death count of more than 21,646 and a death rate per 100,000 individuals of 195 as of early November 2021.¹⁹ Now entering 20 months since the first case of COVID-19 in NYC, it continues to be vital for facial mask protocols to be emphasized as part of a comprehensive infection prevention protocol, especially in light of continued vaccine resistance, to help stall continued spread of SARS-CoV-2.²⁰

We seek to show that despite months of policies for universal masking in NYC, there is still considerable mask noncompliance by the general public in health care settings where the use of masks is particularly imperative. We conducted an observational study investigating proper use of face masks of adults entering the main entrance of 4 hospitals located in NYC.

Methods

We observed mask usage in adults entering 4 hospitals in September 2020 (postsurge in NYC and prior to the availability of COVID-19 vaccinations). Hospitals were chosen to represent several types of health care delivery systems available in the United States and included a city, state, federal, and private hospital. Data collection was completed during peak traffic hours (8:00 am to 12:00 pm) on a weekday and continued until a total of 100 unique patients were observed at each site. Each hospital entrance was barricaded, and hospital staff were stationed at these entry points to take each individual’s temperature, screen for symptoms and exposure risk, verify patients’ appointments, and ensure proper mask wearing (in optimal circumstances). Data collectors (J.L. and N.M.) were stationed just past the barricade of each hospital’s entrance and observed those who entered. Individuals were not approached about the study, demographics, or the use and/or views about usage of facial masks. Children and hospital employees were excluded from data collection, with the exception of 1 hospital with a dedicated employee entrance where employees were observed for mask compliance. Except for vented/valved masks or makeshift masks fashioned out of scarfs, bandanas, or similar materials, the type of mask an individual wore was not distinguished (medical masks, cotton masks, or respirator-type masks were not differentiated).

Mask usage was observed and classified into several categories: correctly fitting mask over the nose and mouth, no face mask, mask usage with nose exposed, mask usage with mouth exposed, mask usage with both nose and mouth exposed (ie, mask on the chin/neck area), loosely fitting mask, vented/valved mask, or other form of face covering (eg, bandana, scarf).

Results

We observed a consistent rate of mask compliance between 72% and 85%, with an average of 78% of the 600 individuals observed wearing correctly fitting masks across the 4 hospitals included in this study (Table). The employee entrance included in this study had the highest compliance rate of 85%. An overall low rate of complete mask noncompliance was observed, with only 9 individuals (1.5%) in the entire study not wearing any mask. The federal hospital had the highest rate of mask noncompliance. We also observed a low rate of nose and mouth exposure, with 1.8% of individuals wearing a mask with the nose and mouth exposed (ie, mask tucked under the chin). No individuals were observed with the mouth exposed but with the nose covered by a mask. Additionally, only 3 individuals (0.5%) wore a mask with a vent/valve. The most common way that masks were worn incorrectly was with the nose exposed, accounting for 9.5% of individuals observed. Overall, only 9 individuals (1.5%) wore a nontraditional face covering, with a bandana being the most commonly observed makeshift mask.

Signage regarding the requirement to wear masks and to social distance was universally instituted at all hospital entry points (both inside and outside the hospital) in this study. However, there were no illustrations demonstrating correct and incorrect forms of mask usage. All signage merely displayed a graphic of a facial mask noting the requirement to wear a mask prior to entering the building. Hospital staff also had face masks available for patients who failed to bring a mask or who wore an inappropriate mask (ie, vented/valved masks).

Comment

Mask Effectiveness—Masks reduce the spread of SARS-CoV-2 by preventing both droplets and potentially virus-bearing aerosols.^6,21,22 It has been demonstrated that well-fitted cotton homemade masks and medical masks provide the most effective method of reducing droplet dispersion. Loosely fitted masks as well as bandana-style facial coverings minimally reduce small aerosolized droplets, and an uncovered mouth and nose can disperse particles at a distance much greater than 6 feet.²²

Mask Compliance—We report an overall high compliance rate with mask wearing among individuals visiting a hospital; however, compliance was still imperfect. Overall, 78% of observed individuals wore a correctly fitting mask when entering a hospital, even with hospital staff positioned at entry points to ensure proper mask usage. With all the resources available at health care centers, we anticipated a much higher compliance rate for correctly fitting masks at hospital entrances. We hypothesize that given only 78% of individuals showed proper mask compliance in a setting with enforcement by health care personnel, the mask compliance rate in the larger community is likely much lower. It is imperative to enforce continued mask compliance in medical centers and other public areas given notable vaccine noncompliance in certain parts of the country.

Tools to Prevent Disease Transmission—Mask usage by the general public in NYC helped in its response to the COVID-19 pandemic. Yang et al²³ demonstrated through mathematical modeling that mask usage in NYC was associated with a 6.6% reduction in transmission overall and a 20% decrease in transmission for individuals 65 years and older during the first month of the universal mask policy going into effect. The authors extrapolated these data during the NYC reopening and found that universal masking reduced transmission by approximately 9% to 11%, accounting for the increase in hours spent outside home quarantine. The authors also hypothesized that if universal masking was as effective in its reduction of transmission for everyone in NYC as it was for older adults, the potential reduction in transmission of SARS-CoV-2 could be as high as 28% to 32%.²³

Temperature checks at entrance barricades were standard protocol during the observation period. Although the main purpose of this study was to investigate compliance with and proper use of facial masks in a health care setting, it should be mentioned that, although temperature checks were being done on almost every person entering a hospital, the uniformity and practicality of this intervention has not been backed by substantial evidence. Although many nontouch thermometers are intended to capture a forehead temperature for the most accurate reading, the authors will share that in their observation, medical personnel screening individuals at hospital entrances were observed checking temperatures at any easily accessible body part, such as the forearm, hand, or neck. Furthermore, it has been reported that only approximately 40% of individuals with COVID-19 present with a fever.²⁴ Many hospitals, including the 4 that were included in this investigation, have formal protocols for patients presenting with a fever, especially those presenting to an ambulatory center. Patients are usually instructed to call ahead if they have a fever, and a decision regarding next steps will be discussed with a health care provider. In addition, 1 meta-analysis on the symptoms of COVID-19 suggested that approximately 12% of infected patients are asymptomatic, likely a conservative estimate.²⁵ Although we do not suggest that hospitals stop temperature checks, consistent temperature checks in anatomic locations intended for the specific thermometer used must be employed. Alternatively, a thermographic camera system that could detect heat signatures may be a way to screen faster, only necessitating that those above a threshold be assessed further.

The results of this study suggest that much greater effort is being placed on these temperature checks than on other equally important components of the entrance health assessment. This initial encounter at hospital entrances should serve as an opportunity for education on proper choice and use of masks with clear instructions that masks should not be removed unless directed by a health care provider and in a designated area, such as an examination room. The COVID-19 pandemic in the United States is likely the first time an individual is wearing these types of masks. Reiterating when and how often a mask should be changed (eg, when wet or soiled), how a soiled mask is not an effective mask, how a used mask should be discarded, ways to prevent self-contamination (ie, proper donning and doffing), and the importance of other infection-prevention behaviors—hand hygiene; social distancing; avoidance of touching the eyes, nose, and mouth with unwashed hands; and regular disinfecting of surfaces—should be practiced.^11,26-29 Extended use and reuse of masks also can result in transmission of infection.³⁰

Throughout the pandemic, our personal experience is that some patients often overtly refuse to wear a mask, citing underlying respiratory issues. The implications of patients not wearing a mask in a medical office and endangering other patients and staff are beyond the scope of this analysis. We will, however, comment briefly on the evidence behind this common concern. Matuschek et al³¹ found substantial adverse changes in respiratory rate, oxygen saturation, and CO₂ levels in patients with severe chronic obstructive pulmonary disease who were wearing N95 respirators during a 6-minute walk test. Another study by Chan et al³² showed that nonmedical masks in healthy older adults in the community setting had no impact on oxygen saturation. Ultimately, the most effective mask a patient can wear is a mask that will be worn consistently.³²

Populations With Limited Access to Masks—The COVID-19 pandemic disproportionately impacted disadvantaged populations, both in socioeconomic status and minority status. A disproportionate number of COVID-19 hospitalizations and deaths occurred in lower-income and minority populations.¹⁰ In fact, Lamb et al³³ reported that NYC neighborhoods with a larger proportion of uninsured individuals with limited access to health care and overall lower socioeconomic status had a higher rate of SARS-CoV-2 positivity. A retrospective study in Louisiana showed that Black individuals accounted for 77% of hospitalizations and 71% of deaths due to COVID-19 in a population where only 31% of individuals identified as Black.¹⁰ Chu et al⁶ even asserted that policies should be put into place to address equity issues for populations with limited access to masks. We agree that policies should be put into action to ensure that individuals lacking the means to obtain appropriate masks or unable to obtain an adequate supply of masks be provided this new necessity. It has been calculated that the impact of masks in reducing virus transmission would be greatest if mask availability to disadvantaged populations is ensured.¹⁸ We support a plan for masks to be covered by government-sponsored health plans.

Study Limitations—Several limitations exist in our study that should be discussed. Although the data collectors observed a large number of individuals, each hospital entrance was only observed for 1 half-day morning session. There may be variations in the number of people wearing a mask at different times of day and different days of the week with fluctuations in hospital traffic. Although data were collected at a variety of hospitals representing the diverse health care delivery models available in the United States, the NYC hospitals included in this study may have different resources available for infection-prevention strategies than hospitals across the country, given NYC’s unique population density and demographics.

Study Strengths—The generalizability of the study should be recognized. Data were collected by all major health care delivery models available in the United States—private, state, city, and federal hospital systems. This study can be easily replicated in other health care delivery systems to further investigate potential gaps in mask usage and infection prevention. Repeating this study in areas where a large portion of the population does not believe in the virus also will likely show lower levels of mask use.

Conclusion

As the country grapples with vaccine hesitancy and with the new variants of SARS-CoV-2, continued universal masking is still imperative. The effectiveness of universal masking has been demonstrated, and with the combination of vaccinations, we can be assured that the world will continue to emerge from the pandemic.

From the Department of Medicine, University of California San Diego (Dr. Haponyuk), Department of Emergency Medicine, University of Tennessee (Dr. Dejong), the Department of Family Medicine, University of New Mexico (Dr. Gutfrucht), and the Department of Internal Medicine, University of New Mexico (Dr. Barrett)

Objective: Naloxone availability can reduce the risk of death from opioid overdoses, although prescriber, legislative, and payment barriers to accessing this life-saving medication exist. A previously underreported barrier involves same-day availability, the lack of which may force patients to travel to multiple pharmacies and having delays in access or risking not filling their prescription. This study sought to determine same-day availability of naloxone in pharmacies in the state of New Mexico.

Methods: Same-day availability of naloxone was assessed via an audit survey.

Results: Of the 183 pharamacies screened, only 84.7% had same-day availability, including only 72% in Albuquerque, the state’s most populous city/municipality.

Conclusion: These results highlight the extent of a previously underexplored challenge to patient care and barrier to patient safety, and future directions for more patient-centered care.

Keywords: naloxone; barriers to care; opioid overdose prevention.

The US is enduring an ongoing epidemic of deaths due to opioid use, which have increased in frequency since the onset of the COVID-19 pandemic.¹ One strategy to reduce the risk of mortality from opioid use is to ensure the widespread availability of naloxone. Individual states have implemented harm reduction strategies to increase access to naloxone, including improving availability via a statewide standing order that it may be dispensed without a prescription.^2,3 Such naloxone access laws are being widely adopted and are believed to reduce overdose deaths.⁴

There are many barriers to patients receiving naloxone despite their clinicians providing a prescription for it, including stigmatization, financial cost, and local availability.^5-9 However, the stigma associated with naloxone extends to both patients and pharmacists. Pharmacists in West Virginia, for example, showed widespread concerns about having naloxone available for patients to purchase over the counter, for fear that increasing naloxone access may increase overdoses.⁶ A study in Tennessee also found pharmacists hesitant to recommend naloxone.⁷ Another study of rural pharmacies in Georgia found that just over half carried naloxone despite a state law that naloxone be available without a prescription.⁸ Challenges are not limited to rural areas, however; a study in Philadelphia found that more than one-third of pharmacies required a prescription to dispense naloxone, contrary to state law.⁹ Thus, in a rapidly changing regulatory environment, there are many evolving barriers to patients receiving naloxone.

Methods

To better understand barriers to patients obtaining naloxone, in July and August of 2019 we performed an audit (“secret shopper”) study of all pharmacies in the state, posing as patients wishing to obtain naloxone. A publicly available list of every pharmacy in New Mexico was used to identify 89 pharmacies in Albuquerque (the most populous city in New Mexico) and 106 pharmacies throughout the rest of the state.¹²

Every pharmacy was called via a publicly available phone number during business hours (confirmed via an internet search), at least 2 hours prior to closing. One of 3 researchers telephoned pharmacies posing as a patient and inquired whether naloxone would be available for pick up the same day. If the pharmacy confirmed it was available that day, the call concluded. If naloxone was unavailable for same day pick up, researchers asked when it would be next available. Each pharmacy was called once, and neither insurance information nor cost was offered or requested. All questions were asked in English by native English speakers.

All responses were recorded in a secure spreadsheet. Once all responses were received and reviewed, they were characterized in discrete response categories: same day, within 1 to 2 days, within 3 to 4 days, within a week, or unsure/unknown. Naloxone availability was also tracked by city/municipality, and this was compared to the state’s population distribution.

Results

Responses were recorded from 183 pharmacies. Seventeen locations were eliminated from our analysis because their phone system was inoperable or the pharmacy was permanently closed. Of the pharmacies reached, 84.7% (155/183) reported they have naloxone available for pick up on the same day (Figure 1). Of the 15.3% (28) pharmacies that did not have same-day availability, 60.7% (17 pharmacies) reported availability in 1 to 2 days, 3.6% had availability in 3 to 4 days, 3.6% had availability in 1 week, and 32.1% were unsure of next availability (Figure 2). More than one-third of the state’s patients reside in municipalities where naloxone is immediately available in at least 72% of pharmacies (Table).¹³

Discussion

Increased access to naloxone at the state and community level is associated with reduced risk for death from overdose, and, consequently, widespread availability is recommended.^14-17 Statewide real-time pharmacy availability of naloxone—as patients would experience availability—has not been previously reported. These findings suggest unpredictable same-day availability that may affect experience and care outcomes. That other studies have found similar challenges in naloxone availability in other municipalities and regions suggests this barrier to access is widespread,^6-9 and likely affects patients throughout the country.

Many patients have misgivings about naloxone, and it places an undue burden on them to travel to multiple pharmacies or take repeated trips to fill prescriptions. Additionally, patients without reliable transportation may be unable to return at a later date. Although we found most pharmacies in New Mexico without immediate availability of naloxone reported they could have it within several days, such a delay may reduce the likelihood that patients will fill their prescription at all. It is also concerning that many pharmacies are unsure of when naloxone will be available, particularly when some of these may be the only pharmacy easily accessible to patients or the one where they regularly fill their prescriptions.

Barriers to naloxone availability requires further study due to possible negative consequences for patient safety and risks for exacerbating health disparities among vulnerable populations. Further research may focus on examining the effects on patients when naloxone dispensing is delayed or impossible, why there is variability in naloxone availability between different pharmacies and municipalities, the reasons for uncertainty when naloxone will be available, and effective solutions. Expanded naloxone distribution in community locations and in clinics offers one potential patient-centered solution that should be explored, but it is likely that more widespread and systemic solutions will require policy and regulatory changes at the state and national levels.

Conclusion

Urgent solutions are needed to address the epidemic of deaths from opioid overdoses. Naloxone availability is an important tool for reducing these deaths, resulting in numerous state laws attempting to increase access. Despite this, there are persistent barriers to patients receiving naloxone, including a lack of same-day availability at pharmacies. Our results suggest that this underexplored barrier is widespread. Improving both availability and accessibility of naloxone may include legislative policy solutions as well as patient-oriented solutions, such as distribution in clinics and hospitals when opioid prescriptions are first written. Further research should be conducted to determine patient-centered, effective solutions that can improve outcomes.

Corresponding author: Eileen Barrett, MD, MPH, Department of Internal Medicine, University of New Mexico; [email protected].

Financial disclosures: None.

From the Department of Medicine, University of California San Diego (Dr. Haponyuk), Department of Emergency Medicine, University of Tennessee (Dr. Dejong), the Department of Family Medicine, University of New Mexico (Dr. Gutfrucht), and the Department of Internal Medicine, University of New Mexico (Dr. Barrett)

Objective: Naloxone availability can reduce the risk of death from opioid overdoses, although prescriber, legislative, and payment barriers to accessing this life-saving medication exist. A previously underreported barrier involves same-day availability, the lack of which may force patients to travel to multiple pharmacies and having delays in access or risking not filling their prescription. This study sought to determine same-day availability of naloxone in pharmacies in the state of New Mexico.

Methods: Same-day availability of naloxone was assessed via an audit survey.

Results: Of the 183 pharamacies screened, only 84.7% had same-day availability, including only 72% in Albuquerque, the state’s most populous city/municipality.

Conclusion: These results highlight the extent of a previously underexplored challenge to patient care and barrier to patient safety, and future directions for more patient-centered care.

Keywords: naloxone; barriers to care; opioid overdose prevention.

The US is enduring an ongoing epidemic of deaths due to opioid use, which have increased in frequency since the onset of the COVID-19 pandemic.¹ One strategy to reduce the risk of mortality from opioid use is to ensure the widespread availability of naloxone. Individual states have implemented harm reduction strategies to increase access to naloxone, including improving availability via a statewide standing order that it may be dispensed without a prescription.^2,3 Such naloxone access laws are being widely adopted and are believed to reduce overdose deaths.⁴

There are many barriers to patients receiving naloxone despite their clinicians providing a prescription for it, including stigmatization, financial cost, and local availability.^5-9 However, the stigma associated with naloxone extends to both patients and pharmacists. Pharmacists in West Virginia, for example, showed widespread concerns about having naloxone available for patients to purchase over the counter, for fear that increasing naloxone access may increase overdoses.⁶ A study in Tennessee also found pharmacists hesitant to recommend naloxone.⁷ Another study of rural pharmacies in Georgia found that just over half carried naloxone despite a state law that naloxone be available without a prescription.⁸ Challenges are not limited to rural areas, however; a study in Philadelphia found that more than one-third of pharmacies required a prescription to dispense naloxone, contrary to state law.⁹ Thus, in a rapidly changing regulatory environment, there are many evolving barriers to patients receiving naloxone.

Methods

To better understand barriers to patients obtaining naloxone, in July and August of 2019 we performed an audit (“secret shopper”) study of all pharmacies in the state, posing as patients wishing to obtain naloxone. A publicly available list of every pharmacy in New Mexico was used to identify 89 pharmacies in Albuquerque (the most populous city in New Mexico) and 106 pharmacies throughout the rest of the state.¹²

Every pharmacy was called via a publicly available phone number during business hours (confirmed via an internet search), at least 2 hours prior to closing. One of 3 researchers telephoned pharmacies posing as a patient and inquired whether naloxone would be available for pick up the same day. If the pharmacy confirmed it was available that day, the call concluded. If naloxone was unavailable for same day pick up, researchers asked when it would be next available. Each pharmacy was called once, and neither insurance information nor cost was offered or requested. All questions were asked in English by native English speakers.

All responses were recorded in a secure spreadsheet. Once all responses were received and reviewed, they were characterized in discrete response categories: same day, within 1 to 2 days, within 3 to 4 days, within a week, or unsure/unknown. Naloxone availability was also tracked by city/municipality, and this was compared to the state’s population distribution.

Results

Responses were recorded from 183 pharmacies. Seventeen locations were eliminated from our analysis because their phone system was inoperable or the pharmacy was permanently closed. Of the pharmacies reached, 84.7% (155/183) reported they have naloxone available for pick up on the same day (Figure 1). Of the 15.3% (28) pharmacies that did not have same-day availability, 60.7% (17 pharmacies) reported availability in 1 to 2 days, 3.6% had availability in 3 to 4 days, 3.6% had availability in 1 week, and 32.1% were unsure of next availability (Figure 2). More than one-third of the state’s patients reside in municipalities where naloxone is immediately available in at least 72% of pharmacies (Table).¹³

Discussion

Increased access to naloxone at the state and community level is associated with reduced risk for death from overdose, and, consequently, widespread availability is recommended.^14-17 Statewide real-time pharmacy availability of naloxone—as patients would experience availability—has not been previously reported. These findings suggest unpredictable same-day availability that may affect experience and care outcomes. That other studies have found similar challenges in naloxone availability in other municipalities and regions suggests this barrier to access is widespread,^6-9 and likely affects patients throughout the country.

Many patients have misgivings about naloxone, and it places an undue burden on them to travel to multiple pharmacies or take repeated trips to fill prescriptions. Additionally, patients without reliable transportation may be unable to return at a later date. Although we found most pharmacies in New Mexico without immediate availability of naloxone reported they could have it within several days, such a delay may reduce the likelihood that patients will fill their prescription at all. It is also concerning that many pharmacies are unsure of when naloxone will be available, particularly when some of these may be the only pharmacy easily accessible to patients or the one where they regularly fill their prescriptions.

Barriers to naloxone availability requires further study due to possible negative consequences for patient safety and risks for exacerbating health disparities among vulnerable populations. Further research may focus on examining the effects on patients when naloxone dispensing is delayed or impossible, why there is variability in naloxone availability between different pharmacies and municipalities, the reasons for uncertainty when naloxone will be available, and effective solutions. Expanded naloxone distribution in community locations and in clinics offers one potential patient-centered solution that should be explored, but it is likely that more widespread and systemic solutions will require policy and regulatory changes at the state and national levels.

Conclusion

Urgent solutions are needed to address the epidemic of deaths from opioid overdoses. Naloxone availability is an important tool for reducing these deaths, resulting in numerous state laws attempting to increase access. Despite this, there are persistent barriers to patients receiving naloxone, including a lack of same-day availability at pharmacies. Our results suggest that this underexplored barrier is widespread. Improving both availability and accessibility of naloxone may include legislative policy solutions as well as patient-oriented solutions, such as distribution in clinics and hospitals when opioid prescriptions are first written. Further research should be conducted to determine patient-centered, effective solutions that can improve outcomes.

Corresponding author: Eileen Barrett, MD, MPH, Department of Internal Medicine, University of New Mexico; [email protected].

Financial disclosures: None.

From the Department of Medicine, University of California San Diego (Dr. Haponyuk), Department of Emergency Medicine, University of Tennessee (Dr. Dejong), the Department of Family Medicine, University of New Mexico (Dr. Gutfrucht), and the Department of Internal Medicine, University of New Mexico (Dr. Barrett)

Objective: Naloxone availability can reduce the risk of death from opioid overdoses, although prescriber, legislative, and payment barriers to accessing this life-saving medication exist. A previously underreported barrier involves same-day availability, the lack of which may force patients to travel to multiple pharmacies and having delays in access or risking not filling their prescription. This study sought to determine same-day availability of naloxone in pharmacies in the state of New Mexico.

Methods: Same-day availability of naloxone was assessed via an audit survey.

Results: Of the 183 pharamacies screened, only 84.7% had same-day availability, including only 72% in Albuquerque, the state’s most populous city/municipality.

Conclusion: These results highlight the extent of a previously underexplored challenge to patient care and barrier to patient safety, and future directions for more patient-centered care.

Keywords: naloxone; barriers to care; opioid overdose prevention.

The US is enduring an ongoing epidemic of deaths due to opioid use, which have increased in frequency since the onset of the COVID-19 pandemic.¹ One strategy to reduce the risk of mortality from opioid use is to ensure the widespread availability of naloxone. Individual states have implemented harm reduction strategies to increase access to naloxone, including improving availability via a statewide standing order that it may be dispensed without a prescription.^2,3 Such naloxone access laws are being widely adopted and are believed to reduce overdose deaths.⁴

There are many barriers to patients receiving naloxone despite their clinicians providing a prescription for it, including stigmatization, financial cost, and local availability.^5-9 However, the stigma associated with naloxone extends to both patients and pharmacists. Pharmacists in West Virginia, for example, showed widespread concerns about having naloxone available for patients to purchase over the counter, for fear that increasing naloxone access may increase overdoses.⁶ A study in Tennessee also found pharmacists hesitant to recommend naloxone.⁷ Another study of rural pharmacies in Georgia found that just over half carried naloxone despite a state law that naloxone be available without a prescription.⁸ Challenges are not limited to rural areas, however; a study in Philadelphia found that more than one-third of pharmacies required a prescription to dispense naloxone, contrary to state law.⁹ Thus, in a rapidly changing regulatory environment, there are many evolving barriers to patients receiving naloxone.

Methods

To better understand barriers to patients obtaining naloxone, in July and August of 2019 we performed an audit (“secret shopper”) study of all pharmacies in the state, posing as patients wishing to obtain naloxone. A publicly available list of every pharmacy in New Mexico was used to identify 89 pharmacies in Albuquerque (the most populous city in New Mexico) and 106 pharmacies throughout the rest of the state.¹²

Every pharmacy was called via a publicly available phone number during business hours (confirmed via an internet search), at least 2 hours prior to closing. One of 3 researchers telephoned pharmacies posing as a patient and inquired whether naloxone would be available for pick up the same day. If the pharmacy confirmed it was available that day, the call concluded. If naloxone was unavailable for same day pick up, researchers asked when it would be next available. Each pharmacy was called once, and neither insurance information nor cost was offered or requested. All questions were asked in English by native English speakers.

All responses were recorded in a secure spreadsheet. Once all responses were received and reviewed, they were characterized in discrete response categories: same day, within 1 to 2 days, within 3 to 4 days, within a week, or unsure/unknown. Naloxone availability was also tracked by city/municipality, and this was compared to the state’s population distribution.

Results

Responses were recorded from 183 pharmacies. Seventeen locations were eliminated from our analysis because their phone system was inoperable or the pharmacy was permanently closed. Of the pharmacies reached, 84.7% (155/183) reported they have naloxone available for pick up on the same day (Figure 1). Of the 15.3% (28) pharmacies that did not have same-day availability, 60.7% (17 pharmacies) reported availability in 1 to 2 days, 3.6% had availability in 3 to 4 days, 3.6% had availability in 1 week, and 32.1% were unsure of next availability (Figure 2). More than one-third of the state’s patients reside in municipalities where naloxone is immediately available in at least 72% of pharmacies (Table).¹³

Discussion

Increased access to naloxone at the state and community level is associated with reduced risk for death from overdose, and, consequently, widespread availability is recommended.^14-17 Statewide real-time pharmacy availability of naloxone—as patients would experience availability—has not been previously reported. These findings suggest unpredictable same-day availability that may affect experience and care outcomes. That other studies have found similar challenges in naloxone availability in other municipalities and regions suggests this barrier to access is widespread,^6-9 and likely affects patients throughout the country.

Many patients have misgivings about naloxone, and it places an undue burden on them to travel to multiple pharmacies or take repeated trips to fill prescriptions. Additionally, patients without reliable transportation may be unable to return at a later date. Although we found most pharmacies in New Mexico without immediate availability of naloxone reported they could have it within several days, such a delay may reduce the likelihood that patients will fill their prescription at all. It is also concerning that many pharmacies are unsure of when naloxone will be available, particularly when some of these may be the only pharmacy easily accessible to patients or the one where they regularly fill their prescriptions.

Barriers to naloxone availability requires further study due to possible negative consequences for patient safety and risks for exacerbating health disparities among vulnerable populations. Further research may focus on examining the effects on patients when naloxone dispensing is delayed or impossible, why there is variability in naloxone availability between different pharmacies and municipalities, the reasons for uncertainty when naloxone will be available, and effective solutions. Expanded naloxone distribution in community locations and in clinics offers one potential patient-centered solution that should be explored, but it is likely that more widespread and systemic solutions will require policy and regulatory changes at the state and national levels.

Conclusion

Urgent solutions are needed to address the epidemic of deaths from opioid overdoses. Naloxone availability is an important tool for reducing these deaths, resulting in numerous state laws attempting to increase access. Despite this, there are persistent barriers to patients receiving naloxone, including a lack of same-day availability at pharmacies. Our results suggest that this underexplored barrier is widespread. Improving both availability and accessibility of naloxone may include legislative policy solutions as well as patient-oriented solutions, such as distribution in clinics and hospitals when opioid prescriptions are first written. Further research should be conducted to determine patient-centered, effective solutions that can improve outcomes.

Corresponding author: Eileen Barrett, MD, MPH, Department of Internal Medicine, University of New Mexico; [email protected].

Financial disclosures: None.

From Illawarra Shoalhaven Local Health District, New South Wales, Australia (Maren Jones, Dr. Hewitt, Philippa King, Rhiannon Thorn, Edward Davidson, and Tiana-Lee Elphick), and Illawarra Health and Medical Research Institute, University of Wollongong, Wollongong, New South Wales, Australia (Dr. Hewitt)

Objective: To assess the association between change scores in the Functional Independence Measure (FIM) with evaluative measures used in physiotherapy to objectively show that use of the FIM in isolation is limited.

Design: Retrospective observational study.

Setting: Five rehabilitation inpatient wards from 1 public local health district in NSW Australia.

Participants: Patient data over a 5-year time frame (2015 to 2019) were reviewed (N = 2378). The patient data from the 3 most prevalent impairment groups (Australasian Rehabilitation Outcome Centre classification) were identified for inclusion in this study: Reconditioning (n = 742, mean age 76.88 years); Orthopedic Fracture (n = 585, mean age 77.46 years); and Orthopedic Replacement (n = 377, mean age 73.84 years).

Measurements: The difference between the admission and discharge scores were calculated for each measure. Kruskal-Wallis and χ² tests were used to analyze the data.

Results: Pearson correlation (r) coefficients between FIM Motor change to the de Morton’s Mobility Index (DEMMI) change was r = 0.396, FIM Motor change to the Timed Up and Go (TUG) change was r = -0.217, and the FIM Motor change to the Ten Meter Walk Test (10MWT) change was .194.

Conclusion: The FIM Motor change scores showed a weak positive association to the DEMMI change and no association to the TUG and 10MWT change, demonstrating that the outcome measures do not measure the same attributes. To review rehabilitation effectiveness from a management perspective, it is recommended that all measures are reviewed to assess the burden of care, functional mobility, and dynamic balance.

Keywords: physiotherapy; rehabilitation; clinical outcome measures.

Patients receive interdisciplinary inpatient rehabilitation treatment after they have sustained a lower limb fracture, a lower limb joint replacement, or have generalized deconditioning (muscle wasting and disuse atrophy) following hospitalization for surgery or illness. The degree of a patient’s impairment or loss of functional capacity, as well as their ability to manage at home safely, is assessed using standardized outcome measures during their recovery and rehabilitation.^1,2

Physiotherapists routinely use validated outcome measures to assess patient progress and to measure goal attainment through assessment of functional independence, dynamic balance performance, and ambulatory ability. These objective assessments provide clinicians with information about the effectiveness of the rehabilitation program, as well as the patient’s ability to manage in their home environment, to determine the need for assistive devices, level of caregiver support, future level of autonomy, and strategies for falls prevention.^3-7

There is a view among service providers that rehabilitation decisions can be based on a singular measure of function known as the Functional Independence Measure (FIM). This is an understandable position because not only is the FIM an internationally recognized, valid, and reliable tool, but, as a singular measure, it also means measurement consistency across rehabilitation sites is more likely. However, rehabilitation is complex, and it is risky to base decisions on a single measure, which might not capture the results of rehabilitation treatment ingredients on individual patient targets.^8,9

The patient’s progress is objectively assessed using functional outcome measures such as the FIM. Other measures used typically in our service include the de Morton’s Mobility Index (DEMMI), Timed Up and Go (TUG), and the Ten Meter Walk Test (10MWT), which measure patient mobility, balance during directional changes, and walking ability, respectively. Additional measures include patient progression to a less supportive level of assistance (ie, number of persons required to assist or level of supervision) or the selection of a walking aid (eg, forearm support frame, crutches). This progression—or lack thereof—assists in decision-making regarding the individual’s future once they are discharged from rehabilitation. Such considerations would include the need to modify the home environment, selection of assistive devices, community access (walking indoors, outdoors, and shopping), personal care needs, and age-appropriate care facility recommendations (ie, level of care). The use of outcome measures also indicates the need for further referrals to other care providers upon discharge from the rehabilitation facility.

There is widespread support in the literature for the use of the FIM, DEMMI, TUG, and 10MWT in rehabilitation population groups. For example, DEMMI has been validated in hip fracture patients during rehabilitation,¹⁰ as well as among older people hospitalized for medical illness.^11-13 It has also been shown to be a predictor of discharge destination for patients living with frailty in geriatric rehabilitation settings,¹⁴ and to have moderate predictive validity for functional independence after 4 weeks of rehabilitation.¹⁵ Similarly, TUG has been validated for use among hospitalized and community-dwelling individuals,^16-18 and for patients after joint arthroplasty^19,20 or hip fracture.²¹ It has also been shown to be an indicator of fall risk,^22-24 as well as a predictor of fracture incidence.²⁵ Furthermore, TUG has been identified as an indicator of a patient’s ability to walk in the community without the need for a walking device.²⁶ It has also been shown to be an early identifier of patients in need of rehabilitation.²⁷ Normative values for TUG have been reported, and the association with gait time established.²⁸

Gait speed has been shown to predict adverse outcomes in community-dwelling older people.²⁹ In fact, the 10MWT has been established as a powerful tool to benchmark rehabilitation recovery after a medical event.³⁰ Results of the test relate to overall quality of walking, health status, morbidity, and the rate of mortality.^31-33 Meaningful improvement, minimum detectable change (0.19-0.34 m/s), and responsiveness in common physical performance in older adults has been reported.^26,34,36

Structural and functional impairment has been used to define rehabilitation classes by the Australasian Rehabilitation Outcome Centre (AROC) in the Australian National Sub-Acute and Non-Acute Patient Classification (AN-SNAP) Version 4.^37-43 Variables used for grouping are age, care type, function, and impairment for rehabilitation. FIM was developed in order to assess patients’ outcomes after inpatient multidisciplinary care, and is an internationally accepted measure of functioning.⁴⁴ It is a holistic outcome measure, which can be used to determine the patient’s level of disability and burden of care, and is widely used in both public and private inpatient rehabilitation settings. Each patient classification is reported separately within the case mix structure.⁴⁵ Inpatient rehabilitation centers are evaluated and compared by the AROC,⁴⁶ with an emphasis on length of stay and the FIM change. The most successful centers demonstrate shorter length of stay and greater FIM improvement. Although the FIM is a valuable measure, it does not provide a complete picture of the individual patient’s rehabilitation gain: ie, the specific attributes of patients’ mobility, walking ability, or balance during directional changes.

A large-scale analysis of the association between the holistic disability measure of the FIM and the more mobility- and ambulation-focused physiotherapy outcomes has not been documented.

The well-documented DEMMI accumulates points for the patient’s mobility in a similar fashion to the FIM, but with more mobility detail. These 2 outcome measures allow for the full range of patients, from the very dependent up to and including the independently ambulant patients. The DEMMI may show a positive relationship to the FIM, yet the association is unknown. The association of the TUG to the 10MWT has been established²⁸; however, their relationship to the FIM is unknown.

Current practice in the participating public health inpatient rehabilitation wards is to use the DEMMI, TUG, 10MWT, and FIM to ensure physiotherapy and allow the wider multidisciplinary team to more effectively evaluate patient mobility outcomes. The 3 most frequent patient groups identified within the current patient population are expected to present clinical differences and will be analyzed for comparison. If an association is found between the outcome measures in question, clinical efficiency could be improved.

The aim of the current study is to assess the association between change scores in the FIM with evaluative measures of outcomes typically used in physiotherapy to objectively show that use of the FIM in isolation is limited in our population of patients.

Methods

Study design and setting

This retrospective descriptive observational study complied with the STROBE-RECORD guidance and checklist (available at mdedge.com/jcomjournal) and analyzed the routinely collected data from rehabilitation patients who were admitted to 5 different rehabilitation wards in 4 different public hospitals from 1 regional local health district (20-24 beds per ward) from 2015 to 2019. As this study conducted secondary analyses using existing de-identified data from a public health facility and did not involve interaction with any human subjects, ethical approval was not required.⁴⁶ Approval to conduct this study was granted by the health district’s institutional review committee, as per the National Statement on Ethical Conduct in Human Research 2015.

Participants

Patient data over a 5-year time frame were reviewed (N = 2378). The patient data from the 3 most prevalent impairment groups were identified for inclusion in this study: reconditioning, orthopedic fracture, and orthopedic replacement. (See Table 1 for the specific AN-SNAP impairment groups used in this study.)

Patient data from the less-frequent impairment groups were excluded (n = 673, 28.19%), including stroke (n = 343), brain dysfunction (n = 45), amputation of limb (n = 45), spinal cord dysfunction (n  = 36), neurological dysfunction (n = 34), cardiac (n = 24), and others (n = 25) who may have benefitted from other outcome measures due to their medical condition. Ten patient data sets were excluded for missing discharge outcome measure data, from when the patient became ill and returned to acute services or was discharged at short notice. To be included in the study, both the admission and discharge scores from the FIM and the admission and discharge scores from at least 1 of the physiotherapy outcome measures were required for each patient (n = 1704, 71.39%): Reconditioning (n = 742), Orthopedic Fracture (n = 585), and Orthopedic Replacement (n = 377). Information regarding the type of walking aid and the amount of assistance required for safe ambulation was also recorded. These items were included in the study’s descriptive analysis. Only 1.7% of these descriptors were missing.

Outcome measures

DEMMI tasks of bed mobility, sitting balance, transfers, walking, and balance were scored with an assigned value according to the patient’s performance. This was then tallied and the results scaled, to provide an overall score out of 100 available points. The total score from admission and discharge was then compared. Improvement (change) was identified by the increase in scores.

The TUG assesses a patient’s dynamic balance performance.⁴⁷ The number of seconds it took the patient to complete the procedure was recorded at admission and discharge. Improvement (change) was identified by the reduction in time taken at discharge from the admission score.

The 10MWT measures the unidirectional walking speed of a person over 10 meters and is recorded in seconds and reported in meters per second. Improvement (change) was identified by the reduction in the time taken to increase walking speed.

Concurrent to the physiotherapy measures were the FIM scores, recorded by the accredited nursing staff from each rehabilitation ward. Improvement is demonstrated by the accumulation of points on the ordinal scale of the FIM Total, including mobility, dressing, bladder and bowel care, cognition, and social interaction, and is represented as a score between 18 and 126. The FIM Motor category is reported as a score between 13 and 91.

The 2 data sets were matched by unique identifier and admission dates, then de-identified for analysis.

Statistical analysis

Patient demographic information was analyzed using descriptive statistics (mean, SD, frequencies, percentages) for each impairment group (orthopedic fracture, orthopedic replacement, reconditioning). Differences in continuous demographic variables for each impairment group were assessed using Kruskal-Wallis tests and χ² tests for categorical variables. Functional outcome scores were compared at admission, discharge, and change between the impairment groups. Association of the functional outcome change scores was determined with the Pearson correlation coefficient (r) between the FIM and the DEMMI, TUG, and 10MWT. Graphs were plotted for each of these (Figure available online at mdedge.com/jcomjournal). A strong, moderate, and weak association was described as > 0.6, > 0.4, and > 0.2, respectively.⁴⁶ Statistical significance was set at P < .05. Analyses were conducted using Stata (StataCorp LLC, USA).

Results

The patient descriptive data (site from which data were collected, admission length of stay, age at admission, discharge destination, walk aid improvement, and walk assistance improvement) from the 3 impairment groups are reported in Table 2. The functional outcomes for DEMMI, TUG, 10MWT, FIM Motor, FIM Total at admission, discharge, and the change scores are presented in Table 3.

Orthopedic fracture patients had the greatest improvement in their functional outcomes, with a DEMMI improvement of 18 points, TUG score change of 23.49 seconds (s), 10MWT change of 0.30 meters/second (m/s), FIM Motor change of 20.62, and a FIM Total change of 21.9 points. The outcome measures exceeded the minimum detectable change as reported in the literature for DEMMI (8.8 points⁴⁸), TUG (2.08 s²⁶), walking speed 0.19 m/s²⁶, and FIM Motor (14.6 points⁴⁹).

Association of functional outcomes (change scores)

There was a significant weak positive correlation between DEMMI change score and both the FIM Motor (r = 0.396) and FIM Total change scores (r = 0.373). When viewing the specific items within the FIM Motor labelled FIM Walk change, FIM MobilityBedChair change, and FIM stairs change, r values were 0.100, 0.379, and 0.126, respectively. In addition, there was a weak negative correlation between TUG change scores and both FIM Motor (r = -0.217) and FIM Total change scores (r = -0.207). There was a very weak positive correlation between 10MWT (m/s) change scores and both FIM Motor (r = 0.194) and FIM Total change scores (r = 0.187) (Table 4, Figure). There was a moderate correlation between 10MWT change (s) and TUG change (s) (r = 0.72, P < .001).

Discussion

The purpose of this study was to ascertain the association between the DEMMI, TUG, 10MWT, and FIM measures using retrospective data collected from 5 public hospital inpatient rehabilitation wards. The results of this retrospective analysis demonstrate that a variety of objective outcome measures are required for the multidisciplinary team to accurately measure a patient’s functional improvement during their inpatient rehabilitation stay. No single outcome measure in this study fully reported all mobility attributes, and we note the risk of basing decisions on a single measure evaluating rehabilitation outcomes. Although the internationally used FIM has a strong place in rehabilitation reporting and benchmarking, it does not predict change nor provide a proxy for the patient’s whole-body motor control as they extend their mobility, dynamic balance, and ambulatory ability. Multiple objective outcome measures should therefore be required to evaluate the patient’s progress and functional performance toward discharge planning.

The FIM is a measure of disability or care needs, incorporating cognitive, social, and physical components of disability. It is a valid, holistic measure of an individual’s functional ability at a given time. Rehabilitation sites internationally utilize this assessment tool to evaluate a patient’s progress and the efficacy of intervention. The strength of this measure is its widespread use and the inclusion of the personal activities of daily living to provide an overall evaluation encompassing all aspects of a person’s ability to function independently. However, as our study results suggest, patient improvement measured by the FIM Motor components were not correlated to other widely used physiotherapy measures of ambulation and balance, such as the 10MWT or TUG. This is perhaps largely because the FIM Motor components only consider the level of assistance (eg, physical assistance, assistive device, independence) and do not consider assessment of balance and gait ability as assessed in the 10MWT and TUG. The 10MWT and TUG provide assessment of velocity and dynamic balance during walking, which have been shown to predict an individual’s risk of falling.^22,23 This is a pertinent issue in the rehabilitation and geriatric population.²⁹ Furthermore, the use of the FIM as a benchmarking tool to compare facility efficiency may not provide a complete assessment of all outcomes achieved on the inpatient rehabilitation ward, such as reduced falls risk or improved ambulatory ability and balance.

It has previously been confirmed that there is a significant positive correlation between the 10MWT and the TUG.²⁷ This retrospective analysis has also supported these findings. This is possibly due to the similarity in the assessments, as they both incorporate ambulation ability and dynamic movement.

Each of the 4 outcome measures assess different yet vital aspects of an individual’s functional mobility and ambulation ability during their subacute rehabilitation journey. The diversity of patient age, functional impairment, and mobility level needs a range of outcomes to provide baselines, targets, and goal attainment for discharge home.

Consistent with the AROC AN-SNAP reporting of Length of Stay and FIM change separated into the weighted impairment groups, the data analysis of this study demonstrated significant differences between the Reconditioning, Orthopedic Fracture, and Orthopedic Replacement patient data. Tables 2 and 3 describe the differences between the groups. The fracture population in this study improved the most across each outcome measure. In contrast, the reconditioning population showed the least improvement. This may be expected due to the pathophysiological differences between the groups. Furthermore, for the elderly who sustain fractures because of a fall, rehabilitation will be required to address not only the presenting injury but also the premorbid falls risk factors which may include polypharmacy or impaired balance.

Any conclusions drawn from the findings of this study need to take into consideration that it has focused on patients from 1 local health district and therefore may not be generalizable to a wider national or international context. As this study was a retrospective study, controlling for data collection quality, measurement bias due to nonblinding and missing data is a limitation. However, clinicians regularly completed these outcome assessments and recorded this information as part of their standard care practices within this health district. There may have been slight differences in definitions of practice between the 5 rehabilitation sites. To ensure reliability, each individual site’s protocols for the FIM, DEMMI, TUG, and 10MWT were reviewed and confirmed to be consistent.

Conclusion

The FIM Motor change scores showed a weak positive association to the DEMMI change, and no association to the TUG and 10MWT change, demonstrating that the outcome measures do not measure the same attributes. Thorough reporting of clinical outcomes is much more meaningful to assess and guide the physiotherapy component of rehabilitation. To review rehabilitation effectiveness from a management perspective, it is recommended that all measures are reviewed to assess the burden of care, mobility, functional capacity, and dynamic balance.

Acknowledgements: The authors thank Anne Smith, MSHLM, BAppSc, Head of the Physiotherapy Department, and the physiotherapists and allied health assistants who have contributed to the collection of this valuable data over several years. They also thank Lina Baytieh, MS, BS, from Research Central, Illawarra Shoalhaven Local Health District, for her assistance with the analysis.

Corresponding author: Maren Jones, MPH, BS, Physiotherapy Department, Port Kembla Hospital, Illawarra Shoalhaven Local Health District, Warrawong, New South Wales, 2505 Australia; [email protected].

Financial disclosures: None.

From Illawarra Shoalhaven Local Health District, New South Wales, Australia (Maren Jones, Dr. Hewitt, Philippa King, Rhiannon Thorn, Edward Davidson, and Tiana-Lee Elphick), and Illawarra Health and Medical Research Institute, University of Wollongong, Wollongong, New South Wales, Australia (Dr. Hewitt)

Objective: To assess the association between change scores in the Functional Independence Measure (FIM) with evaluative measures used in physiotherapy to objectively show that use of the FIM in isolation is limited.

Design: Retrospective observational study.

Setting: Five rehabilitation inpatient wards from 1 public local health district in NSW Australia.

Participants: Patient data over a 5-year time frame (2015 to 2019) were reviewed (N = 2378). The patient data from the 3 most prevalent impairment groups (Australasian Rehabilitation Outcome Centre classification) were identified for inclusion in this study: Reconditioning (n = 742, mean age 76.88 years); Orthopedic Fracture (n = 585, mean age 77.46 years); and Orthopedic Replacement (n = 377, mean age 73.84 years).

Measurements: The difference between the admission and discharge scores were calculated for each measure. Kruskal-Wallis and χ² tests were used to analyze the data.

Results: Pearson correlation (r) coefficients between FIM Motor change to the de Morton’s Mobility Index (DEMMI) change was r = 0.396, FIM Motor change to the Timed Up and Go (TUG) change was r = -0.217, and the FIM Motor change to the Ten Meter Walk Test (10MWT) change was .194.

Conclusion: The FIM Motor change scores showed a weak positive association to the DEMMI change and no association to the TUG and 10MWT change, demonstrating that the outcome measures do not measure the same attributes. To review rehabilitation effectiveness from a management perspective, it is recommended that all measures are reviewed to assess the burden of care, functional mobility, and dynamic balance.

Keywords: physiotherapy; rehabilitation; clinical outcome measures.

Patients receive interdisciplinary inpatient rehabilitation treatment after they have sustained a lower limb fracture, a lower limb joint replacement, or have generalized deconditioning (muscle wasting and disuse atrophy) following hospitalization for surgery or illness. The degree of a patient’s impairment or loss of functional capacity, as well as their ability to manage at home safely, is assessed using standardized outcome measures during their recovery and rehabilitation.^1,2

Physiotherapists routinely use validated outcome measures to assess patient progress and to measure goal attainment through assessment of functional independence, dynamic balance performance, and ambulatory ability. These objective assessments provide clinicians with information about the effectiveness of the rehabilitation program, as well as the patient’s ability to manage in their home environment, to determine the need for assistive devices, level of caregiver support, future level of autonomy, and strategies for falls prevention.^3-7

There is a view among service providers that rehabilitation decisions can be based on a singular measure of function known as the Functional Independence Measure (FIM). This is an understandable position because not only is the FIM an internationally recognized, valid, and reliable tool, but, as a singular measure, it also means measurement consistency across rehabilitation sites is more likely. However, rehabilitation is complex, and it is risky to base decisions on a single measure, which might not capture the results of rehabilitation treatment ingredients on individual patient targets.^8,9

The patient’s progress is objectively assessed using functional outcome measures such as the FIM. Other measures used typically in our service include the de Morton’s Mobility Index (DEMMI), Timed Up and Go (TUG), and the Ten Meter Walk Test (10MWT), which measure patient mobility, balance during directional changes, and walking ability, respectively. Additional measures include patient progression to a less supportive level of assistance (ie, number of persons required to assist or level of supervision) or the selection of a walking aid (eg, forearm support frame, crutches). This progression—or lack thereof—assists in decision-making regarding the individual’s future once they are discharged from rehabilitation. Such considerations would include the need to modify the home environment, selection of assistive devices, community access (walking indoors, outdoors, and shopping), personal care needs, and age-appropriate care facility recommendations (ie, level of care). The use of outcome measures also indicates the need for further referrals to other care providers upon discharge from the rehabilitation facility.

There is widespread support in the literature for the use of the FIM, DEMMI, TUG, and 10MWT in rehabilitation population groups. For example, DEMMI has been validated in hip fracture patients during rehabilitation,¹⁰ as well as among older people hospitalized for medical illness.^11-13 It has also been shown to be a predictor of discharge destination for patients living with frailty in geriatric rehabilitation settings,¹⁴ and to have moderate predictive validity for functional independence after 4 weeks of rehabilitation.¹⁵ Similarly, TUG has been validated for use among hospitalized and community-dwelling individuals,^16-18 and for patients after joint arthroplasty^19,20 or hip fracture.²¹ It has also been shown to be an indicator of fall risk,^22-24 as well as a predictor of fracture incidence.²⁵ Furthermore, TUG has been identified as an indicator of a patient’s ability to walk in the community without the need for a walking device.²⁶ It has also been shown to be an early identifier of patients in need of rehabilitation.²⁷ Normative values for TUG have been reported, and the association with gait time established.²⁸

Gait speed has been shown to predict adverse outcomes in community-dwelling older people.²⁹ In fact, the 10MWT has been established as a powerful tool to benchmark rehabilitation recovery after a medical event.³⁰ Results of the test relate to overall quality of walking, health status, morbidity, and the rate of mortality.^31-33 Meaningful improvement, minimum detectable change (0.19-0.34 m/s), and responsiveness in common physical performance in older adults has been reported.^26,34,36

Structural and functional impairment has been used to define rehabilitation classes by the Australasian Rehabilitation Outcome Centre (AROC) in the Australian National Sub-Acute and Non-Acute Patient Classification (AN-SNAP) Version 4.^37-43 Variables used for grouping are age, care type, function, and impairment for rehabilitation. FIM was developed in order to assess patients’ outcomes after inpatient multidisciplinary care, and is an internationally accepted measure of functioning.⁴⁴ It is a holistic outcome measure, which can be used to determine the patient’s level of disability and burden of care, and is widely used in both public and private inpatient rehabilitation settings. Each patient classification is reported separately within the case mix structure.⁴⁵ Inpatient rehabilitation centers are evaluated and compared by the AROC,⁴⁶ with an emphasis on length of stay and the FIM change. The most successful centers demonstrate shorter length of stay and greater FIM improvement. Although the FIM is a valuable measure, it does not provide a complete picture of the individual patient’s rehabilitation gain: ie, the specific attributes of patients’ mobility, walking ability, or balance during directional changes.

A large-scale analysis of the association between the holistic disability measure of the FIM and the more mobility- and ambulation-focused physiotherapy outcomes has not been documented.

The well-documented DEMMI accumulates points for the patient’s mobility in a similar fashion to the FIM, but with more mobility detail. These 2 outcome measures allow for the full range of patients, from the very dependent up to and including the independently ambulant patients. The DEMMI may show a positive relationship to the FIM, yet the association is unknown. The association of the TUG to the 10MWT has been established²⁸; however, their relationship to the FIM is unknown.

Current practice in the participating public health inpatient rehabilitation wards is to use the DEMMI, TUG, 10MWT, and FIM to ensure physiotherapy and allow the wider multidisciplinary team to more effectively evaluate patient mobility outcomes. The 3 most frequent patient groups identified within the current patient population are expected to present clinical differences and will be analyzed for comparison. If an association is found between the outcome measures in question, clinical efficiency could be improved.

The aim of the current study is to assess the association between change scores in the FIM with evaluative measures of outcomes typically used in physiotherapy to objectively show that use of the FIM in isolation is limited in our population of patients.

Methods

Study design and setting

This retrospective descriptive observational study complied with the STROBE-RECORD guidance and checklist (available at mdedge.com/jcomjournal) and analyzed the routinely collected data from rehabilitation patients who were admitted to 5 different rehabilitation wards in 4 different public hospitals from 1 regional local health district (20-24 beds per ward) from 2015 to 2019. As this study conducted secondary analyses using existing de-identified data from a public health facility and did not involve interaction with any human subjects, ethical approval was not required.⁴⁶ Approval to conduct this study was granted by the health district’s institutional review committee, as per the National Statement on Ethical Conduct in Human Research 2015.

Participants

Patient data over a 5-year time frame were reviewed (N = 2378). The patient data from the 3 most prevalent impairment groups were identified for inclusion in this study: reconditioning, orthopedic fracture, and orthopedic replacement. (See Table 1 for the specific AN-SNAP impairment groups used in this study.)

Patient data from the less-frequent impairment groups were excluded (n = 673, 28.19%), including stroke (n = 343), brain dysfunction (n = 45), amputation of limb (n = 45), spinal cord dysfunction (n  = 36), neurological dysfunction (n = 34), cardiac (n = 24), and others (n = 25) who may have benefitted from other outcome measures due to their medical condition. Ten patient data sets were excluded for missing discharge outcome measure data, from when the patient became ill and returned to acute services or was discharged at short notice. To be included in the study, both the admission and discharge scores from the FIM and the admission and discharge scores from at least 1 of the physiotherapy outcome measures were required for each patient (n = 1704, 71.39%): Reconditioning (n = 742), Orthopedic Fracture (n = 585), and Orthopedic Replacement (n = 377). Information regarding the type of walking aid and the amount of assistance required for safe ambulation was also recorded. These items were included in the study’s descriptive analysis. Only 1.7% of these descriptors were missing.

Outcome measures

DEMMI tasks of bed mobility, sitting balance, transfers, walking, and balance were scored with an assigned value according to the patient’s performance. This was then tallied and the results scaled, to provide an overall score out of 100 available points. The total score from admission and discharge was then compared. Improvement (change) was identified by the increase in scores.

The TUG assesses a patient’s dynamic balance performance.⁴⁷ The number of seconds it took the patient to complete the procedure was recorded at admission and discharge. Improvement (change) was identified by the reduction in time taken at discharge from the admission score.

The 10MWT measures the unidirectional walking speed of a person over 10 meters and is recorded in seconds and reported in meters per second. Improvement (change) was identified by the reduction in the time taken to increase walking speed.

Concurrent to the physiotherapy measures were the FIM scores, recorded by the accredited nursing staff from each rehabilitation ward. Improvement is demonstrated by the accumulation of points on the ordinal scale of the FIM Total, including mobility, dressing, bladder and bowel care, cognition, and social interaction, and is represented as a score between 18 and 126. The FIM Motor category is reported as a score between 13 and 91.

The 2 data sets were matched by unique identifier and admission dates, then de-identified for analysis.

Statistical analysis

Patient demographic information was analyzed using descriptive statistics (mean, SD, frequencies, percentages) for each impairment group (orthopedic fracture, orthopedic replacement, reconditioning). Differences in continuous demographic variables for each impairment group were assessed using Kruskal-Wallis tests and χ² tests for categorical variables. Functional outcome scores were compared at admission, discharge, and change between the impairment groups. Association of the functional outcome change scores was determined with the Pearson correlation coefficient (r) between the FIM and the DEMMI, TUG, and 10MWT. Graphs were plotted for each of these (Figure available online at mdedge.com/jcomjournal). A strong, moderate, and weak association was described as > 0.6, > 0.4, and > 0.2, respectively.⁴⁶ Statistical significance was set at P < .05. Analyses were conducted using Stata (StataCorp LLC, USA).

Results

The patient descriptive data (site from which data were collected, admission length of stay, age at admission, discharge destination, walk aid improvement, and walk assistance improvement) from the 3 impairment groups are reported in Table 2. The functional outcomes for DEMMI, TUG, 10MWT, FIM Motor, FIM Total at admission, discharge, and the change scores are presented in Table 3.

Orthopedic fracture patients had the greatest improvement in their functional outcomes, with a DEMMI improvement of 18 points, TUG score change of 23.49 seconds (s), 10MWT change of 0.30 meters/second (m/s), FIM Motor change of 20.62, and a FIM Total change of 21.9 points. The outcome measures exceeded the minimum detectable change as reported in the literature for DEMMI (8.8 points⁴⁸), TUG (2.08 s²⁶), walking speed 0.19 m/s²⁶, and FIM Motor (14.6 points⁴⁹).

Association of functional outcomes (change scores)

There was a significant weak positive correlation between DEMMI change score and both the FIM Motor (r = 0.396) and FIM Total change scores (r = 0.373). When viewing the specific items within the FIM Motor labelled FIM Walk change, FIM MobilityBedChair change, and FIM stairs change, r values were 0.100, 0.379, and 0.126, respectively. In addition, there was a weak negative correlation between TUG change scores and both FIM Motor (r = -0.217) and FIM Total change scores (r = -0.207). There was a very weak positive correlation between 10MWT (m/s) change scores and both FIM Motor (r = 0.194) and FIM Total change scores (r = 0.187) (Table 4, Figure). There was a moderate correlation between 10MWT change (s) and TUG change (s) (r = 0.72, P < .001).

Discussion

The purpose of this study was to ascertain the association between the DEMMI, TUG, 10MWT, and FIM measures using retrospective data collected from 5 public hospital inpatient rehabilitation wards. The results of this retrospective analysis demonstrate that a variety of objective outcome measures are required for the multidisciplinary team to accurately measure a patient’s functional improvement during their inpatient rehabilitation stay. No single outcome measure in this study fully reported all mobility attributes, and we note the risk of basing decisions on a single measure evaluating rehabilitation outcomes. Although the internationally used FIM has a strong place in rehabilitation reporting and benchmarking, it does not predict change nor provide a proxy for the patient’s whole-body motor control as they extend their mobility, dynamic balance, and ambulatory ability. Multiple objective outcome measures should therefore be required to evaluate the patient’s progress and functional performance toward discharge planning.

The FIM is a measure of disability or care needs, incorporating cognitive, social, and physical components of disability. It is a valid, holistic measure of an individual’s functional ability at a given time. Rehabilitation sites internationally utilize this assessment tool to evaluate a patient’s progress and the efficacy of intervention. The strength of this measure is its widespread use and the inclusion of the personal activities of daily living to provide an overall evaluation encompassing all aspects of a person’s ability to function independently. However, as our study results suggest, patient improvement measured by the FIM Motor components were not correlated to other widely used physiotherapy measures of ambulation and balance, such as the 10MWT or TUG. This is perhaps largely because the FIM Motor components only consider the level of assistance (eg, physical assistance, assistive device, independence) and do not consider assessment of balance and gait ability as assessed in the 10MWT and TUG. The 10MWT and TUG provide assessment of velocity and dynamic balance during walking, which have been shown to predict an individual’s risk of falling.^22,23 This is a pertinent issue in the rehabilitation and geriatric population.²⁹ Furthermore, the use of the FIM as a benchmarking tool to compare facility efficiency may not provide a complete assessment of all outcomes achieved on the inpatient rehabilitation ward, such as reduced falls risk or improved ambulatory ability and balance.

It has previously been confirmed that there is a significant positive correlation between the 10MWT and the TUG.²⁷ This retrospective analysis has also supported these findings. This is possibly due to the similarity in the assessments, as they both incorporate ambulation ability and dynamic movement.

Each of the 4 outcome measures assess different yet vital aspects of an individual’s functional mobility and ambulation ability during their subacute rehabilitation journey. The diversity of patient age, functional impairment, and mobility level needs a range of outcomes to provide baselines, targets, and goal attainment for discharge home.

Consistent with the AROC AN-SNAP reporting of Length of Stay and FIM change separated into the weighted impairment groups, the data analysis of this study demonstrated significant differences between the Reconditioning, Orthopedic Fracture, and Orthopedic Replacement patient data. Tables 2 and 3 describe the differences between the groups. The fracture population in this study improved the most across each outcome measure. In contrast, the reconditioning population showed the least improvement. This may be expected due to the pathophysiological differences between the groups. Furthermore, for the elderly who sustain fractures because of a fall, rehabilitation will be required to address not only the presenting injury but also the premorbid falls risk factors which may include polypharmacy or impaired balance.

Any conclusions drawn from the findings of this study need to take into consideration that it has focused on patients from 1 local health district and therefore may not be generalizable to a wider national or international context. As this study was a retrospective study, controlling for data collection quality, measurement bias due to nonblinding and missing data is a limitation. However, clinicians regularly completed these outcome assessments and recorded this information as part of their standard care practices within this health district. There may have been slight differences in definitions of practice between the 5 rehabilitation sites. To ensure reliability, each individual site’s protocols for the FIM, DEMMI, TUG, and 10MWT were reviewed and confirmed to be consistent.

Conclusion

The FIM Motor change scores showed a weak positive association to the DEMMI change, and no association to the TUG and 10MWT change, demonstrating that the outcome measures do not measure the same attributes. Thorough reporting of clinical outcomes is much more meaningful to assess and guide the physiotherapy component of rehabilitation. To review rehabilitation effectiveness from a management perspective, it is recommended that all measures are reviewed to assess the burden of care, mobility, functional capacity, and dynamic balance.

Acknowledgements: The authors thank Anne Smith, MSHLM, BAppSc, Head of the Physiotherapy Department, and the physiotherapists and allied health assistants who have contributed to the collection of this valuable data over several years. They also thank Lina Baytieh, MS, BS, from Research Central, Illawarra Shoalhaven Local Health District, for her assistance with the analysis.

Corresponding author: Maren Jones, MPH, BS, Physiotherapy Department, Port Kembla Hospital, Illawarra Shoalhaven Local Health District, Warrawong, New South Wales, 2505 Australia; [email protected].

Financial disclosures: None.

From Illawarra Shoalhaven Local Health District, New South Wales, Australia (Maren Jones, Dr. Hewitt, Philippa King, Rhiannon Thorn, Edward Davidson, and Tiana-Lee Elphick), and Illawarra Health and Medical Research Institute, University of Wollongong, Wollongong, New South Wales, Australia (Dr. Hewitt)

Objective: To assess the association between change scores in the Functional Independence Measure (FIM) with evaluative measures used in physiotherapy to objectively show that use of the FIM in isolation is limited.

Design: Retrospective observational study.

Setting: Five rehabilitation inpatient wards from 1 public local health district in NSW Australia.

Participants: Patient data over a 5-year time frame (2015 to 2019) were reviewed (N = 2378). The patient data from the 3 most prevalent impairment groups (Australasian Rehabilitation Outcome Centre classification) were identified for inclusion in this study: Reconditioning (n = 742, mean age 76.88 years); Orthopedic Fracture (n = 585, mean age 77.46 years); and Orthopedic Replacement (n = 377, mean age 73.84 years).

Measurements: The difference between the admission and discharge scores were calculated for each measure. Kruskal-Wallis and χ² tests were used to analyze the data.

Results: Pearson correlation (r) coefficients between FIM Motor change to the de Morton’s Mobility Index (DEMMI) change was r = 0.396, FIM Motor change to the Timed Up and Go (TUG) change was r = -0.217, and the FIM Motor change to the Ten Meter Walk Test (10MWT) change was .194.

Conclusion: The FIM Motor change scores showed a weak positive association to the DEMMI change and no association to the TUG and 10MWT change, demonstrating that the outcome measures do not measure the same attributes. To review rehabilitation effectiveness from a management perspective, it is recommended that all measures are reviewed to assess the burden of care, functional mobility, and dynamic balance.

Keywords: physiotherapy; rehabilitation; clinical outcome measures.

Patients receive interdisciplinary inpatient rehabilitation treatment after they have sustained a lower limb fracture, a lower limb joint replacement, or have generalized deconditioning (muscle wasting and disuse atrophy) following hospitalization for surgery or illness. The degree of a patient’s impairment or loss of functional capacity, as well as their ability to manage at home safely, is assessed using standardized outcome measures during their recovery and rehabilitation.^1,2

Physiotherapists routinely use validated outcome measures to assess patient progress and to measure goal attainment through assessment of functional independence, dynamic balance performance, and ambulatory ability. These objective assessments provide clinicians with information about the effectiveness of the rehabilitation program, as well as the patient’s ability to manage in their home environment, to determine the need for assistive devices, level of caregiver support, future level of autonomy, and strategies for falls prevention.^3-7

There is a view among service providers that rehabilitation decisions can be based on a singular measure of function known as the Functional Independence Measure (FIM). This is an understandable position because not only is the FIM an internationally recognized, valid, and reliable tool, but, as a singular measure, it also means measurement consistency across rehabilitation sites is more likely. However, rehabilitation is complex, and it is risky to base decisions on a single measure, which might not capture the results of rehabilitation treatment ingredients on individual patient targets.^8,9

The patient’s progress is objectively assessed using functional outcome measures such as the FIM. Other measures used typically in our service include the de Morton’s Mobility Index (DEMMI), Timed Up and Go (TUG), and the Ten Meter Walk Test (10MWT), which measure patient mobility, balance during directional changes, and walking ability, respectively. Additional measures include patient progression to a less supportive level of assistance (ie, number of persons required to assist or level of supervision) or the selection of a walking aid (eg, forearm support frame, crutches). This progression—or lack thereof—assists in decision-making regarding the individual’s future once they are discharged from rehabilitation. Such considerations would include the need to modify the home environment, selection of assistive devices, community access (walking indoors, outdoors, and shopping), personal care needs, and age-appropriate care facility recommendations (ie, level of care). The use of outcome measures also indicates the need for further referrals to other care providers upon discharge from the rehabilitation facility.

There is widespread support in the literature for the use of the FIM, DEMMI, TUG, and 10MWT in rehabilitation population groups. For example, DEMMI has been validated in hip fracture patients during rehabilitation,¹⁰ as well as among older people hospitalized for medical illness.^11-13 It has also been shown to be a predictor of discharge destination for patients living with frailty in geriatric rehabilitation settings,¹⁴ and to have moderate predictive validity for functional independence after 4 weeks of rehabilitation.¹⁵ Similarly, TUG has been validated for use among hospitalized and community-dwelling individuals,^16-18 and for patients after joint arthroplasty^19,20 or hip fracture.²¹ It has also been shown to be an indicator of fall risk,^22-24 as well as a predictor of fracture incidence.²⁵ Furthermore, TUG has been identified as an indicator of a patient’s ability to walk in the community without the need for a walking device.²⁶ It has also been shown to be an early identifier of patients in need of rehabilitation.²⁷ Normative values for TUG have been reported, and the association with gait time established.²⁸

Gait speed has been shown to predict adverse outcomes in community-dwelling older people.²⁹ In fact, the 10MWT has been established as a powerful tool to benchmark rehabilitation recovery after a medical event.³⁰ Results of the test relate to overall quality of walking, health status, morbidity, and the rate of mortality.^31-33 Meaningful improvement, minimum detectable change (0.19-0.34 m/s), and responsiveness in common physical performance in older adults has been reported.^26,34,36

Structural and functional impairment has been used to define rehabilitation classes by the Australasian Rehabilitation Outcome Centre (AROC) in the Australian National Sub-Acute and Non-Acute Patient Classification (AN-SNAP) Version 4.^37-43 Variables used for grouping are age, care type, function, and impairment for rehabilitation. FIM was developed in order to assess patients’ outcomes after inpatient multidisciplinary care, and is an internationally accepted measure of functioning.⁴⁴ It is a holistic outcome measure, which can be used to determine the patient’s level of disability and burden of care, and is widely used in both public and private inpatient rehabilitation settings. Each patient classification is reported separately within the case mix structure.⁴⁵ Inpatient rehabilitation centers are evaluated and compared by the AROC,⁴⁶ with an emphasis on length of stay and the FIM change. The most successful centers demonstrate shorter length of stay and greater FIM improvement. Although the FIM is a valuable measure, it does not provide a complete picture of the individual patient’s rehabilitation gain: ie, the specific attributes of patients’ mobility, walking ability, or balance during directional changes.

A large-scale analysis of the association between the holistic disability measure of the FIM and the more mobility- and ambulation-focused physiotherapy outcomes has not been documented.

The well-documented DEMMI accumulates points for the patient’s mobility in a similar fashion to the FIM, but with more mobility detail. These 2 outcome measures allow for the full range of patients, from the very dependent up to and including the independently ambulant patients. The DEMMI may show a positive relationship to the FIM, yet the association is unknown. The association of the TUG to the 10MWT has been established²⁸; however, their relationship to the FIM is unknown.

Current practice in the participating public health inpatient rehabilitation wards is to use the DEMMI, TUG, 10MWT, and FIM to ensure physiotherapy and allow the wider multidisciplinary team to more effectively evaluate patient mobility outcomes. The 3 most frequent patient groups identified within the current patient population are expected to present clinical differences and will be analyzed for comparison. If an association is found between the outcome measures in question, clinical efficiency could be improved.

The aim of the current study is to assess the association between change scores in the FIM with evaluative measures of outcomes typically used in physiotherapy to objectively show that use of the FIM in isolation is limited in our population of patients.

Methods

Study design and setting

This retrospective descriptive observational study complied with the STROBE-RECORD guidance and checklist (available at mdedge.com/jcomjournal) and analyzed the routinely collected data from rehabilitation patients who were admitted to 5 different rehabilitation wards in 4 different public hospitals from 1 regional local health district (20-24 beds per ward) from 2015 to 2019. As this study conducted secondary analyses using existing de-identified data from a public health facility and did not involve interaction with any human subjects, ethical approval was not required.⁴⁶ Approval to conduct this study was granted by the health district’s institutional review committee, as per the National Statement on Ethical Conduct in Human Research 2015.

Participants

Patient data over a 5-year time frame were reviewed (N = 2378). The patient data from the 3 most prevalent impairment groups were identified for inclusion in this study: reconditioning, orthopedic fracture, and orthopedic replacement. (See Table 1 for the specific AN-SNAP impairment groups used in this study.)

Patient data from the less-frequent impairment groups were excluded (n = 673, 28.19%), including stroke (n = 343), brain dysfunction (n = 45), amputation of limb (n = 45), spinal cord dysfunction (n  = 36), neurological dysfunction (n = 34), cardiac (n = 24), and others (n = 25) who may have benefitted from other outcome measures due to their medical condition. Ten patient data sets were excluded for missing discharge outcome measure data, from when the patient became ill and returned to acute services or was discharged at short notice. To be included in the study, both the admission and discharge scores from the FIM and the admission and discharge scores from at least 1 of the physiotherapy outcome measures were required for each patient (n = 1704, 71.39%): Reconditioning (n = 742), Orthopedic Fracture (n = 585), and Orthopedic Replacement (n = 377). Information regarding the type of walking aid and the amount of assistance required for safe ambulation was also recorded. These items were included in the study’s descriptive analysis. Only 1.7% of these descriptors were missing.

Outcome measures

DEMMI tasks of bed mobility, sitting balance, transfers, walking, and balance were scored with an assigned value according to the patient’s performance. This was then tallied and the results scaled, to provide an overall score out of 100 available points. The total score from admission and discharge was then compared. Improvement (change) was identified by the increase in scores.

The TUG assesses a patient’s dynamic balance performance.⁴⁷ The number of seconds it took the patient to complete the procedure was recorded at admission and discharge. Improvement (change) was identified by the reduction in time taken at discharge from the admission score.

The 10MWT measures the unidirectional walking speed of a person over 10 meters and is recorded in seconds and reported in meters per second. Improvement (change) was identified by the reduction in the time taken to increase walking speed.

Concurrent to the physiotherapy measures were the FIM scores, recorded by the accredited nursing staff from each rehabilitation ward. Improvement is demonstrated by the accumulation of points on the ordinal scale of the FIM Total, including mobility, dressing, bladder and bowel care, cognition, and social interaction, and is represented as a score between 18 and 126. The FIM Motor category is reported as a score between 13 and 91.

The 2 data sets were matched by unique identifier and admission dates, then de-identified for analysis.

Statistical analysis

Patient demographic information was analyzed using descriptive statistics (mean, SD, frequencies, percentages) for each impairment group (orthopedic fracture, orthopedic replacement, reconditioning). Differences in continuous demographic variables for each impairment group were assessed using Kruskal-Wallis tests and χ² tests for categorical variables. Functional outcome scores were compared at admission, discharge, and change between the impairment groups. Association of the functional outcome change scores was determined with the Pearson correlation coefficient (r) between the FIM and the DEMMI, TUG, and 10MWT. Graphs were plotted for each of these (Figure available online at mdedge.com/jcomjournal). A strong, moderate, and weak association was described as > 0.6, > 0.4, and > 0.2, respectively.⁴⁶ Statistical significance was set at P < .05. Analyses were conducted using Stata (StataCorp LLC, USA).

Results

The patient descriptive data (site from which data were collected, admission length of stay, age at admission, discharge destination, walk aid improvement, and walk assistance improvement) from the 3 impairment groups are reported in Table 2. The functional outcomes for DEMMI, TUG, 10MWT, FIM Motor, FIM Total at admission, discharge, and the change scores are presented in Table 3.

Orthopedic fracture patients had the greatest improvement in their functional outcomes, with a DEMMI improvement of 18 points, TUG score change of 23.49 seconds (s), 10MWT change of 0.30 meters/second (m/s), FIM Motor change of 20.62, and a FIM Total change of 21.9 points. The outcome measures exceeded the minimum detectable change as reported in the literature for DEMMI (8.8 points⁴⁸), TUG (2.08 s²⁶), walking speed 0.19 m/s²⁶, and FIM Motor (14.6 points⁴⁹).

Association of functional outcomes (change scores)

There was a significant weak positive correlation between DEMMI change score and both the FIM Motor (r = 0.396) and FIM Total change scores (r = 0.373). When viewing the specific items within the FIM Motor labelled FIM Walk change, FIM MobilityBedChair change, and FIM stairs change, r values were 0.100, 0.379, and 0.126, respectively. In addition, there was a weak negative correlation between TUG change scores and both FIM Motor (r = -0.217) and FIM Total change scores (r = -0.207). There was a very weak positive correlation between 10MWT (m/s) change scores and both FIM Motor (r = 0.194) and FIM Total change scores (r = 0.187) (Table 4, Figure). There was a moderate correlation between 10MWT change (s) and TUG change (s) (r = 0.72, P < .001).

Discussion

The purpose of this study was to ascertain the association between the DEMMI, TUG, 10MWT, and FIM measures using retrospective data collected from 5 public hospital inpatient rehabilitation wards. The results of this retrospective analysis demonstrate that a variety of objective outcome measures are required for the multidisciplinary team to accurately measure a patient’s functional improvement during their inpatient rehabilitation stay. No single outcome measure in this study fully reported all mobility attributes, and we note the risk of basing decisions on a single measure evaluating rehabilitation outcomes. Although the internationally used FIM has a strong place in rehabilitation reporting and benchmarking, it does not predict change nor provide a proxy for the patient’s whole-body motor control as they extend their mobility, dynamic balance, and ambulatory ability. Multiple objective outcome measures should therefore be required to evaluate the patient’s progress and functional performance toward discharge planning.

The FIM is a measure of disability or care needs, incorporating cognitive, social, and physical components of disability. It is a valid, holistic measure of an individual’s functional ability at a given time. Rehabilitation sites internationally utilize this assessment tool to evaluate a patient’s progress and the efficacy of intervention. The strength of this measure is its widespread use and the inclusion of the personal activities of daily living to provide an overall evaluation encompassing all aspects of a person’s ability to function independently. However, as our study results suggest, patient improvement measured by the FIM Motor components were not correlated to other widely used physiotherapy measures of ambulation and balance, such as the 10MWT or TUG. This is perhaps largely because the FIM Motor components only consider the level of assistance (eg, physical assistance, assistive device, independence) and do not consider assessment of balance and gait ability as assessed in the 10MWT and TUG. The 10MWT and TUG provide assessment of velocity and dynamic balance during walking, which have been shown to predict an individual’s risk of falling.^22,23 This is a pertinent issue in the rehabilitation and geriatric population.²⁹ Furthermore, the use of the FIM as a benchmarking tool to compare facility efficiency may not provide a complete assessment of all outcomes achieved on the inpatient rehabilitation ward, such as reduced falls risk or improved ambulatory ability and balance.

It has previously been confirmed that there is a significant positive correlation between the 10MWT and the TUG.²⁷ This retrospective analysis has also supported these findings. This is possibly due to the similarity in the assessments, as they both incorporate ambulation ability and dynamic movement.

Each of the 4 outcome measures assess different yet vital aspects of an individual’s functional mobility and ambulation ability during their subacute rehabilitation journey. The diversity of patient age, functional impairment, and mobility level needs a range of outcomes to provide baselines, targets, and goal attainment for discharge home.

Consistent with the AROC AN-SNAP reporting of Length of Stay and FIM change separated into the weighted impairment groups, the data analysis of this study demonstrated significant differences between the Reconditioning, Orthopedic Fracture, and Orthopedic Replacement patient data. Tables 2 and 3 describe the differences between the groups. The fracture population in this study improved the most across each outcome measure. In contrast, the reconditioning population showed the least improvement. This may be expected due to the pathophysiological differences between the groups. Furthermore, for the elderly who sustain fractures because of a fall, rehabilitation will be required to address not only the presenting injury but also the premorbid falls risk factors which may include polypharmacy or impaired balance.

Any conclusions drawn from the findings of this study need to take into consideration that it has focused on patients from 1 local health district and therefore may not be generalizable to a wider national or international context. As this study was a retrospective study, controlling for data collection quality, measurement bias due to nonblinding and missing data is a limitation. However, clinicians regularly completed these outcome assessments and recorded this information as part of their standard care practices within this health district. There may have been slight differences in definitions of practice between the 5 rehabilitation sites. To ensure reliability, each individual site’s protocols for the FIM, DEMMI, TUG, and 10MWT were reviewed and confirmed to be consistent.

Conclusion

The FIM Motor change scores showed a weak positive association to the DEMMI change, and no association to the TUG and 10MWT change, demonstrating that the outcome measures do not measure the same attributes. Thorough reporting of clinical outcomes is much more meaningful to assess and guide the physiotherapy component of rehabilitation. To review rehabilitation effectiveness from a management perspective, it is recommended that all measures are reviewed to assess the burden of care, mobility, functional capacity, and dynamic balance.

Acknowledgements: The authors thank Anne Smith, MSHLM, BAppSc, Head of the Physiotherapy Department, and the physiotherapists and allied health assistants who have contributed to the collection of this valuable data over several years. They also thank Lina Baytieh, MS, BS, from Research Central, Illawarra Shoalhaven Local Health District, for her assistance with the analysis.

Corresponding author: Maren Jones, MPH, BS, Physiotherapy Department, Port Kembla Hospital, Illawarra Shoalhaven Local Health District, Warrawong, New South Wales, 2505 Australia; [email protected].

Financial disclosures: None.

As one of the youngest fields of pediatric practice in the United States, pediatric hospital medicine (PHM) has grown rapidly over the past 2 decades. Approximately 10% of recent graduates from pediatric residency programs in the United States have entered PHM, with two-thirds reporting an intention to remain as hospitalists long term.^1,2

In October 2016, the American Board of Medical Specialties (ABMS) approved a petition for PHM to become the newest pediatric subspecialty.³ The application for subspeciality status, led by the Joint Council of Pediatric Hospital Medicine, articulated that subspecialty certification would more clearly define subspecialty hospitalists’ scope of practice, create a “new and larger cadre” of quality improvement (QI) experts, and strengthen opportunities for professional development related to child health safety within healthcare systems.⁴ Approximately 1500 pediatric hospitalists sat for the first PHM board-certification exam in November 2019, illustrating broad interest and commitment to this subspecialty.⁵

Characterizing the current responsibilities, practice settings, and professional interests of pediatric hospitalists is critical to understanding the continued development of the field. However, the most recent national survey of pediatric hospitalists’ roles and responsibilities was conducted more than a decade ago, and shared definitions of what constitutes PHM across institutions are lacking.⁶ Furthermore, studies suggest wide variability in PHM workload.^7-9 We therefore aimed to describe the characteristics, responsibilities, and practice settings of pediatricians who reported practicing PHM in the United States and determine how exclusive PHM practice, compared with PHM practice in combination with primary or subspecialty care, was associated with professional responsibilities and interests. We hypothesized that those reporting exclusive PHM practice would be more likely to report interest in QI leadership and intention to take the PHM certifying exam than those practicing PHM in combination with primary or subspecialty care.

Participants and Survey

Pediatricians enrolling in the American Board of Pediatrics (ABP) Maintenance of Certification (MOC) program in 2017 and 2018 were asked to complete a voluntary survey about their professional roles and scope of practice (Appendix Methods). The survey, offered to all MOC enrollees, included a hospital medicine module administered to those reporting PHM practice, given the ABP’s interest in characterizing PHM roles, responsibilities, practice settings, and interests in QI. Respondents were excluded if they were practicing outside of the United States, if they were unemployed or in a volunteer position, or if they were in fellowship training.

To ascertain areas of clinical practice, respondents were provided with a list of clinical practice areas and asked, “In which of the following areas are you practicing?” Those selecting “hospital medicine” were classified as self-identified hospitalists (hereafter, “hospitalists”). Given variation across institutions in physician roles and responsibilities, we stratified hospitalists into three groups: (1) exclusive PHM practice, representing those who reported PHM as their only area of practice; (2) PHM in combination with general pediatrics, representing those who reported practicing PHM and general pediatrics; and (3) PHM in combination with other subspecialties, representing those who reported practicing PHM in addition to one or more subspecialties. Respondents who reported practicing hospital medicine, general pediatrics, and another subspecialty were classified in the subspecialty group. The ABP’s institutional review board of record deemed the survey exempt from human subjects review.

Hospitalist Characteristics and Clinical Roles

To characterize respondents, we examined their age, gender, medical school location (American medical school or international medical school), and survey year (2017 or 2018). We also examined the following practice characteristics: US Census region, part-time versus full-time employment, academic appointment (yes or no), proportion of time spent providing direct and/or consultative patient care and fulfilling nonclinical responsibilities (research, administration, medical education, and QI), hospital setting (children’s hospital, community hospital, or mix of these hospital types), and work schedule type (shift schedule, on-service work in blocks, or a combination of shift and block schedules).

To examine variation in clinical roles, we determined the proportion of total direct and/or consultative clinical care that was spent in each of the following areas: (1) inpatient pediatric care, defined as inpatient general or subspecialty care in patients up to 21 years of age; (2) neonatal care, defined as labor and delivery, inpatient normal newborn care, and/or neonatal intensive care; (3) outpatient practice, defined as outpatient general or subspecialty care in patients up to 21 years of age; (4) emergency department care; and (5) other, which included pediatric intensive care as well inpatient adult care. Recognizing that scope of practice may differ at community hospitals and children’s hospitals, we stratified this analysis by practice setting (children’s hospital, community hospital).

Dependent Variables

We examined four dependent variables, two that were hypothesis driven and two that were exploratory. To test our hypothesis that respondents practicing PHM exclusively would be more likely to report interest in QI leadership or consultation (given the emphasis on QI in the ABMS application for subspecialty status), we examined the frequency with which respondents endorsed being “somewhat interested” or “very interested” in “serving as a leader or consultant for QI activities.” To test our hypothesis that respondents practicing PHM exclusively would be more likely to report plans to take the PHM certifying exam, we noted the frequency with which respondents reported “yes” to the question, “Do you plan to take a certifying exam in hospitalist medicine when it becomes available?” As an exploratory outcome, we examined satisfaction with allocation of professional time, available on the 2017 survey only; satisfaction was defined as an affirmative response to the question, “Is the allocation of your total professional time approximately what you wanted in your current position?” Finally, intention to maintain more than one ABP certification, also reported only in 2017 and examined as an exploratory outcome, was defined as a reported intention to maintain more than one ABP certification, including general pediatrics, PHM, or any other subspecialty.

Statistical Analysis

We used chi-square tests and analysis of variance as appropriate to examine differences in sociodemographic and professional characteristics among respondents who reported exclusive PHM practice, PHM in combination with general pediatrics, and PHM in combination with another subspecialty. To examine differences across the three PHM groups in their allocation of time to various clinical responsibilities (eg, inpatient care, newborn care), we used Kruskal-Wallis equality-of-population rank tests, stratifying by hospital type. We used multivariable logistic regression to identify associations between exclusive PHM practice and our four dependent variables, adjusting for the sociodemographic and professional characteristics described above. All analyses were conducted using Stata 15 (StataCorp LLC), using two-sided tests, and defining P < .05 as statistically significant.

Study Sample

Of the 19,763 pediatricians enrolling in MOC in 2017 and 2018, 13,839 responded the survey, representing a response rate of 70.0%. There were no significant differences between survey respondents and nonrespondents with respect to gender; differences between respondents and nonrespondents in age, medical school location, and initial year of ABP certification year were small (mean age, 48.1 years and 47.1 years, respectively [P < .01]; 77.0% of respondents were graduates of US medical schools compared with 73.7% of nonrespondents [P < .01]; mean certification year for respondents was 2003 compared with 2004 for nonrespondents [P < .01]). After applying the described exclusion criteria, 1662 of 12,665 respondents self-identified as hospitalists, reflecting 13.1% of the sample and the focus of this analysis (Appendix Figure).

Participant Characteristics and Areas of Practice

Of 1662 self-identified hospitalists, 881 (53.0%) also reported practicing general pediatrics, and 653 (39.3%) also reported practicing at least one subspecialty in addition to PHM. The most frequently reported additional subspecialty practice areas included: (1) neonatology (n = 155, 9.3%); (2) adolescent medicine (n = 138, 8.3%); (3) pediatric critical care (n = 89, 5.4%); (4) pediatric emergency medicine (n = 80, 4.8%); and (5) medicine-pediatrics (n = 30, 4.7%, asked only on the 2018 survey). When stratified into mutually exclusive groups, 491 respondents (29.5%) identified as practicing PHM exclusively, 518 (31.2%) identified as practicing PHM in combination with general pediatrics, and 653 (39.3%) identified as practicing PHM in combination with one or more other subspecialties.

Table 1 summarizes the characteristics of respondents in these three groups. Respondents reporting exclusive PHM practice were, on average, younger, more likely to be female, and more likely to be graduates of US medical schools than those reporting PHM in combination with general or subspecialty pediatrics. In total, approximately two-thirds of the sample (n = 1068, 64.3%) reported holding an academic appointment, including 72.9% (n = 358) of those reporting exclusive PHM practice compared with 56.9% (n = 295) of those also reporting general pediatrics and 63.6% (n = 415) of those also reporting subspecialty care (P < .001). Respondents who reported practicing PHM exclusively most frequently worked at children’s hospitals (64.6%, n = 317), compared with 40.0% (n = 207) and 42.1% (n = 275) of those practicing PHM in combination with general and subspecialty pediatrics, respectively (P < .001).

Clinical and Nonclinical Roles and Responsibilities

The majority of respondents reported that they spent >75% of their professional time in direct clinical or consultative care, including 62.1% (n = 305) of those reporting PHM exclusively and 77.8% (n = 403) and 66.6% (n = 435) of those reporting PHM with general and subspecialty pediatrics, respectively (P < .001). Overall, <10% reported spending less than 50% of their time proving direct patient care, including 11.2% (n = 55) of those reporting exclusive PHM practice, 11.2% (n = 73) reporting PHM in combination with a subspecialty, and 6% (n = 31) in combination with general pediatrics. The mean proportion of time spent in nonclinical roles was 22.4% (SD, 20.4%), and the mean proportions of time spent in any one area (administration, research, education, or QI) were all <10%.

The proportion of time allocated to inpatient pediatric care, neonatal care, emergency care, and outpatient pediatric care varied substantially across PHM practice groups and settings. Among respondents who practiced at children’s hospitals, the median percentage of clinical time dedicated to inpatient pediatric care was 66.5% (interquartile range [IQR], 15%-100%), with neonatal care being the second most common clinical practice area (Figure, part A; Appendix Table). At community hospitals, the percentage of clinical time dedicated to inpatient pediatric care was lower, with a median of 10% (IQR, 3%-40%) (Figure, part B). Among those reporting exclusive PHM practice, the median proportion of clinical time spent delivering inpatient pediatric care was 100% (IQR, 80%-100%) at children’s hospitals and 40% (IQR, 20%-85%) at community hospitals. At community hospitals, neonatal care accounted for a similar proportion of clinical time as inpatient pediatric care for these respondents (median, 40% [IQR, 0%-70%]). With the exception of emergency room care, we observed significant differences in how clinical time was allocated by respondents reporting exclusive PHM practice compared with those reporting PHM in combination with general or specialty care (all P values < .001, Appendix Table).

Professional Development Interests

Approximately two-thirds of respondents reported interest in QI leadership or consultation (Table 2), with those reporting exclusive PHM practice significantly more likely to report this (70.3% [n = 345] compared with 57.7% [n = 297] of those practicing PHM with general pediatrics and 66.3% [n = 431] of those practicing PHM with another subspecialty, P < .001). Similarly, 69% (n = 339) of respondents who reported exclusive PHM practice described an intention to take the PHM certifying examination, compared with 20.4% (n = 105) of those practicing PHM and general pediatrics and 17.7% (n = 115) of those practicing PHM and subspeciality pediatrics (P < .001). A total of 82.5% (n = 846) of respondents reported that they were satisfied with the allocation of their professional time; there were no significant differences between those reporting exclusive PHM practice and those reporting PHM in combination with general or subspecialty pediatrics. Of hospitalists reporting exclusive PHM practice, 67.8% (n = 166) reported an intention to maintain more than one ABP certification, compared with 22.1% (n = 78) of those practicing PHM and general pediatrics and 53.9% (n = 230) of those practicing PHM and subspecialty pediatrics (P < .001).