User login
VA Lessons From Partnering in COVID-19 Clinical Trials
The US Department of Veterans Affairs (VA), through its Office of Research and Development (ORD), supports an extensive and experienced clinical research enterprise, including the first multisite trials in the US.1 These resources contribute to the ORD support for the largest US integrated health care system, with a primary focus on the care and well-being of veterans. While the history of VA research has facilitated the creation of an experienced and organized research enterprise, the COVID-19 pandemic challenged VA to contribute even more significantly. These challenges became pronounced given the urgency associated with standing up VA sites for both therapeutic and vaccine trials.
VA Clinical Research Enterprise
The VA recognized an early need for an organized research response not only to address operational challenges resulting from COVID-19 but also ensure that the agency would be ready to support new scientific efforts focused specifically on the virus and related outcomes.2 As a result, the ORD took decisive action first by establishing itself as a central headquarters for VA COVID-19 research activities, and second, by leveraging existing resources, initiatives, and infrastructure to develop new mechanisms that would ensure that the VA was well positioned to develop or participate in research endeavors being driven by the VA as well federal, industry, and non-VA partners.
Prior to the pandemic, the ORD, through its Cooperative Studies Program (CSP), had strategies to address challenges associated with clinical trial startup and improved efficient conduct.3 For example, the VA Network of Dedicated Enrollment Sites (NODES) is a consortium of 23 VA medical centers (VAMCs) dedicated to rapid startup and recruitment into VA-sponsored clinical trials. NODES provides site-level expertise on clinical trial management, including troubleshooting challenges that may occur during clinical research execution.4 Another initiative, Access to Clinical Trials (ACT) for Veterans, engaged industry, academic, patient advocacy, and other partners to identify potential regulatory and operational hurdles to efficient startup activities specific to externally sponsored multisite clinical trials. Under ACT for Veterans, stakeholders emphasized the importance of developing a single VA point of contact for external partners to work with to more efficiently understand and navigate the VA system. In turn, such a resource could be designed to facilitate substantive research and long-term relationships with compatible external partners. Targeted to launch in April 2020, the Partnered Research Program (PRP) was expedited to respond to the pandemic.
During the pandemic, new VA efforts included the creation of the VA CoronavirUs Research and Efficacy Studies (VA CURES) network, initially established as a clinical trial master protocol framework to support and maximize VA-funded COVID-19 trial efficiency.5 VA CURES joined the consortium of trials networks funded by the National Heart, Lung, and Blood Institute. It began treatment trials under Accelerating COVID-19 Therapeutic Interventions and Vaccination (ACTIV), specifically ACTIV-4. The VA also partnered with the National Institutes of Allergy and Infectious Diseases (NIAID) by organizing the VA International Coordinating Center (VA ICC) for other ACTIV trials (ACTIV-2 and -3). When approached to startup studies that included veterans and the VA health care system, these capabilities comprised the VA research response.
A Need for a New Approach
As the impact of the pandemic expanded and the need for effective treatments and vaccines grew, national calls were made to assess the capabilities and readiness of available clinical trials networks. Additionally, the US Department of Health and Human Services Biomedical Advanced Research and Development Authority, ACTIV, NIAID Division of Clinical Research and Division of AIDS, and many pharmaceutical companies were starting to roll out trials of new therapeutics and vaccines. These groups approached the VA to help evaluate the safety and efficacy of several therapeutics and vaccines because they recognized several advantages of the VA enterprise, including its position as the nation’s largest integrated health care system, its diverse patient population, and its expertise in conducting clinical trials.
Although the VA was well positioned as an important player in a collaborative investigational approach to COVID-19 research, these trials required startup approaches that were significantly different from those it had employed in traditional, prepandemic, clinical research. Despite the VA being a single federal agency, each VAMC conducting research establishes its own practices to address both operational and regulatory requirements. This structure results in individual units that operate under different standard operating procedures. Efforts must be taken centrally to organize them into a singular network for the entire health care system. During a national crisis, when there was a need for rapid trial startup to answer safety and efficacy questions and participate under a common approach to protocol execution, this variability was neither manageable nor acceptable. Additionally, the intense resource demands associated with such research, coupled with frequent reporting requirements by VA leaders, Congress, and the White House, required that VAMCs function more like a single unit. Therefore, the ORD needed to develop VAMCs’ abilities to work collectively toward a common goal, share knowledge and experience, and capitalize on potential efficiencies concerning legal, regulatory, and operational processes.
Beginning August 2020, 39 VAMCs joined 7 large-scale collaborative COVID-19 therapeutic and vaccine trials. Through its COVID-19 Research Response Team, the ORD identified, engaged, and directed appropriate resources to support the VAMC under a centralized framework for study management (Table). Centralized management not only afforded VAMCs the opportunity to work more collectively and efficiently but also provided an important advantage by enabling the VA to collect and organize its experiences (and on occasion data) to provide a base for continual learning and improvement efforts. While others have described efforts undertaken across networks to advance learning health systems, the VA’s national scope and integration of research and clinical care allow greater opportunities to learn in a practical setting.6
Challenges and Best Practices
Using surveys, webinars, interviews, and observation from site and VA Central Office personnel, the ORD identified specific variables that prevented the VAMCs from quickly starting up as a clinical trial site. We also documented strategies, solutions, and recommendations for improving startup time lines. These were organized into 8 categories: (1) site infrastructure needs and capabilities; (2) study management roles and responsibilities; (3) educational resources and training; (4) local review requirements and procedures; (5) study design demands; (6) contracting and budgeting; (7) central-level systems and processes; and (8) communication between external partners and within the VA.
Site Infrastructure Needs and Capabilities
A primary impediment to rapid study startup was a lack of basic infrastructure, including staff, space, and the agility necessary for the changing demands of high-priority, high-enrolling trials. This observation is not unique to the VA.7 Initially, certain facilities located in hot spots where COVID-19 was more prevalent became high-interest targets for study placement, despite varying degrees of available research infrastructure. Furthermore, pandemic shutdowns and quarantines permitted fewer employees onsite. This resulted in inadequate staffing in personnel needed to support required startup activities and those needed to handle the high volume of study participants who were being recruited, screened, enrolled, and followed. Additionally, as clinical care needs and infection control practices were prioritized, clinical research space was often appropriated for these needs, making it difficult to find the space to conduct trials. Lastly, supply chain issues also posed unique challenges, sometimes making it difficult for participating VAMCs to obtain needed materials, such as IV solution bags of specific sizes and contents, safety injection needles, and IV line filters.
The VA was able to use central purchasing/contracting at coordinating centers or the VA Central Office to support investigators and assist with finding supplies and clinical research space. VAMCs with research operating budgets to cover startup costs were better positioned to handle funding delays. During the pandemic, the ORD further contracted to supply administrative support to research offices to address regulatory and other requirements needed for startup activities. The ability to expand such central contracts to procure clinical research staff and outpatient clinical research space may also prove useful in meeting key needs at a site.
Management Roles and Responsibilities
Ambiguous and variable roles and responsibilities among the various partners and stakeholders represented a challenge given the large-scale, national, or international operations involved in the trials. VA attempts to operate uniformly were further limited given that each sponsor or group had preferred methods for operating and/or organizing work under urgent time lines. For example, one trial involved a coordinating center, a contract research organization, and federal partners that each worked with individual sites. Consequently, VA study teams would receive messages that were conflicting or unclear.
The VA learned that studies need a single “source of truth” and/or central command structure in times of urgency. To mitigate conflicting messages, vaccine trials relied on a clearinghouse through the PRP to interpret requirements or work on behalf of all sites before key actions were taken. For studies with the NIAID, the VA relied on experienced staff at the CSP coordinating center at the Perry Point, Maryland, VAMC before beginning. This approach especially helped with the challenges of understaffing and sites’ lack of familiarity with complex platform trial designs and already-established network practices within the ACTIV-2 and ACTIV-3 studies.
Educational Resources and Training
Since VA participation in externally sponsored, multisite clinical trials traditionally relies on an individual VAMC study team and its local resources, transitioning to centralized approaches for COVID-19 multisite studies created barriers. Many VAMCs were unfamiliar with newer capabilities for more rapid regulatory reviews and approvals involving commercial institutional review boards (IRBs) and central VA information security and privacy reviews. While tools and resources were available to facilitate these processes, real-time use had not been fully tested. As a result, everyone had to learn as they went along.
The simultaneous establishment of workflows required the ORD to centralize operations and provide training and guidance to field personnel. Although many principal investigators and clinical research coordinators had trial experience, training required unlearning previous understandings of requirements to meet urgent time lines. ORD enterprise road maps, central tools, and training materials also were made available on a study-by-study basis. Open communication was vital to train on central study materials while opportunities to discuss, question, and share experiences and ideas were promoted. The ORD also sent regular emails to prepare for upcoming work and/or raise awareness of identified challenges.
Local Review Requirements/Procedures
The clinical trials were impacted by varying VAMC review requirements and approval processes. Although VA policy defines standard requirements, the timing and procedures are left to the individual facility to determine any local factors to accommodate and/or resource availability. While such an approach is well understood within the VA, external sponsors were not as familiar and assumed a more uniform approach across all sites. In response, some VAMCs established ad hoc research and development committee review procedures, allowing study teams to obtain the necessary reviews in a timely fashion. However, not all VAMCs had the infrastructure (especially when clinical personnel had been redeployed to other priorities) to respond with such agility. One critical role of the VA Central Office coordinating entities was to communicate and manage external sponsor and group expectations surrounding individual site review time lines. However, establishing policies and procedures that focus on streamlining local review processes helped to broadly mitigate the COVID-19 trial challenges.
Study Design Demands
The design of COVID-19 studies combined with the uncertainty of the pandemic required rapid protocol changes and adaptations that were often difficult to deliver. The multinetwork trials that the VA collaborated on were platform or master protocol designs. These designs emphasized overall goals (eg, treating patients requiring intensive care unit care). However, because this trial strategy also introduces complexities that may impact review and execution among those unfamiliar with it, there is a need for increased discussion and understanding of this methodology.8 For example, there can be shared control groups, reliance on specific criteria for halting because of safety or futility concerns, or continuation and expansion applied through an external review board. Delays may arise when changes to study protocols occur rapidly or frequently and necessitate new regulatory reviews, negotiation of new agreements, modifications to contracts, changes to entry criteria, etc.
While the VA has adopted a quality by design framework, VA investigators noted many missed opportunities related to looking at outcomes with new diagnostics, studies of serology, outcomes related to vaccinations, and understanding the natural history of disease in these trials.9 The limited opportunities for investigator input suggested that the advantages offered by platform designs were not maximized during pandemic-focused urgencies. It was unclear whether this barrier was created by a general lack of awareness by sponsors or a lack of opportunities. At the very least, quality by design approaches may help avoid redundancies in documentation or study processes at the central and site levels.
Contracting and Budgeting
Given external sponsorship of COVID-19 trials, efficient contracting and budgeting were critical for a rapid start up. The variability of processes associated with these trials created several challenges that were compounded by issues, such as site sub-agreements and budget documents that did not always go to the correct groups and individuals. Furthermore, the VA’s ability to use contracted resources (eg, tents, trailers, personnel) that external sponsors had built into their contracts was more difficult for VA as a federal agency governed by other statues and policies. This also put VAMCs at a disadvantage from a timing perspective, as the VA often required additional time to find equivalent solutions that met federal regulations.
Although the VA was able to establish contract solutions to some issues, time was still lost while working to secure initial funding. Additionally, for needs such as home phlebotomy—commended for convenience to veterans and research staff—and engaging a specialized research team in the Office of General Counsel, early awareness of protocol needs and sponsor solutions could allow VA to pursue alternatives sooner.
Central-Level Systems and Processes
Not all challenges were at the VAMC level. As the ORD explored solutions, it learned that various tools and study platforms were available but not considered. Applications, such as eConsent, and file-sharing platforms that met existing information security and privacy requirements were needed but had to comply with the Privacy Act of 1974, Federal Information Security Modernization Act, and other requirements. Using sponsor-provided devices, such as drug temperature monitoring equipment, required additional review to ensure that they met system requirements for a national health care system. In addition, the VA uses a clinical trials management review system; however, its implementation was new at the time these trials began. Furthermore, the system engaged with some commercial IRBs but not all. This resulted in additional delays as VAMCs and central resources worked to familiarize themselves with the system and procedures.
The ability to work collaboratively across the VA includes having a framework in which key startup processes are standardized. This allows for efficiency and minimizes variability. Also, all stakeholders should understand the importance of holding discussions to identify appropriate solutions, guidance, and instruction. Finally, the VA must strive to be more nimble when adapting technological, regulatory, and financial processes.
Internal and External Communication
The value of communication—both internal and external—cannot be understated. Minimizing confusion, managing expectations, and ensuring consistent messaging were essential for rapid trial execution. Despite being the second largest federal agency, the VA did not have a seat at the study leadership table for several protocols. When it joined later, several study aspects were set and/or difficult to revise. Challenges affecting time and securing resources have been noted. The ability to plan and then share expectations and responsibilities across and within the respective participating organizations early in the process was perhaps the single factor that was most addressable. The VA enterprise organization and integration with other units could accentuate key communications that would be essential in time-sensitive activities.
VA as a Partner for Future Research
Before the pandemic, the VA had already undertaken a path to enhance its ability to partner as part of the national biomedical research enterprise. The need for COVID-19 therapeutic and vaccine trials accelerated opportunities to plan and develop processes and capabilities to advance this path. As a key strength for VA scientific activities, clinical trials represent a primary medium by which to develop its partnerships. Learning and development have become part of a culture that expedites opportunities for veterans who actively seek ways to contribute to medical knowledge and treatments for their peers and the nation.
CONCLUSIONS
Challenges associated with rapid startup and completion of clinical trials have been discussed for some time. During the pandemic, needs and barriers were magnified because of the heightened urgency for evidence-based therapeutics and vaccines. While the VA faced similar problems as well as those specific to it as a health care system, it had the opportunity to learn and more systematically implement solutions to help in its partnered efforts.10 As an enterprise, the VA hopes to apply lessons learned, strategies, and best practices to further its goals to enhance veteran access to clinical trials and respond to any future need to quickly establish evidence bases in pandemics and other health emergencies that warrant the rapid implementation of research.
Acknowledgments
The activities reported here were supported by the US Department of Veterans Affairs, Office of Research and Development.
1. Hays MT; US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development. A historical look at the establishment of the Department of Veterans Affairs Research & Development Program. Accessed August 28, 2023. https://www.research.va.gov/pubs/docs/ORD-85yrHistory.pdf
2. Garcia AP, Huang GD, Arnheim L, Ramoni R, Clancy C. The VA research enterprise: a platform for national partnerships toward evidence building and scientific innovation. Fed Pract. 2023;40(suppl 5):S12-S17. doi:10.12788/fp.0425
3. Johnston SC, Lewis-Hall F, Bajpai A, et al. It’s time to harmonize clinical trial site standards. NAM Perspectives. October 9, 2017. Accessed August 28, 2023. https://nam.edu/wp-content/uploads/2017/10/Its-Time-to-Harmonize-Clinical-Trial-Site-1.pdf
4. Condon DL, Beck D, Kenworthy-Heinige T, et al. A cross-cutting approach to enhancing clinical trial site success: the Department of Veterans Affairs’ Network of Dedicated Enrollment Sites (NODES) model. Contemp Clin Trials Commun. 2017;6:78-84. Published 2017 Mar 29. doi:10.1016/j.conctc.2017.03.006
5. US Food and Drug Administration. Master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry. March 2022. Accessed August 23, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/master-protocols-efficient-clinical-trial-design-strategies-expedite-development-oncology-drugs-and
6. IOM Roundtable on Value & Science-Driven Care; Institute of Medicine. Continuous learning and improvement in health care. In: Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. National Academies Press (US); 2015:chap 2. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK284654 7. Institute of Medicine (US). Building an infrastructure to support clinical trials. In: Envisioning a Transformed Clinical Trials Enterprise in the United States. National Academies Press (US); 2012:chap 5. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK114656
8. Park JJH, Harari O, Dron L, Lester RT, Thorlund K, Mills EJ. An overview of platform trials with a checklist for clinical readers. J Clin Epidemiol. 2020;125:1-8. doi:10.1016/j.jclinepi.2020.04.025
9. Meeker-O’Connell A, Glessner C, Behm M, et al. Enhancing clinical evidence by proactively building quality into clinical trials. Clin Trials. 2016;13(4):439-444. doi:10.1177/1740774516643491
10. McClure J, Asghar A, Krajec A, et al. Clinical trial facilitators: a novel approach to support the execution of clinical research at the study site level. Contemp Clin Trials Commun. 2023;33:101106. doi:10.1016/j.conctc.2023.101106
The US Department of Veterans Affairs (VA), through its Office of Research and Development (ORD), supports an extensive and experienced clinical research enterprise, including the first multisite trials in the US.1 These resources contribute to the ORD support for the largest US integrated health care system, with a primary focus on the care and well-being of veterans. While the history of VA research has facilitated the creation of an experienced and organized research enterprise, the COVID-19 pandemic challenged VA to contribute even more significantly. These challenges became pronounced given the urgency associated with standing up VA sites for both therapeutic and vaccine trials.
VA Clinical Research Enterprise
The VA recognized an early need for an organized research response not only to address operational challenges resulting from COVID-19 but also ensure that the agency would be ready to support new scientific efforts focused specifically on the virus and related outcomes.2 As a result, the ORD took decisive action first by establishing itself as a central headquarters for VA COVID-19 research activities, and second, by leveraging existing resources, initiatives, and infrastructure to develop new mechanisms that would ensure that the VA was well positioned to develop or participate in research endeavors being driven by the VA as well federal, industry, and non-VA partners.
Prior to the pandemic, the ORD, through its Cooperative Studies Program (CSP), had strategies to address challenges associated with clinical trial startup and improved efficient conduct.3 For example, the VA Network of Dedicated Enrollment Sites (NODES) is a consortium of 23 VA medical centers (VAMCs) dedicated to rapid startup and recruitment into VA-sponsored clinical trials. NODES provides site-level expertise on clinical trial management, including troubleshooting challenges that may occur during clinical research execution.4 Another initiative, Access to Clinical Trials (ACT) for Veterans, engaged industry, academic, patient advocacy, and other partners to identify potential regulatory and operational hurdles to efficient startup activities specific to externally sponsored multisite clinical trials. Under ACT for Veterans, stakeholders emphasized the importance of developing a single VA point of contact for external partners to work with to more efficiently understand and navigate the VA system. In turn, such a resource could be designed to facilitate substantive research and long-term relationships with compatible external partners. Targeted to launch in April 2020, the Partnered Research Program (PRP) was expedited to respond to the pandemic.
During the pandemic, new VA efforts included the creation of the VA CoronavirUs Research and Efficacy Studies (VA CURES) network, initially established as a clinical trial master protocol framework to support and maximize VA-funded COVID-19 trial efficiency.5 VA CURES joined the consortium of trials networks funded by the National Heart, Lung, and Blood Institute. It began treatment trials under Accelerating COVID-19 Therapeutic Interventions and Vaccination (ACTIV), specifically ACTIV-4. The VA also partnered with the National Institutes of Allergy and Infectious Diseases (NIAID) by organizing the VA International Coordinating Center (VA ICC) for other ACTIV trials (ACTIV-2 and -3). When approached to startup studies that included veterans and the VA health care system, these capabilities comprised the VA research response.
A Need for a New Approach
As the impact of the pandemic expanded and the need for effective treatments and vaccines grew, national calls were made to assess the capabilities and readiness of available clinical trials networks. Additionally, the US Department of Health and Human Services Biomedical Advanced Research and Development Authority, ACTIV, NIAID Division of Clinical Research and Division of AIDS, and many pharmaceutical companies were starting to roll out trials of new therapeutics and vaccines. These groups approached the VA to help evaluate the safety and efficacy of several therapeutics and vaccines because they recognized several advantages of the VA enterprise, including its position as the nation’s largest integrated health care system, its diverse patient population, and its expertise in conducting clinical trials.
Although the VA was well positioned as an important player in a collaborative investigational approach to COVID-19 research, these trials required startup approaches that were significantly different from those it had employed in traditional, prepandemic, clinical research. Despite the VA being a single federal agency, each VAMC conducting research establishes its own practices to address both operational and regulatory requirements. This structure results in individual units that operate under different standard operating procedures. Efforts must be taken centrally to organize them into a singular network for the entire health care system. During a national crisis, when there was a need for rapid trial startup to answer safety and efficacy questions and participate under a common approach to protocol execution, this variability was neither manageable nor acceptable. Additionally, the intense resource demands associated with such research, coupled with frequent reporting requirements by VA leaders, Congress, and the White House, required that VAMCs function more like a single unit. Therefore, the ORD needed to develop VAMCs’ abilities to work collectively toward a common goal, share knowledge and experience, and capitalize on potential efficiencies concerning legal, regulatory, and operational processes.
Beginning August 2020, 39 VAMCs joined 7 large-scale collaborative COVID-19 therapeutic and vaccine trials. Through its COVID-19 Research Response Team, the ORD identified, engaged, and directed appropriate resources to support the VAMC under a centralized framework for study management (Table). Centralized management not only afforded VAMCs the opportunity to work more collectively and efficiently but also provided an important advantage by enabling the VA to collect and organize its experiences (and on occasion data) to provide a base for continual learning and improvement efforts. While others have described efforts undertaken across networks to advance learning health systems, the VA’s national scope and integration of research and clinical care allow greater opportunities to learn in a practical setting.6
Challenges and Best Practices
Using surveys, webinars, interviews, and observation from site and VA Central Office personnel, the ORD identified specific variables that prevented the VAMCs from quickly starting up as a clinical trial site. We also documented strategies, solutions, and recommendations for improving startup time lines. These were organized into 8 categories: (1) site infrastructure needs and capabilities; (2) study management roles and responsibilities; (3) educational resources and training; (4) local review requirements and procedures; (5) study design demands; (6) contracting and budgeting; (7) central-level systems and processes; and (8) communication between external partners and within the VA.
Site Infrastructure Needs and Capabilities
A primary impediment to rapid study startup was a lack of basic infrastructure, including staff, space, and the agility necessary for the changing demands of high-priority, high-enrolling trials. This observation is not unique to the VA.7 Initially, certain facilities located in hot spots where COVID-19 was more prevalent became high-interest targets for study placement, despite varying degrees of available research infrastructure. Furthermore, pandemic shutdowns and quarantines permitted fewer employees onsite. This resulted in inadequate staffing in personnel needed to support required startup activities and those needed to handle the high volume of study participants who were being recruited, screened, enrolled, and followed. Additionally, as clinical care needs and infection control practices were prioritized, clinical research space was often appropriated for these needs, making it difficult to find the space to conduct trials. Lastly, supply chain issues also posed unique challenges, sometimes making it difficult for participating VAMCs to obtain needed materials, such as IV solution bags of specific sizes and contents, safety injection needles, and IV line filters.
The VA was able to use central purchasing/contracting at coordinating centers or the VA Central Office to support investigators and assist with finding supplies and clinical research space. VAMCs with research operating budgets to cover startup costs were better positioned to handle funding delays. During the pandemic, the ORD further contracted to supply administrative support to research offices to address regulatory and other requirements needed for startup activities. The ability to expand such central contracts to procure clinical research staff and outpatient clinical research space may also prove useful in meeting key needs at a site.
Management Roles and Responsibilities
Ambiguous and variable roles and responsibilities among the various partners and stakeholders represented a challenge given the large-scale, national, or international operations involved in the trials. VA attempts to operate uniformly were further limited given that each sponsor or group had preferred methods for operating and/or organizing work under urgent time lines. For example, one trial involved a coordinating center, a contract research organization, and federal partners that each worked with individual sites. Consequently, VA study teams would receive messages that were conflicting or unclear.
The VA learned that studies need a single “source of truth” and/or central command structure in times of urgency. To mitigate conflicting messages, vaccine trials relied on a clearinghouse through the PRP to interpret requirements or work on behalf of all sites before key actions were taken. For studies with the NIAID, the VA relied on experienced staff at the CSP coordinating center at the Perry Point, Maryland, VAMC before beginning. This approach especially helped with the challenges of understaffing and sites’ lack of familiarity with complex platform trial designs and already-established network practices within the ACTIV-2 and ACTIV-3 studies.
Educational Resources and Training
Since VA participation in externally sponsored, multisite clinical trials traditionally relies on an individual VAMC study team and its local resources, transitioning to centralized approaches for COVID-19 multisite studies created barriers. Many VAMCs were unfamiliar with newer capabilities for more rapid regulatory reviews and approvals involving commercial institutional review boards (IRBs) and central VA information security and privacy reviews. While tools and resources were available to facilitate these processes, real-time use had not been fully tested. As a result, everyone had to learn as they went along.
The simultaneous establishment of workflows required the ORD to centralize operations and provide training and guidance to field personnel. Although many principal investigators and clinical research coordinators had trial experience, training required unlearning previous understandings of requirements to meet urgent time lines. ORD enterprise road maps, central tools, and training materials also were made available on a study-by-study basis. Open communication was vital to train on central study materials while opportunities to discuss, question, and share experiences and ideas were promoted. The ORD also sent regular emails to prepare for upcoming work and/or raise awareness of identified challenges.
Local Review Requirements/Procedures
The clinical trials were impacted by varying VAMC review requirements and approval processes. Although VA policy defines standard requirements, the timing and procedures are left to the individual facility to determine any local factors to accommodate and/or resource availability. While such an approach is well understood within the VA, external sponsors were not as familiar and assumed a more uniform approach across all sites. In response, some VAMCs established ad hoc research and development committee review procedures, allowing study teams to obtain the necessary reviews in a timely fashion. However, not all VAMCs had the infrastructure (especially when clinical personnel had been redeployed to other priorities) to respond with such agility. One critical role of the VA Central Office coordinating entities was to communicate and manage external sponsor and group expectations surrounding individual site review time lines. However, establishing policies and procedures that focus on streamlining local review processes helped to broadly mitigate the COVID-19 trial challenges.
Study Design Demands
The design of COVID-19 studies combined with the uncertainty of the pandemic required rapid protocol changes and adaptations that were often difficult to deliver. The multinetwork trials that the VA collaborated on were platform or master protocol designs. These designs emphasized overall goals (eg, treating patients requiring intensive care unit care). However, because this trial strategy also introduces complexities that may impact review and execution among those unfamiliar with it, there is a need for increased discussion and understanding of this methodology.8 For example, there can be shared control groups, reliance on specific criteria for halting because of safety or futility concerns, or continuation and expansion applied through an external review board. Delays may arise when changes to study protocols occur rapidly or frequently and necessitate new regulatory reviews, negotiation of new agreements, modifications to contracts, changes to entry criteria, etc.
While the VA has adopted a quality by design framework, VA investigators noted many missed opportunities related to looking at outcomes with new diagnostics, studies of serology, outcomes related to vaccinations, and understanding the natural history of disease in these trials.9 The limited opportunities for investigator input suggested that the advantages offered by platform designs were not maximized during pandemic-focused urgencies. It was unclear whether this barrier was created by a general lack of awareness by sponsors or a lack of opportunities. At the very least, quality by design approaches may help avoid redundancies in documentation or study processes at the central and site levels.
Contracting and Budgeting
Given external sponsorship of COVID-19 trials, efficient contracting and budgeting were critical for a rapid start up. The variability of processes associated with these trials created several challenges that were compounded by issues, such as site sub-agreements and budget documents that did not always go to the correct groups and individuals. Furthermore, the VA’s ability to use contracted resources (eg, tents, trailers, personnel) that external sponsors had built into their contracts was more difficult for VA as a federal agency governed by other statues and policies. This also put VAMCs at a disadvantage from a timing perspective, as the VA often required additional time to find equivalent solutions that met federal regulations.
Although the VA was able to establish contract solutions to some issues, time was still lost while working to secure initial funding. Additionally, for needs such as home phlebotomy—commended for convenience to veterans and research staff—and engaging a specialized research team in the Office of General Counsel, early awareness of protocol needs and sponsor solutions could allow VA to pursue alternatives sooner.
Central-Level Systems and Processes
Not all challenges were at the VAMC level. As the ORD explored solutions, it learned that various tools and study platforms were available but not considered. Applications, such as eConsent, and file-sharing platforms that met existing information security and privacy requirements were needed but had to comply with the Privacy Act of 1974, Federal Information Security Modernization Act, and other requirements. Using sponsor-provided devices, such as drug temperature monitoring equipment, required additional review to ensure that they met system requirements for a national health care system. In addition, the VA uses a clinical trials management review system; however, its implementation was new at the time these trials began. Furthermore, the system engaged with some commercial IRBs but not all. This resulted in additional delays as VAMCs and central resources worked to familiarize themselves with the system and procedures.
The ability to work collaboratively across the VA includes having a framework in which key startup processes are standardized. This allows for efficiency and minimizes variability. Also, all stakeholders should understand the importance of holding discussions to identify appropriate solutions, guidance, and instruction. Finally, the VA must strive to be more nimble when adapting technological, regulatory, and financial processes.
Internal and External Communication
The value of communication—both internal and external—cannot be understated. Minimizing confusion, managing expectations, and ensuring consistent messaging were essential for rapid trial execution. Despite being the second largest federal agency, the VA did not have a seat at the study leadership table for several protocols. When it joined later, several study aspects were set and/or difficult to revise. Challenges affecting time and securing resources have been noted. The ability to plan and then share expectations and responsibilities across and within the respective participating organizations early in the process was perhaps the single factor that was most addressable. The VA enterprise organization and integration with other units could accentuate key communications that would be essential in time-sensitive activities.
VA as a Partner for Future Research
Before the pandemic, the VA had already undertaken a path to enhance its ability to partner as part of the national biomedical research enterprise. The need for COVID-19 therapeutic and vaccine trials accelerated opportunities to plan and develop processes and capabilities to advance this path. As a key strength for VA scientific activities, clinical trials represent a primary medium by which to develop its partnerships. Learning and development have become part of a culture that expedites opportunities for veterans who actively seek ways to contribute to medical knowledge and treatments for their peers and the nation.
CONCLUSIONS
Challenges associated with rapid startup and completion of clinical trials have been discussed for some time. During the pandemic, needs and barriers were magnified because of the heightened urgency for evidence-based therapeutics and vaccines. While the VA faced similar problems as well as those specific to it as a health care system, it had the opportunity to learn and more systematically implement solutions to help in its partnered efforts.10 As an enterprise, the VA hopes to apply lessons learned, strategies, and best practices to further its goals to enhance veteran access to clinical trials and respond to any future need to quickly establish evidence bases in pandemics and other health emergencies that warrant the rapid implementation of research.
Acknowledgments
The activities reported here were supported by the US Department of Veterans Affairs, Office of Research and Development.
The US Department of Veterans Affairs (VA), through its Office of Research and Development (ORD), supports an extensive and experienced clinical research enterprise, including the first multisite trials in the US.1 These resources contribute to the ORD support for the largest US integrated health care system, with a primary focus on the care and well-being of veterans. While the history of VA research has facilitated the creation of an experienced and organized research enterprise, the COVID-19 pandemic challenged VA to contribute even more significantly. These challenges became pronounced given the urgency associated with standing up VA sites for both therapeutic and vaccine trials.
VA Clinical Research Enterprise
The VA recognized an early need for an organized research response not only to address operational challenges resulting from COVID-19 but also ensure that the agency would be ready to support new scientific efforts focused specifically on the virus and related outcomes.2 As a result, the ORD took decisive action first by establishing itself as a central headquarters for VA COVID-19 research activities, and second, by leveraging existing resources, initiatives, and infrastructure to develop new mechanisms that would ensure that the VA was well positioned to develop or participate in research endeavors being driven by the VA as well federal, industry, and non-VA partners.
Prior to the pandemic, the ORD, through its Cooperative Studies Program (CSP), had strategies to address challenges associated with clinical trial startup and improved efficient conduct.3 For example, the VA Network of Dedicated Enrollment Sites (NODES) is a consortium of 23 VA medical centers (VAMCs) dedicated to rapid startup and recruitment into VA-sponsored clinical trials. NODES provides site-level expertise on clinical trial management, including troubleshooting challenges that may occur during clinical research execution.4 Another initiative, Access to Clinical Trials (ACT) for Veterans, engaged industry, academic, patient advocacy, and other partners to identify potential regulatory and operational hurdles to efficient startup activities specific to externally sponsored multisite clinical trials. Under ACT for Veterans, stakeholders emphasized the importance of developing a single VA point of contact for external partners to work with to more efficiently understand and navigate the VA system. In turn, such a resource could be designed to facilitate substantive research and long-term relationships with compatible external partners. Targeted to launch in April 2020, the Partnered Research Program (PRP) was expedited to respond to the pandemic.
During the pandemic, new VA efforts included the creation of the VA CoronavirUs Research and Efficacy Studies (VA CURES) network, initially established as a clinical trial master protocol framework to support and maximize VA-funded COVID-19 trial efficiency.5 VA CURES joined the consortium of trials networks funded by the National Heart, Lung, and Blood Institute. It began treatment trials under Accelerating COVID-19 Therapeutic Interventions and Vaccination (ACTIV), specifically ACTIV-4. The VA also partnered with the National Institutes of Allergy and Infectious Diseases (NIAID) by organizing the VA International Coordinating Center (VA ICC) for other ACTIV trials (ACTIV-2 and -3). When approached to startup studies that included veterans and the VA health care system, these capabilities comprised the VA research response.
A Need for a New Approach
As the impact of the pandemic expanded and the need for effective treatments and vaccines grew, national calls were made to assess the capabilities and readiness of available clinical trials networks. Additionally, the US Department of Health and Human Services Biomedical Advanced Research and Development Authority, ACTIV, NIAID Division of Clinical Research and Division of AIDS, and many pharmaceutical companies were starting to roll out trials of new therapeutics and vaccines. These groups approached the VA to help evaluate the safety and efficacy of several therapeutics and vaccines because they recognized several advantages of the VA enterprise, including its position as the nation’s largest integrated health care system, its diverse patient population, and its expertise in conducting clinical trials.
Although the VA was well positioned as an important player in a collaborative investigational approach to COVID-19 research, these trials required startup approaches that were significantly different from those it had employed in traditional, prepandemic, clinical research. Despite the VA being a single federal agency, each VAMC conducting research establishes its own practices to address both operational and regulatory requirements. This structure results in individual units that operate under different standard operating procedures. Efforts must be taken centrally to organize them into a singular network for the entire health care system. During a national crisis, when there was a need for rapid trial startup to answer safety and efficacy questions and participate under a common approach to protocol execution, this variability was neither manageable nor acceptable. Additionally, the intense resource demands associated with such research, coupled with frequent reporting requirements by VA leaders, Congress, and the White House, required that VAMCs function more like a single unit. Therefore, the ORD needed to develop VAMCs’ abilities to work collectively toward a common goal, share knowledge and experience, and capitalize on potential efficiencies concerning legal, regulatory, and operational processes.
Beginning August 2020, 39 VAMCs joined 7 large-scale collaborative COVID-19 therapeutic and vaccine trials. Through its COVID-19 Research Response Team, the ORD identified, engaged, and directed appropriate resources to support the VAMC under a centralized framework for study management (Table). Centralized management not only afforded VAMCs the opportunity to work more collectively and efficiently but also provided an important advantage by enabling the VA to collect and organize its experiences (and on occasion data) to provide a base for continual learning and improvement efforts. While others have described efforts undertaken across networks to advance learning health systems, the VA’s national scope and integration of research and clinical care allow greater opportunities to learn in a practical setting.6
Challenges and Best Practices
Using surveys, webinars, interviews, and observation from site and VA Central Office personnel, the ORD identified specific variables that prevented the VAMCs from quickly starting up as a clinical trial site. We also documented strategies, solutions, and recommendations for improving startup time lines. These were organized into 8 categories: (1) site infrastructure needs and capabilities; (2) study management roles and responsibilities; (3) educational resources and training; (4) local review requirements and procedures; (5) study design demands; (6) contracting and budgeting; (7) central-level systems and processes; and (8) communication between external partners and within the VA.
Site Infrastructure Needs and Capabilities
A primary impediment to rapid study startup was a lack of basic infrastructure, including staff, space, and the agility necessary for the changing demands of high-priority, high-enrolling trials. This observation is not unique to the VA.7 Initially, certain facilities located in hot spots where COVID-19 was more prevalent became high-interest targets for study placement, despite varying degrees of available research infrastructure. Furthermore, pandemic shutdowns and quarantines permitted fewer employees onsite. This resulted in inadequate staffing in personnel needed to support required startup activities and those needed to handle the high volume of study participants who were being recruited, screened, enrolled, and followed. Additionally, as clinical care needs and infection control practices were prioritized, clinical research space was often appropriated for these needs, making it difficult to find the space to conduct trials. Lastly, supply chain issues also posed unique challenges, sometimes making it difficult for participating VAMCs to obtain needed materials, such as IV solution bags of specific sizes and contents, safety injection needles, and IV line filters.
The VA was able to use central purchasing/contracting at coordinating centers or the VA Central Office to support investigators and assist with finding supplies and clinical research space. VAMCs with research operating budgets to cover startup costs were better positioned to handle funding delays. During the pandemic, the ORD further contracted to supply administrative support to research offices to address regulatory and other requirements needed for startup activities. The ability to expand such central contracts to procure clinical research staff and outpatient clinical research space may also prove useful in meeting key needs at a site.
Management Roles and Responsibilities
Ambiguous and variable roles and responsibilities among the various partners and stakeholders represented a challenge given the large-scale, national, or international operations involved in the trials. VA attempts to operate uniformly were further limited given that each sponsor or group had preferred methods for operating and/or organizing work under urgent time lines. For example, one trial involved a coordinating center, a contract research organization, and federal partners that each worked with individual sites. Consequently, VA study teams would receive messages that were conflicting or unclear.
The VA learned that studies need a single “source of truth” and/or central command structure in times of urgency. To mitigate conflicting messages, vaccine trials relied on a clearinghouse through the PRP to interpret requirements or work on behalf of all sites before key actions were taken. For studies with the NIAID, the VA relied on experienced staff at the CSP coordinating center at the Perry Point, Maryland, VAMC before beginning. This approach especially helped with the challenges of understaffing and sites’ lack of familiarity with complex platform trial designs and already-established network practices within the ACTIV-2 and ACTIV-3 studies.
Educational Resources and Training
Since VA participation in externally sponsored, multisite clinical trials traditionally relies on an individual VAMC study team and its local resources, transitioning to centralized approaches for COVID-19 multisite studies created barriers. Many VAMCs were unfamiliar with newer capabilities for more rapid regulatory reviews and approvals involving commercial institutional review boards (IRBs) and central VA information security and privacy reviews. While tools and resources were available to facilitate these processes, real-time use had not been fully tested. As a result, everyone had to learn as they went along.
The simultaneous establishment of workflows required the ORD to centralize operations and provide training and guidance to field personnel. Although many principal investigators and clinical research coordinators had trial experience, training required unlearning previous understandings of requirements to meet urgent time lines. ORD enterprise road maps, central tools, and training materials also were made available on a study-by-study basis. Open communication was vital to train on central study materials while opportunities to discuss, question, and share experiences and ideas were promoted. The ORD also sent regular emails to prepare for upcoming work and/or raise awareness of identified challenges.
Local Review Requirements/Procedures
The clinical trials were impacted by varying VAMC review requirements and approval processes. Although VA policy defines standard requirements, the timing and procedures are left to the individual facility to determine any local factors to accommodate and/or resource availability. While such an approach is well understood within the VA, external sponsors were not as familiar and assumed a more uniform approach across all sites. In response, some VAMCs established ad hoc research and development committee review procedures, allowing study teams to obtain the necessary reviews in a timely fashion. However, not all VAMCs had the infrastructure (especially when clinical personnel had been redeployed to other priorities) to respond with such agility. One critical role of the VA Central Office coordinating entities was to communicate and manage external sponsor and group expectations surrounding individual site review time lines. However, establishing policies and procedures that focus on streamlining local review processes helped to broadly mitigate the COVID-19 trial challenges.
Study Design Demands
The design of COVID-19 studies combined with the uncertainty of the pandemic required rapid protocol changes and adaptations that were often difficult to deliver. The multinetwork trials that the VA collaborated on were platform or master protocol designs. These designs emphasized overall goals (eg, treating patients requiring intensive care unit care). However, because this trial strategy also introduces complexities that may impact review and execution among those unfamiliar with it, there is a need for increased discussion and understanding of this methodology.8 For example, there can be shared control groups, reliance on specific criteria for halting because of safety or futility concerns, or continuation and expansion applied through an external review board. Delays may arise when changes to study protocols occur rapidly or frequently and necessitate new regulatory reviews, negotiation of new agreements, modifications to contracts, changes to entry criteria, etc.
While the VA has adopted a quality by design framework, VA investigators noted many missed opportunities related to looking at outcomes with new diagnostics, studies of serology, outcomes related to vaccinations, and understanding the natural history of disease in these trials.9 The limited opportunities for investigator input suggested that the advantages offered by platform designs were not maximized during pandemic-focused urgencies. It was unclear whether this barrier was created by a general lack of awareness by sponsors or a lack of opportunities. At the very least, quality by design approaches may help avoid redundancies in documentation or study processes at the central and site levels.
Contracting and Budgeting
Given external sponsorship of COVID-19 trials, efficient contracting and budgeting were critical for a rapid start up. The variability of processes associated with these trials created several challenges that were compounded by issues, such as site sub-agreements and budget documents that did not always go to the correct groups and individuals. Furthermore, the VA’s ability to use contracted resources (eg, tents, trailers, personnel) that external sponsors had built into their contracts was more difficult for VA as a federal agency governed by other statues and policies. This also put VAMCs at a disadvantage from a timing perspective, as the VA often required additional time to find equivalent solutions that met federal regulations.
Although the VA was able to establish contract solutions to some issues, time was still lost while working to secure initial funding. Additionally, for needs such as home phlebotomy—commended for convenience to veterans and research staff—and engaging a specialized research team in the Office of General Counsel, early awareness of protocol needs and sponsor solutions could allow VA to pursue alternatives sooner.
Central-Level Systems and Processes
Not all challenges were at the VAMC level. As the ORD explored solutions, it learned that various tools and study platforms were available but not considered. Applications, such as eConsent, and file-sharing platforms that met existing information security and privacy requirements were needed but had to comply with the Privacy Act of 1974, Federal Information Security Modernization Act, and other requirements. Using sponsor-provided devices, such as drug temperature monitoring equipment, required additional review to ensure that they met system requirements for a national health care system. In addition, the VA uses a clinical trials management review system; however, its implementation was new at the time these trials began. Furthermore, the system engaged with some commercial IRBs but not all. This resulted in additional delays as VAMCs and central resources worked to familiarize themselves with the system and procedures.
The ability to work collaboratively across the VA includes having a framework in which key startup processes are standardized. This allows for efficiency and minimizes variability. Also, all stakeholders should understand the importance of holding discussions to identify appropriate solutions, guidance, and instruction. Finally, the VA must strive to be more nimble when adapting technological, regulatory, and financial processes.
Internal and External Communication
The value of communication—both internal and external—cannot be understated. Minimizing confusion, managing expectations, and ensuring consistent messaging were essential for rapid trial execution. Despite being the second largest federal agency, the VA did not have a seat at the study leadership table for several protocols. When it joined later, several study aspects were set and/or difficult to revise. Challenges affecting time and securing resources have been noted. The ability to plan and then share expectations and responsibilities across and within the respective participating organizations early in the process was perhaps the single factor that was most addressable. The VA enterprise organization and integration with other units could accentuate key communications that would be essential in time-sensitive activities.
VA as a Partner for Future Research
Before the pandemic, the VA had already undertaken a path to enhance its ability to partner as part of the national biomedical research enterprise. The need for COVID-19 therapeutic and vaccine trials accelerated opportunities to plan and develop processes and capabilities to advance this path. As a key strength for VA scientific activities, clinical trials represent a primary medium by which to develop its partnerships. Learning and development have become part of a culture that expedites opportunities for veterans who actively seek ways to contribute to medical knowledge and treatments for their peers and the nation.
CONCLUSIONS
Challenges associated with rapid startup and completion of clinical trials have been discussed for some time. During the pandemic, needs and barriers were magnified because of the heightened urgency for evidence-based therapeutics and vaccines. While the VA faced similar problems as well as those specific to it as a health care system, it had the opportunity to learn and more systematically implement solutions to help in its partnered efforts.10 As an enterprise, the VA hopes to apply lessons learned, strategies, and best practices to further its goals to enhance veteran access to clinical trials and respond to any future need to quickly establish evidence bases in pandemics and other health emergencies that warrant the rapid implementation of research.
Acknowledgments
The activities reported here were supported by the US Department of Veterans Affairs, Office of Research and Development.
1. Hays MT; US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development. A historical look at the establishment of the Department of Veterans Affairs Research & Development Program. Accessed August 28, 2023. https://www.research.va.gov/pubs/docs/ORD-85yrHistory.pdf
2. Garcia AP, Huang GD, Arnheim L, Ramoni R, Clancy C. The VA research enterprise: a platform for national partnerships toward evidence building and scientific innovation. Fed Pract. 2023;40(suppl 5):S12-S17. doi:10.12788/fp.0425
3. Johnston SC, Lewis-Hall F, Bajpai A, et al. It’s time to harmonize clinical trial site standards. NAM Perspectives. October 9, 2017. Accessed August 28, 2023. https://nam.edu/wp-content/uploads/2017/10/Its-Time-to-Harmonize-Clinical-Trial-Site-1.pdf
4. Condon DL, Beck D, Kenworthy-Heinige T, et al. A cross-cutting approach to enhancing clinical trial site success: the Department of Veterans Affairs’ Network of Dedicated Enrollment Sites (NODES) model. Contemp Clin Trials Commun. 2017;6:78-84. Published 2017 Mar 29. doi:10.1016/j.conctc.2017.03.006
5. US Food and Drug Administration. Master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry. March 2022. Accessed August 23, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/master-protocols-efficient-clinical-trial-design-strategies-expedite-development-oncology-drugs-and
6. IOM Roundtable on Value & Science-Driven Care; Institute of Medicine. Continuous learning and improvement in health care. In: Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. National Academies Press (US); 2015:chap 2. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK284654 7. Institute of Medicine (US). Building an infrastructure to support clinical trials. In: Envisioning a Transformed Clinical Trials Enterprise in the United States. National Academies Press (US); 2012:chap 5. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK114656
8. Park JJH, Harari O, Dron L, Lester RT, Thorlund K, Mills EJ. An overview of platform trials with a checklist for clinical readers. J Clin Epidemiol. 2020;125:1-8. doi:10.1016/j.jclinepi.2020.04.025
9. Meeker-O’Connell A, Glessner C, Behm M, et al. Enhancing clinical evidence by proactively building quality into clinical trials. Clin Trials. 2016;13(4):439-444. doi:10.1177/1740774516643491
10. McClure J, Asghar A, Krajec A, et al. Clinical trial facilitators: a novel approach to support the execution of clinical research at the study site level. Contemp Clin Trials Commun. 2023;33:101106. doi:10.1016/j.conctc.2023.101106
1. Hays MT; US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development. A historical look at the establishment of the Department of Veterans Affairs Research & Development Program. Accessed August 28, 2023. https://www.research.va.gov/pubs/docs/ORD-85yrHistory.pdf
2. Garcia AP, Huang GD, Arnheim L, Ramoni R, Clancy C. The VA research enterprise: a platform for national partnerships toward evidence building and scientific innovation. Fed Pract. 2023;40(suppl 5):S12-S17. doi:10.12788/fp.0425
3. Johnston SC, Lewis-Hall F, Bajpai A, et al. It’s time to harmonize clinical trial site standards. NAM Perspectives. October 9, 2017. Accessed August 28, 2023. https://nam.edu/wp-content/uploads/2017/10/Its-Time-to-Harmonize-Clinical-Trial-Site-1.pdf
4. Condon DL, Beck D, Kenworthy-Heinige T, et al. A cross-cutting approach to enhancing clinical trial site success: the Department of Veterans Affairs’ Network of Dedicated Enrollment Sites (NODES) model. Contemp Clin Trials Commun. 2017;6:78-84. Published 2017 Mar 29. doi:10.1016/j.conctc.2017.03.006
5. US Food and Drug Administration. Master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry. March 2022. Accessed August 23, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/master-protocols-efficient-clinical-trial-design-strategies-expedite-development-oncology-drugs-and
6. IOM Roundtable on Value & Science-Driven Care; Institute of Medicine. Continuous learning and improvement in health care. In: Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. National Academies Press (US); 2015:chap 2. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK284654 7. Institute of Medicine (US). Building an infrastructure to support clinical trials. In: Envisioning a Transformed Clinical Trials Enterprise in the United States. National Academies Press (US); 2012:chap 5. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK114656
8. Park JJH, Harari O, Dron L, Lester RT, Thorlund K, Mills EJ. An overview of platform trials with a checklist for clinical readers. J Clin Epidemiol. 2020;125:1-8. doi:10.1016/j.jclinepi.2020.04.025
9. Meeker-O’Connell A, Glessner C, Behm M, et al. Enhancing clinical evidence by proactively building quality into clinical trials. Clin Trials. 2016;13(4):439-444. doi:10.1177/1740774516643491
10. McClure J, Asghar A, Krajec A, et al. Clinical trial facilitators: a novel approach to support the execution of clinical research at the study site level. Contemp Clin Trials Commun. 2023;33:101106. doi:10.1016/j.conctc.2023.101106