User login
Bringing you the latest news, research and reviews, exclusive interviews, podcasts, quizzes, and more.
gambling
compulsive behaviors
ammunition
assault rifle
black jack
Boko Haram
bondage
child abuse
cocaine
Daech
drug paraphernalia
explosion
gun
human trafficking
ISIL
ISIS
Islamic caliphate
Islamic state
mixed martial arts
MMA
molestation
national rifle association
NRA
nsfw
pedophile
pedophilia
poker
porn
pornography
psychedelic drug
recreational drug
sex slave rings
slot machine
terrorism
terrorist
Texas hold 'em
UFC
substance abuse
abuseed
abuseer
abusees
abuseing
abusely
abuses
aeolus
aeolused
aeoluser
aeoluses
aeolusing
aeolusly
aeoluss
ahole
aholeed
aholeer
aholees
aholeing
aholely
aholes
alcohol
alcoholed
alcoholer
alcoholes
alcoholing
alcoholly
alcohols
allman
allmaned
allmaner
allmanes
allmaning
allmanly
allmans
alted
altes
alting
altly
alts
analed
analer
anales
analing
anally
analprobe
analprobeed
analprobeer
analprobees
analprobeing
analprobely
analprobes
anals
anilingus
anilingused
anilinguser
anilinguses
anilingusing
anilingusly
anilinguss
anus
anused
anuser
anuses
anusing
anusly
anuss
areola
areolaed
areolaer
areolaes
areolaing
areolaly
areolas
areole
areoleed
areoleer
areolees
areoleing
areolely
areoles
arian
arianed
arianer
arianes
arianing
arianly
arians
aryan
aryaned
aryaner
aryanes
aryaning
aryanly
aryans
asiaed
asiaer
asiaes
asiaing
asialy
asias
ass
ass hole
ass lick
ass licked
ass licker
ass lickes
ass licking
ass lickly
ass licks
assbang
assbanged
assbangeded
assbangeder
assbangedes
assbangeding
assbangedly
assbangeds
assbanger
assbanges
assbanging
assbangly
assbangs
assbangsed
assbangser
assbangses
assbangsing
assbangsly
assbangss
assed
asser
asses
assesed
asseser
asseses
assesing
assesly
assess
assfuck
assfucked
assfucker
assfuckered
assfuckerer
assfuckeres
assfuckering
assfuckerly
assfuckers
assfuckes
assfucking
assfuckly
assfucks
asshat
asshated
asshater
asshates
asshating
asshatly
asshats
assholeed
assholeer
assholees
assholeing
assholely
assholes
assholesed
assholeser
assholeses
assholesing
assholesly
assholess
assing
assly
assmaster
assmastered
assmasterer
assmasteres
assmastering
assmasterly
assmasters
assmunch
assmunched
assmuncher
assmunches
assmunching
assmunchly
assmunchs
asss
asswipe
asswipeed
asswipeer
asswipees
asswipeing
asswipely
asswipes
asswipesed
asswipeser
asswipeses
asswipesing
asswipesly
asswipess
azz
azzed
azzer
azzes
azzing
azzly
azzs
babeed
babeer
babees
babeing
babely
babes
babesed
babeser
babeses
babesing
babesly
babess
ballsac
ballsaced
ballsacer
ballsaces
ballsacing
ballsack
ballsacked
ballsacker
ballsackes
ballsacking
ballsackly
ballsacks
ballsacly
ballsacs
ballsed
ballser
ballses
ballsing
ballsly
ballss
barf
barfed
barfer
barfes
barfing
barfly
barfs
bastard
bastarded
bastarder
bastardes
bastarding
bastardly
bastards
bastardsed
bastardser
bastardses
bastardsing
bastardsly
bastardss
bawdy
bawdyed
bawdyer
bawdyes
bawdying
bawdyly
bawdys
beaner
beanered
beanerer
beaneres
beanering
beanerly
beaners
beardedclam
beardedclamed
beardedclamer
beardedclames
beardedclaming
beardedclamly
beardedclams
beastiality
beastialityed
beastialityer
beastialityes
beastialitying
beastialityly
beastialitys
beatch
beatched
beatcher
beatches
beatching
beatchly
beatchs
beater
beatered
beaterer
beateres
beatering
beaterly
beaters
beered
beerer
beeres
beering
beerly
beeyotch
beeyotched
beeyotcher
beeyotches
beeyotching
beeyotchly
beeyotchs
beotch
beotched
beotcher
beotches
beotching
beotchly
beotchs
biatch
biatched
biatcher
biatches
biatching
biatchly
biatchs
big tits
big titsed
big titser
big titses
big titsing
big titsly
big titss
bigtits
bigtitsed
bigtitser
bigtitses
bigtitsing
bigtitsly
bigtitss
bimbo
bimboed
bimboer
bimboes
bimboing
bimboly
bimbos
bisexualed
bisexualer
bisexuales
bisexualing
bisexually
bisexuals
bitch
bitched
bitcheded
bitcheder
bitchedes
bitcheding
bitchedly
bitcheds
bitcher
bitches
bitchesed
bitcheser
bitcheses
bitchesing
bitchesly
bitchess
bitching
bitchly
bitchs
bitchy
bitchyed
bitchyer
bitchyes
bitchying
bitchyly
bitchys
bleached
bleacher
bleaches
bleaching
bleachly
bleachs
blow job
blow jobed
blow jober
blow jobes
blow jobing
blow jobly
blow jobs
blowed
blower
blowes
blowing
blowjob
blowjobed
blowjober
blowjobes
blowjobing
blowjobly
blowjobs
blowjobsed
blowjobser
blowjobses
blowjobsing
blowjobsly
blowjobss
blowly
blows
boink
boinked
boinker
boinkes
boinking
boinkly
boinks
bollock
bollocked
bollocker
bollockes
bollocking
bollockly
bollocks
bollocksed
bollockser
bollockses
bollocksing
bollocksly
bollockss
bollok
bolloked
bolloker
bollokes
bolloking
bollokly
bolloks
boner
bonered
bonerer
boneres
bonering
bonerly
boners
bonersed
bonerser
bonerses
bonersing
bonersly
bonerss
bong
bonged
bonger
bonges
bonging
bongly
bongs
boob
boobed
boober
boobes
boobies
boobiesed
boobieser
boobieses
boobiesing
boobiesly
boobiess
boobing
boobly
boobs
boobsed
boobser
boobses
boobsing
boobsly
boobss
booby
boobyed
boobyer
boobyes
boobying
boobyly
boobys
booger
boogered
boogerer
boogeres
boogering
boogerly
boogers
bookie
bookieed
bookieer
bookiees
bookieing
bookiely
bookies
bootee
booteeed
booteeer
booteees
booteeing
booteely
bootees
bootie
bootieed
bootieer
bootiees
bootieing
bootiely
booties
booty
bootyed
bootyer
bootyes
bootying
bootyly
bootys
boozeed
boozeer
boozees
boozeing
boozely
boozer
boozered
boozerer
boozeres
boozering
boozerly
boozers
boozes
boozy
boozyed
boozyer
boozyes
boozying
boozyly
boozys
bosomed
bosomer
bosomes
bosoming
bosomly
bosoms
bosomy
bosomyed
bosomyer
bosomyes
bosomying
bosomyly
bosomys
bugger
buggered
buggerer
buggeres
buggering
buggerly
buggers
bukkake
bukkakeed
bukkakeer
bukkakees
bukkakeing
bukkakely
bukkakes
bull shit
bull shited
bull shiter
bull shites
bull shiting
bull shitly
bull shits
bullshit
bullshited
bullshiter
bullshites
bullshiting
bullshitly
bullshits
bullshitsed
bullshitser
bullshitses
bullshitsing
bullshitsly
bullshitss
bullshitted
bullshitteded
bullshitteder
bullshittedes
bullshitteding
bullshittedly
bullshitteds
bullturds
bullturdsed
bullturdser
bullturdses
bullturdsing
bullturdsly
bullturdss
bung
bunged
bunger
bunges
bunging
bungly
bungs
busty
bustyed
bustyer
bustyes
bustying
bustyly
bustys
butt
butt fuck
butt fucked
butt fucker
butt fuckes
butt fucking
butt fuckly
butt fucks
butted
buttes
buttfuck
buttfucked
buttfucker
buttfuckered
buttfuckerer
buttfuckeres
buttfuckering
buttfuckerly
buttfuckers
buttfuckes
buttfucking
buttfuckly
buttfucks
butting
buttly
buttplug
buttpluged
buttpluger
buttpluges
buttpluging
buttplugly
buttplugs
butts
caca
cacaed
cacaer
cacaes
cacaing
cacaly
cacas
cahone
cahoneed
cahoneer
cahonees
cahoneing
cahonely
cahones
cameltoe
cameltoeed
cameltoeer
cameltoees
cameltoeing
cameltoely
cameltoes
carpetmuncher
carpetmunchered
carpetmuncherer
carpetmuncheres
carpetmunchering
carpetmuncherly
carpetmunchers
cawk
cawked
cawker
cawkes
cawking
cawkly
cawks
chinc
chinced
chincer
chinces
chincing
chincly
chincs
chincsed
chincser
chincses
chincsing
chincsly
chincss
chink
chinked
chinker
chinkes
chinking
chinkly
chinks
chode
chodeed
chodeer
chodees
chodeing
chodely
chodes
chodesed
chodeser
chodeses
chodesing
chodesly
chodess
clit
clited
cliter
clites
cliting
clitly
clitoris
clitorised
clitoriser
clitorises
clitorising
clitorisly
clitoriss
clitorus
clitorused
clitoruser
clitoruses
clitorusing
clitorusly
clitoruss
clits
clitsed
clitser
clitses
clitsing
clitsly
clitss
clitty
clittyed
clittyer
clittyes
clittying
clittyly
clittys
cocain
cocaine
cocained
cocaineed
cocaineer
cocainees
cocaineing
cocainely
cocainer
cocaines
cocaining
cocainly
cocains
cock
cock sucker
cock suckered
cock suckerer
cock suckeres
cock suckering
cock suckerly
cock suckers
cockblock
cockblocked
cockblocker
cockblockes
cockblocking
cockblockly
cockblocks
cocked
cocker
cockes
cockholster
cockholstered
cockholsterer
cockholsteres
cockholstering
cockholsterly
cockholsters
cocking
cockknocker
cockknockered
cockknockerer
cockknockeres
cockknockering
cockknockerly
cockknockers
cockly
cocks
cocksed
cockser
cockses
cocksing
cocksly
cocksmoker
cocksmokered
cocksmokerer
cocksmokeres
cocksmokering
cocksmokerly
cocksmokers
cockss
cocksucker
cocksuckered
cocksuckerer
cocksuckeres
cocksuckering
cocksuckerly
cocksuckers
coital
coitaled
coitaler
coitales
coitaling
coitally
coitals
commie
commieed
commieer
commiees
commieing
commiely
commies
condomed
condomer
condomes
condoming
condomly
condoms
coon
cooned
cooner
coones
cooning
coonly
coons
coonsed
coonser
coonses
coonsing
coonsly
coonss
corksucker
corksuckered
corksuckerer
corksuckeres
corksuckering
corksuckerly
corksuckers
cracked
crackwhore
crackwhoreed
crackwhoreer
crackwhorees
crackwhoreing
crackwhorely
crackwhores
crap
craped
craper
crapes
craping
craply
crappy
crappyed
crappyer
crappyes
crappying
crappyly
crappys
cum
cumed
cumer
cumes
cuming
cumly
cummin
cummined
cumminer
cummines
cumming
cumminged
cumminger
cumminges
cumminging
cummingly
cummings
cummining
cumminly
cummins
cums
cumshot
cumshoted
cumshoter
cumshotes
cumshoting
cumshotly
cumshots
cumshotsed
cumshotser
cumshotses
cumshotsing
cumshotsly
cumshotss
cumslut
cumsluted
cumsluter
cumslutes
cumsluting
cumslutly
cumsluts
cumstain
cumstained
cumstainer
cumstaines
cumstaining
cumstainly
cumstains
cunilingus
cunilingused
cunilinguser
cunilinguses
cunilingusing
cunilingusly
cunilinguss
cunnilingus
cunnilingused
cunnilinguser
cunnilinguses
cunnilingusing
cunnilingusly
cunnilinguss
cunny
cunnyed
cunnyer
cunnyes
cunnying
cunnyly
cunnys
cunt
cunted
cunter
cuntes
cuntface
cuntfaceed
cuntfaceer
cuntfacees
cuntfaceing
cuntfacely
cuntfaces
cunthunter
cunthuntered
cunthunterer
cunthunteres
cunthuntering
cunthunterly
cunthunters
cunting
cuntlick
cuntlicked
cuntlicker
cuntlickered
cuntlickerer
cuntlickeres
cuntlickering
cuntlickerly
cuntlickers
cuntlickes
cuntlicking
cuntlickly
cuntlicks
cuntly
cunts
cuntsed
cuntser
cuntses
cuntsing
cuntsly
cuntss
dago
dagoed
dagoer
dagoes
dagoing
dagoly
dagos
dagosed
dagoser
dagoses
dagosing
dagosly
dagoss
dammit
dammited
dammiter
dammites
dammiting
dammitly
dammits
damn
damned
damneded
damneder
damnedes
damneding
damnedly
damneds
damner
damnes
damning
damnit
damnited
damniter
damnites
damniting
damnitly
damnits
damnly
damns
dick
dickbag
dickbaged
dickbager
dickbages
dickbaging
dickbagly
dickbags
dickdipper
dickdippered
dickdipperer
dickdipperes
dickdippering
dickdipperly
dickdippers
dicked
dicker
dickes
dickface
dickfaceed
dickfaceer
dickfacees
dickfaceing
dickfacely
dickfaces
dickflipper
dickflippered
dickflipperer
dickflipperes
dickflippering
dickflipperly
dickflippers
dickhead
dickheaded
dickheader
dickheades
dickheading
dickheadly
dickheads
dickheadsed
dickheadser
dickheadses
dickheadsing
dickheadsly
dickheadss
dicking
dickish
dickished
dickisher
dickishes
dickishing
dickishly
dickishs
dickly
dickripper
dickrippered
dickripperer
dickripperes
dickrippering
dickripperly
dickrippers
dicks
dicksipper
dicksippered
dicksipperer
dicksipperes
dicksippering
dicksipperly
dicksippers
dickweed
dickweeded
dickweeder
dickweedes
dickweeding
dickweedly
dickweeds
dickwhipper
dickwhippered
dickwhipperer
dickwhipperes
dickwhippering
dickwhipperly
dickwhippers
dickzipper
dickzippered
dickzipperer
dickzipperes
dickzippering
dickzipperly
dickzippers
diddle
diddleed
diddleer
diddlees
diddleing
diddlely
diddles
dike
dikeed
dikeer
dikees
dikeing
dikely
dikes
dildo
dildoed
dildoer
dildoes
dildoing
dildoly
dildos
dildosed
dildoser
dildoses
dildosing
dildosly
dildoss
diligaf
diligafed
diligafer
diligafes
diligafing
diligafly
diligafs
dillweed
dillweeded
dillweeder
dillweedes
dillweeding
dillweedly
dillweeds
dimwit
dimwited
dimwiter
dimwites
dimwiting
dimwitly
dimwits
dingle
dingleed
dingleer
dinglees
dingleing
dinglely
dingles
dipship
dipshiped
dipshiper
dipshipes
dipshiping
dipshiply
dipships
dizzyed
dizzyer
dizzyes
dizzying
dizzyly
dizzys
doggiestyleed
doggiestyleer
doggiestylees
doggiestyleing
doggiestylely
doggiestyles
doggystyleed
doggystyleer
doggystylees
doggystyleing
doggystylely
doggystyles
dong
donged
donger
donges
donging
dongly
dongs
doofus
doofused
doofuser
doofuses
doofusing
doofusly
doofuss
doosh
dooshed
doosher
dooshes
dooshing
dooshly
dooshs
dopeyed
dopeyer
dopeyes
dopeying
dopeyly
dopeys
douchebag
douchebaged
douchebager
douchebages
douchebaging
douchebagly
douchebags
douchebagsed
douchebagser
douchebagses
douchebagsing
douchebagsly
douchebagss
doucheed
doucheer
douchees
doucheing
douchely
douches
douchey
doucheyed
doucheyer
doucheyes
doucheying
doucheyly
doucheys
drunk
drunked
drunker
drunkes
drunking
drunkly
drunks
dumass
dumassed
dumasser
dumasses
dumassing
dumassly
dumasss
dumbass
dumbassed
dumbasser
dumbasses
dumbassesed
dumbasseser
dumbasseses
dumbassesing
dumbassesly
dumbassess
dumbassing
dumbassly
dumbasss
dummy
dummyed
dummyer
dummyes
dummying
dummyly
dummys
dyke
dykeed
dykeer
dykees
dykeing
dykely
dykes
dykesed
dykeser
dykeses
dykesing
dykesly
dykess
erotic
eroticed
eroticer
erotices
eroticing
eroticly
erotics
extacy
extacyed
extacyer
extacyes
extacying
extacyly
extacys
extasy
extasyed
extasyer
extasyes
extasying
extasyly
extasys
fack
facked
facker
fackes
facking
fackly
facks
fag
faged
fager
fages
fagg
fagged
faggeded
faggeder
faggedes
faggeding
faggedly
faggeds
fagger
fagges
fagging
faggit
faggited
faggiter
faggites
faggiting
faggitly
faggits
faggly
faggot
faggoted
faggoter
faggotes
faggoting
faggotly
faggots
faggs
faging
fagly
fagot
fagoted
fagoter
fagotes
fagoting
fagotly
fagots
fags
fagsed
fagser
fagses
fagsing
fagsly
fagss
faig
faiged
faiger
faiges
faiging
faigly
faigs
faigt
faigted
faigter
faigtes
faigting
faigtly
faigts
fannybandit
fannybandited
fannybanditer
fannybandites
fannybanditing
fannybanditly
fannybandits
farted
farter
fartes
farting
fartknocker
fartknockered
fartknockerer
fartknockeres
fartknockering
fartknockerly
fartknockers
fartly
farts
felch
felched
felcher
felchered
felcherer
felcheres
felchering
felcherly
felchers
felches
felching
felchinged
felchinger
felchinges
felchinging
felchingly
felchings
felchly
felchs
fellate
fellateed
fellateer
fellatees
fellateing
fellately
fellates
fellatio
fellatioed
fellatioer
fellatioes
fellatioing
fellatioly
fellatios
feltch
feltched
feltcher
feltchered
feltcherer
feltcheres
feltchering
feltcherly
feltchers
feltches
feltching
feltchly
feltchs
feom
feomed
feomer
feomes
feoming
feomly
feoms
fisted
fisteded
fisteder
fistedes
fisteding
fistedly
fisteds
fisting
fistinged
fistinger
fistinges
fistinging
fistingly
fistings
fisty
fistyed
fistyer
fistyes
fistying
fistyly
fistys
floozy
floozyed
floozyer
floozyes
floozying
floozyly
floozys
foad
foaded
foader
foades
foading
foadly
foads
fondleed
fondleer
fondlees
fondleing
fondlely
fondles
foobar
foobared
foobarer
foobares
foobaring
foobarly
foobars
freex
freexed
freexer
freexes
freexing
freexly
freexs
frigg
frigga
friggaed
friggaer
friggaes
friggaing
friggaly
friggas
frigged
frigger
frigges
frigging
friggly
friggs
fubar
fubared
fubarer
fubares
fubaring
fubarly
fubars
fuck
fuckass
fuckassed
fuckasser
fuckasses
fuckassing
fuckassly
fuckasss
fucked
fuckeded
fuckeder
fuckedes
fuckeding
fuckedly
fuckeds
fucker
fuckered
fuckerer
fuckeres
fuckering
fuckerly
fuckers
fuckes
fuckface
fuckfaceed
fuckfaceer
fuckfacees
fuckfaceing
fuckfacely
fuckfaces
fuckin
fuckined
fuckiner
fuckines
fucking
fuckinged
fuckinger
fuckinges
fuckinging
fuckingly
fuckings
fuckining
fuckinly
fuckins
fuckly
fucknugget
fucknuggeted
fucknuggeter
fucknuggetes
fucknuggeting
fucknuggetly
fucknuggets
fucknut
fucknuted
fucknuter
fucknutes
fucknuting
fucknutly
fucknuts
fuckoff
fuckoffed
fuckoffer
fuckoffes
fuckoffing
fuckoffly
fuckoffs
fucks
fucksed
fuckser
fuckses
fucksing
fucksly
fuckss
fucktard
fucktarded
fucktarder
fucktardes
fucktarding
fucktardly
fucktards
fuckup
fuckuped
fuckuper
fuckupes
fuckuping
fuckuply
fuckups
fuckwad
fuckwaded
fuckwader
fuckwades
fuckwading
fuckwadly
fuckwads
fuckwit
fuckwited
fuckwiter
fuckwites
fuckwiting
fuckwitly
fuckwits
fudgepacker
fudgepackered
fudgepackerer
fudgepackeres
fudgepackering
fudgepackerly
fudgepackers
fuk
fuked
fuker
fukes
fuking
fukly
fuks
fvck
fvcked
fvcker
fvckes
fvcking
fvckly
fvcks
fxck
fxcked
fxcker
fxckes
fxcking
fxckly
fxcks
gae
gaeed
gaeer
gaees
gaeing
gaely
gaes
gai
gaied
gaier
gaies
gaiing
gaily
gais
ganja
ganjaed
ganjaer
ganjaes
ganjaing
ganjaly
ganjas
gayed
gayer
gayes
gaying
gayly
gays
gaysed
gayser
gayses
gaysing
gaysly
gayss
gey
geyed
geyer
geyes
geying
geyly
geys
gfc
gfced
gfcer
gfces
gfcing
gfcly
gfcs
gfy
gfyed
gfyer
gfyes
gfying
gfyly
gfys
ghay
ghayed
ghayer
ghayes
ghaying
ghayly
ghays
ghey
gheyed
gheyer
gheyes
gheying
gheyly
gheys
gigolo
gigoloed
gigoloer
gigoloes
gigoloing
gigololy
gigolos
goatse
goatseed
goatseer
goatsees
goatseing
goatsely
goatses
godamn
godamned
godamner
godamnes
godamning
godamnit
godamnited
godamniter
godamnites
godamniting
godamnitly
godamnits
godamnly
godamns
goddam
goddamed
goddamer
goddames
goddaming
goddamly
goddammit
goddammited
goddammiter
goddammites
goddammiting
goddammitly
goddammits
goddamn
goddamned
goddamner
goddamnes
goddamning
goddamnly
goddamns
goddams
goldenshower
goldenshowered
goldenshowerer
goldenshoweres
goldenshowering
goldenshowerly
goldenshowers
gonad
gonaded
gonader
gonades
gonading
gonadly
gonads
gonadsed
gonadser
gonadses
gonadsing
gonadsly
gonadss
gook
gooked
gooker
gookes
gooking
gookly
gooks
gooksed
gookser
gookses
gooksing
gooksly
gookss
gringo
gringoed
gringoer
gringoes
gringoing
gringoly
gringos
gspot
gspoted
gspoter
gspotes
gspoting
gspotly
gspots
gtfo
gtfoed
gtfoer
gtfoes
gtfoing
gtfoly
gtfos
guido
guidoed
guidoer
guidoes
guidoing
guidoly
guidos
handjob
handjobed
handjober
handjobes
handjobing
handjobly
handjobs
hard on
hard oned
hard oner
hard ones
hard oning
hard only
hard ons
hardknight
hardknighted
hardknighter
hardknightes
hardknighting
hardknightly
hardknights
hebe
hebeed
hebeer
hebees
hebeing
hebely
hebes
heeb
heebed
heeber
heebes
heebing
heebly
heebs
hell
helled
heller
helles
helling
hellly
hells
hemp
hemped
hemper
hempes
hemping
hemply
hemps
heroined
heroiner
heroines
heroining
heroinly
heroins
herp
herped
herper
herpes
herpesed
herpeser
herpeses
herpesing
herpesly
herpess
herping
herply
herps
herpy
herpyed
herpyer
herpyes
herpying
herpyly
herpys
hitler
hitlered
hitlerer
hitleres
hitlering
hitlerly
hitlers
hived
hiver
hives
hiving
hivly
hivs
hobag
hobaged
hobager
hobages
hobaging
hobagly
hobags
homey
homeyed
homeyer
homeyes
homeying
homeyly
homeys
homo
homoed
homoer
homoes
homoey
homoeyed
homoeyer
homoeyes
homoeying
homoeyly
homoeys
homoing
homoly
homos
honky
honkyed
honkyer
honkyes
honkying
honkyly
honkys
hooch
hooched
hoocher
hooches
hooching
hoochly
hoochs
hookah
hookahed
hookaher
hookahes
hookahing
hookahly
hookahs
hooker
hookered
hookerer
hookeres
hookering
hookerly
hookers
hoor
hoored
hoorer
hoores
hooring
hoorly
hoors
hootch
hootched
hootcher
hootches
hootching
hootchly
hootchs
hooter
hootered
hooterer
hooteres
hootering
hooterly
hooters
hootersed
hooterser
hooterses
hootersing
hootersly
hooterss
horny
hornyed
hornyer
hornyes
hornying
hornyly
hornys
houstoned
houstoner
houstones
houstoning
houstonly
houstons
hump
humped
humpeded
humpeder
humpedes
humpeding
humpedly
humpeds
humper
humpes
humping
humpinged
humpinger
humpinges
humpinging
humpingly
humpings
humply
humps
husbanded
husbander
husbandes
husbanding
husbandly
husbands
hussy
hussyed
hussyer
hussyes
hussying
hussyly
hussys
hymened
hymener
hymenes
hymening
hymenly
hymens
inbred
inbreded
inbreder
inbredes
inbreding
inbredly
inbreds
incest
incested
incester
incestes
incesting
incestly
incests
injun
injuned
injuner
injunes
injuning
injunly
injuns
jackass
jackassed
jackasser
jackasses
jackassing
jackassly
jackasss
jackhole
jackholeed
jackholeer
jackholees
jackholeing
jackholely
jackholes
jackoff
jackoffed
jackoffer
jackoffes
jackoffing
jackoffly
jackoffs
jap
japed
japer
japes
japing
japly
japs
japsed
japser
japses
japsing
japsly
japss
jerkoff
jerkoffed
jerkoffer
jerkoffes
jerkoffing
jerkoffly
jerkoffs
jerks
jism
jismed
jismer
jismes
jisming
jismly
jisms
jiz
jized
jizer
jizes
jizing
jizly
jizm
jizmed
jizmer
jizmes
jizming
jizmly
jizms
jizs
jizz
jizzed
jizzeded
jizzeder
jizzedes
jizzeding
jizzedly
jizzeds
jizzer
jizzes
jizzing
jizzly
jizzs
junkie
junkieed
junkieer
junkiees
junkieing
junkiely
junkies
junky
junkyed
junkyer
junkyes
junkying
junkyly
junkys
kike
kikeed
kikeer
kikees
kikeing
kikely
kikes
kikesed
kikeser
kikeses
kikesing
kikesly
kikess
killed
killer
killes
killing
killly
kills
kinky
kinkyed
kinkyer
kinkyes
kinkying
kinkyly
kinkys
kkk
kkked
kkker
kkkes
kkking
kkkly
kkks
klan
klaned
klaner
klanes
klaning
klanly
klans
knobend
knobended
knobender
knobendes
knobending
knobendly
knobends
kooch
kooched
koocher
kooches
koochesed
koocheser
koocheses
koochesing
koochesly
koochess
kooching
koochly
koochs
kootch
kootched
kootcher
kootches
kootching
kootchly
kootchs
kraut
krauted
krauter
krautes
krauting
krautly
krauts
kyke
kykeed
kykeer
kykees
kykeing
kykely
kykes
lech
leched
lecher
leches
leching
lechly
lechs
leper
lepered
leperer
leperes
lepering
leperly
lepers
lesbiansed
lesbianser
lesbianses
lesbiansing
lesbiansly
lesbianss
lesbo
lesboed
lesboer
lesboes
lesboing
lesboly
lesbos
lesbosed
lesboser
lesboses
lesbosing
lesbosly
lesboss
lez
lezbianed
lezbianer
lezbianes
lezbianing
lezbianly
lezbians
lezbiansed
lezbianser
lezbianses
lezbiansing
lezbiansly
lezbianss
lezbo
lezboed
lezboer
lezboes
lezboing
lezboly
lezbos
lezbosed
lezboser
lezboses
lezbosing
lezbosly
lezboss
lezed
lezer
lezes
lezing
lezly
lezs
lezzie
lezzieed
lezzieer
lezziees
lezzieing
lezziely
lezzies
lezziesed
lezzieser
lezzieses
lezziesing
lezziesly
lezziess
lezzy
lezzyed
lezzyer
lezzyes
lezzying
lezzyly
lezzys
lmaoed
lmaoer
lmaoes
lmaoing
lmaoly
lmaos
lmfao
lmfaoed
lmfaoer
lmfaoes
lmfaoing
lmfaoly
lmfaos
loined
loiner
loines
loining
loinly
loins
loinsed
loinser
loinses
loinsing
loinsly
loinss
lubeed
lubeer
lubees
lubeing
lubely
lubes
lusty
lustyed
lustyer
lustyes
lustying
lustyly
lustys
massa
massaed
massaer
massaes
massaing
massaly
massas
masterbate
masterbateed
masterbateer
masterbatees
masterbateing
masterbately
masterbates
masterbating
masterbatinged
masterbatinger
masterbatinges
masterbatinging
masterbatingly
masterbatings
masterbation
masterbationed
masterbationer
masterbationes
masterbationing
masterbationly
masterbations
masturbate
masturbateed
masturbateer
masturbatees
masturbateing
masturbately
masturbates
masturbating
masturbatinged
masturbatinger
masturbatinges
masturbatinging
masturbatingly
masturbatings
masturbation
masturbationed
masturbationer
masturbationes
masturbationing
masturbationly
masturbations
methed
mether
methes
mething
methly
meths
militaryed
militaryer
militaryes
militarying
militaryly
militarys
mofo
mofoed
mofoer
mofoes
mofoing
mofoly
mofos
molest
molested
molester
molestes
molesting
molestly
molests
moolie
moolieed
moolieer
mooliees
moolieing
mooliely
moolies
moron
moroned
moroner
morones
moroning
moronly
morons
motherfucka
motherfuckaed
motherfuckaer
motherfuckaes
motherfuckaing
motherfuckaly
motherfuckas
motherfucker
motherfuckered
motherfuckerer
motherfuckeres
motherfuckering
motherfuckerly
motherfuckers
motherfucking
motherfuckinged
motherfuckinger
motherfuckinges
motherfuckinging
motherfuckingly
motherfuckings
mtherfucker
mtherfuckered
mtherfuckerer
mtherfuckeres
mtherfuckering
mtherfuckerly
mtherfuckers
mthrfucker
mthrfuckered
mthrfuckerer
mthrfuckeres
mthrfuckering
mthrfuckerly
mthrfuckers
mthrfucking
mthrfuckinged
mthrfuckinger
mthrfuckinges
mthrfuckinging
mthrfuckingly
mthrfuckings
muff
muffdiver
muffdivered
muffdiverer
muffdiveres
muffdivering
muffdiverly
muffdivers
muffed
muffer
muffes
muffing
muffly
muffs
murdered
murderer
murderes
murdering
murderly
murders
muthafuckaz
muthafuckazed
muthafuckazer
muthafuckazes
muthafuckazing
muthafuckazly
muthafuckazs
muthafucker
muthafuckered
muthafuckerer
muthafuckeres
muthafuckering
muthafuckerly
muthafuckers
mutherfucker
mutherfuckered
mutherfuckerer
mutherfuckeres
mutherfuckering
mutherfuckerly
mutherfuckers
mutherfucking
mutherfuckinged
mutherfuckinger
mutherfuckinges
mutherfuckinging
mutherfuckingly
mutherfuckings
muthrfucking
muthrfuckinged
muthrfuckinger
muthrfuckinges
muthrfuckinging
muthrfuckingly
muthrfuckings
nad
naded
nader
nades
nading
nadly
nads
nadsed
nadser
nadses
nadsing
nadsly
nadss
nakeded
nakeder
nakedes
nakeding
nakedly
nakeds
napalm
napalmed
napalmer
napalmes
napalming
napalmly
napalms
nappy
nappyed
nappyer
nappyes
nappying
nappyly
nappys
nazi
nazied
nazier
nazies
naziing
nazily
nazis
nazism
nazismed
nazismer
nazismes
nazisming
nazismly
nazisms
negro
negroed
negroer
negroes
negroing
negroly
negros
nigga
niggaed
niggaer
niggaes
niggah
niggahed
niggaher
niggahes
niggahing
niggahly
niggahs
niggaing
niggaly
niggas
niggased
niggaser
niggases
niggasing
niggasly
niggass
niggaz
niggazed
niggazer
niggazes
niggazing
niggazly
niggazs
nigger
niggered
niggerer
niggeres
niggering
niggerly
niggers
niggersed
niggerser
niggerses
niggersing
niggersly
niggerss
niggle
niggleed
niggleer
nigglees
niggleing
nigglely
niggles
niglet
nigleted
nigleter
nigletes
nigleting
nigletly
niglets
nimrod
nimroded
nimroder
nimrodes
nimroding
nimrodly
nimrods
ninny
ninnyed
ninnyer
ninnyes
ninnying
ninnyly
ninnys
nooky
nookyed
nookyer
nookyes
nookying
nookyly
nookys
nuccitelli
nuccitellied
nuccitellier
nuccitellies
nuccitelliing
nuccitellily
nuccitellis
nympho
nymphoed
nymphoer
nymphoes
nymphoing
nympholy
nymphos
opium
opiumed
opiumer
opiumes
opiuming
opiumly
opiums
orgies
orgiesed
orgieser
orgieses
orgiesing
orgiesly
orgiess
orgy
orgyed
orgyer
orgyes
orgying
orgyly
orgys
paddy
paddyed
paddyer
paddyes
paddying
paddyly
paddys
paki
pakied
pakier
pakies
pakiing
pakily
pakis
pantie
pantieed
pantieer
pantiees
pantieing
pantiely
panties
pantiesed
pantieser
pantieses
pantiesing
pantiesly
pantiess
panty
pantyed
pantyer
pantyes
pantying
pantyly
pantys
pastie
pastieed
pastieer
pastiees
pastieing
pastiely
pasties
pasty
pastyed
pastyer
pastyes
pastying
pastyly
pastys
pecker
peckered
peckerer
peckeres
peckering
peckerly
peckers
pedo
pedoed
pedoer
pedoes
pedoing
pedoly
pedophile
pedophileed
pedophileer
pedophilees
pedophileing
pedophilely
pedophiles
pedophilia
pedophiliac
pedophiliaced
pedophiliacer
pedophiliaces
pedophiliacing
pedophiliacly
pedophiliacs
pedophiliaed
pedophiliaer
pedophiliaes
pedophiliaing
pedophilialy
pedophilias
pedos
penial
penialed
penialer
peniales
penialing
penially
penials
penile
penileed
penileer
penilees
penileing
penilely
peniles
penis
penised
peniser
penises
penising
penisly
peniss
perversion
perversioned
perversioner
perversiones
perversioning
perversionly
perversions
peyote
peyoteed
peyoteer
peyotees
peyoteing
peyotely
peyotes
phuck
phucked
phucker
phuckes
phucking
phuckly
phucks
pillowbiter
pillowbitered
pillowbiterer
pillowbiteres
pillowbitering
pillowbiterly
pillowbiters
pimp
pimped
pimper
pimpes
pimping
pimply
pimps
pinko
pinkoed
pinkoer
pinkoes
pinkoing
pinkoly
pinkos
pissed
pisseded
pisseder
pissedes
pisseding
pissedly
pisseds
pisser
pisses
pissing
pissly
pissoff
pissoffed
pissoffer
pissoffes
pissoffing
pissoffly
pissoffs
pisss
polack
polacked
polacker
polackes
polacking
polackly
polacks
pollock
pollocked
pollocker
pollockes
pollocking
pollockly
pollocks
poon
pooned
pooner
poones
pooning
poonly
poons
poontang
poontanged
poontanger
poontanges
poontanging
poontangly
poontangs
porn
porned
porner
pornes
porning
pornly
porno
pornoed
pornoer
pornoes
pornography
pornographyed
pornographyer
pornographyes
pornographying
pornographyly
pornographys
pornoing
pornoly
pornos
porns
prick
pricked
pricker
prickes
pricking
prickly
pricks
prig
priged
priger
priges
priging
prigly
prigs
prostitute
prostituteed
prostituteer
prostitutees
prostituteing
prostitutely
prostitutes
prude
prudeed
prudeer
prudees
prudeing
prudely
prudes
punkass
punkassed
punkasser
punkasses
punkassing
punkassly
punkasss
punky
punkyed
punkyer
punkyes
punkying
punkyly
punkys
puss
pussed
pusser
pusses
pussies
pussiesed
pussieser
pussieses
pussiesing
pussiesly
pussiess
pussing
pussly
pusss
pussy
pussyed
pussyer
pussyes
pussying
pussyly
pussypounder
pussypoundered
pussypounderer
pussypounderes
pussypoundering
pussypounderly
pussypounders
pussys
puto
putoed
putoer
putoes
putoing
putoly
putos
queaf
queafed
queafer
queafes
queafing
queafly
queafs
queef
queefed
queefer
queefes
queefing
queefly
queefs
queer
queered
queerer
queeres
queering
queerly
queero
queeroed
queeroer
queeroes
queeroing
queeroly
queeros
queers
queersed
queerser
queerses
queersing
queersly
queerss
quicky
quickyed
quickyer
quickyes
quickying
quickyly
quickys
quim
quimed
quimer
quimes
quiming
quimly
quims
racy
racyed
racyer
racyes
racying
racyly
racys
rape
raped
rapeded
rapeder
rapedes
rapeding
rapedly
rapeds
rapeed
rapeer
rapees
rapeing
rapely
raper
rapered
raperer
raperes
rapering
raperly
rapers
rapes
rapist
rapisted
rapister
rapistes
rapisting
rapistly
rapists
raunch
raunched
rauncher
raunches
raunching
raunchly
raunchs
rectus
rectused
rectuser
rectuses
rectusing
rectusly
rectuss
reefer
reefered
reeferer
reeferes
reefering
reeferly
reefers
reetard
reetarded
reetarder
reetardes
reetarding
reetardly
reetards
reich
reiched
reicher
reiches
reiching
reichly
reichs
retard
retarded
retardeded
retardeder
retardedes
retardeding
retardedly
retardeds
retarder
retardes
retarding
retardly
retards
rimjob
rimjobed
rimjober
rimjobes
rimjobing
rimjobly
rimjobs
ritard
ritarded
ritarder
ritardes
ritarding
ritardly
ritards
rtard
rtarded
rtarder
rtardes
rtarding
rtardly
rtards
rum
rumed
rumer
rumes
ruming
rumly
rump
rumped
rumper
rumpes
rumping
rumply
rumprammer
rumprammered
rumprammerer
rumprammeres
rumprammering
rumprammerly
rumprammers
rumps
rums
ruski
ruskied
ruskier
ruskies
ruskiing
ruskily
ruskis
sadism
sadismed
sadismer
sadismes
sadisming
sadismly
sadisms
sadist
sadisted
sadister
sadistes
sadisting
sadistly
sadists
scag
scaged
scager
scages
scaging
scagly
scags
scantily
scantilyed
scantilyer
scantilyes
scantilying
scantilyly
scantilys
schlong
schlonged
schlonger
schlonges
schlonging
schlongly
schlongs
scrog
scroged
scroger
scroges
scroging
scrogly
scrogs
scrot
scrote
scroted
scroteed
scroteer
scrotees
scroteing
scrotely
scroter
scrotes
scroting
scrotly
scrots
scrotum
scrotumed
scrotumer
scrotumes
scrotuming
scrotumly
scrotums
scrud
scruded
scruder
scrudes
scruding
scrudly
scruds
scum
scumed
scumer
scumes
scuming
scumly
scums
seaman
seamaned
seamaner
seamanes
seamaning
seamanly
seamans
seamen
seamened
seamener
seamenes
seamening
seamenly
seamens
seduceed
seduceer
seducees
seduceing
seducely
seduces
semen
semened
semener
semenes
semening
semenly
semens
shamedame
shamedameed
shamedameer
shamedamees
shamedameing
shamedamely
shamedames
shit
shite
shiteater
shiteatered
shiteaterer
shiteateres
shiteatering
shiteaterly
shiteaters
shited
shiteed
shiteer
shitees
shiteing
shitely
shiter
shites
shitface
shitfaceed
shitfaceer
shitfacees
shitfaceing
shitfacely
shitfaces
shithead
shitheaded
shitheader
shitheades
shitheading
shitheadly
shitheads
shithole
shitholeed
shitholeer
shitholees
shitholeing
shitholely
shitholes
shithouse
shithouseed
shithouseer
shithousees
shithouseing
shithousely
shithouses
shiting
shitly
shits
shitsed
shitser
shitses
shitsing
shitsly
shitss
shitt
shitted
shitteded
shitteder
shittedes
shitteding
shittedly
shitteds
shitter
shittered
shitterer
shitteres
shittering
shitterly
shitters
shittes
shitting
shittly
shitts
shitty
shittyed
shittyer
shittyes
shittying
shittyly
shittys
shiz
shized
shizer
shizes
shizing
shizly
shizs
shooted
shooter
shootes
shooting
shootly
shoots
sissy
sissyed
sissyer
sissyes
sissying
sissyly
sissys
skag
skaged
skager
skages
skaging
skagly
skags
skank
skanked
skanker
skankes
skanking
skankly
skanks
slave
slaveed
slaveer
slavees
slaveing
slavely
slaves
sleaze
sleazeed
sleazeer
sleazees
sleazeing
sleazely
sleazes
sleazy
sleazyed
sleazyer
sleazyes
sleazying
sleazyly
sleazys
slut
slutdumper
slutdumpered
slutdumperer
slutdumperes
slutdumpering
slutdumperly
slutdumpers
sluted
sluter
slutes
sluting
slutkiss
slutkissed
slutkisser
slutkisses
slutkissing
slutkissly
slutkisss
slutly
sluts
slutsed
slutser
slutses
slutsing
slutsly
slutss
smegma
smegmaed
smegmaer
smegmaes
smegmaing
smegmaly
smegmas
smut
smuted
smuter
smutes
smuting
smutly
smuts
smutty
smuttyed
smuttyer
smuttyes
smuttying
smuttyly
smuttys
snatch
snatched
snatcher
snatches
snatching
snatchly
snatchs
sniper
snipered
sniperer
sniperes
snipering
sniperly
snipers
snort
snorted
snorter
snortes
snorting
snortly
snorts
snuff
snuffed
snuffer
snuffes
snuffing
snuffly
snuffs
sodom
sodomed
sodomer
sodomes
sodoming
sodomly
sodoms
spic
spiced
spicer
spices
spicing
spick
spicked
spicker
spickes
spicking
spickly
spicks
spicly
spics
spik
spoof
spoofed
spoofer
spoofes
spoofing
spoofly
spoofs
spooge
spoogeed
spoogeer
spoogees
spoogeing
spoogely
spooges
spunk
spunked
spunker
spunkes
spunking
spunkly
spunks
steamyed
steamyer
steamyes
steamying
steamyly
steamys
stfu
stfued
stfuer
stfues
stfuing
stfuly
stfus
stiffy
stiffyed
stiffyer
stiffyes
stiffying
stiffyly
stiffys
stoneded
stoneder
stonedes
stoneding
stonedly
stoneds
stupided
stupider
stupides
stupiding
stupidly
stupids
suckeded
suckeder
suckedes
suckeding
suckedly
suckeds
sucker
suckes
sucking
suckinged
suckinger
suckinges
suckinging
suckingly
suckings
suckly
sucks
sumofabiatch
sumofabiatched
sumofabiatcher
sumofabiatches
sumofabiatching
sumofabiatchly
sumofabiatchs
tard
tarded
tarder
tardes
tarding
tardly
tards
tawdry
tawdryed
tawdryer
tawdryes
tawdrying
tawdryly
tawdrys
teabagging
teabagginged
teabagginger
teabagginges
teabagginging
teabaggingly
teabaggings
terd
terded
terder
terdes
terding
terdly
terds
teste
testee
testeed
testeeed
testeeer
testeees
testeeing
testeely
testeer
testees
testeing
testely
testes
testesed
testeser
testeses
testesing
testesly
testess
testicle
testicleed
testicleer
testiclees
testicleing
testiclely
testicles
testis
testised
testiser
testises
testising
testisly
testiss
thrusted
thruster
thrustes
thrusting
thrustly
thrusts
thug
thuged
thuger
thuges
thuging
thugly
thugs
tinkle
tinkleed
tinkleer
tinklees
tinkleing
tinklely
tinkles
tit
tited
titer
tites
titfuck
titfucked
titfucker
titfuckes
titfucking
titfuckly
titfucks
titi
titied
titier
tities
titiing
titily
titing
titis
titly
tits
titsed
titser
titses
titsing
titsly
titss
tittiefucker
tittiefuckered
tittiefuckerer
tittiefuckeres
tittiefuckering
tittiefuckerly
tittiefuckers
titties
tittiesed
tittieser
tittieses
tittiesing
tittiesly
tittiess
titty
tittyed
tittyer
tittyes
tittyfuck
tittyfucked
tittyfucker
tittyfuckered
tittyfuckerer
tittyfuckeres
tittyfuckering
tittyfuckerly
tittyfuckers
tittyfuckes
tittyfucking
tittyfuckly
tittyfucks
tittying
tittyly
tittys
toke
tokeed
tokeer
tokees
tokeing
tokely
tokes
toots
tootsed
tootser
tootses
tootsing
tootsly
tootss
tramp
tramped
tramper
trampes
tramping
tramply
tramps
transsexualed
transsexualer
transsexuales
transsexualing
transsexually
transsexuals
trashy
trashyed
trashyer
trashyes
trashying
trashyly
trashys
tubgirl
tubgirled
tubgirler
tubgirles
tubgirling
tubgirlly
tubgirls
turd
turded
turder
turdes
turding
turdly
turds
tush
tushed
tusher
tushes
tushing
tushly
tushs
twat
twated
twater
twates
twating
twatly
twats
twatsed
twatser
twatses
twatsing
twatsly
twatss
undies
undiesed
undieser
undieses
undiesing
undiesly
undiess
unweded
unweder
unwedes
unweding
unwedly
unweds
uzi
uzied
uzier
uzies
uziing
uzily
uzis
vag
vaged
vager
vages
vaging
vagly
vags
valium
valiumed
valiumer
valiumes
valiuming
valiumly
valiums
venous
virgined
virginer
virgines
virgining
virginly
virgins
vixen
vixened
vixener
vixenes
vixening
vixenly
vixens
vodkaed
vodkaer
vodkaes
vodkaing
vodkaly
vodkas
voyeur
voyeured
voyeurer
voyeures
voyeuring
voyeurly
voyeurs
vulgar
vulgared
vulgarer
vulgares
vulgaring
vulgarly
vulgars
wang
wanged
wanger
wanges
wanging
wangly
wangs
wank
wanked
wanker
wankered
wankerer
wankeres
wankering
wankerly
wankers
wankes
wanking
wankly
wanks
wazoo
wazooed
wazooer
wazooes
wazooing
wazooly
wazoos
wedgie
wedgieed
wedgieer
wedgiees
wedgieing
wedgiely
wedgies
weeded
weeder
weedes
weeding
weedly
weeds
weenie
weenieed
weenieer
weeniees
weenieing
weeniely
weenies
weewee
weeweeed
weeweeer
weeweees
weeweeing
weeweely
weewees
weiner
weinered
weinerer
weineres
weinering
weinerly
weiners
weirdo
weirdoed
weirdoer
weirdoes
weirdoing
weirdoly
weirdos
wench
wenched
wencher
wenches
wenching
wenchly
wenchs
wetback
wetbacked
wetbacker
wetbackes
wetbacking
wetbackly
wetbacks
whitey
whiteyed
whiteyer
whiteyes
whiteying
whiteyly
whiteys
whiz
whized
whizer
whizes
whizing
whizly
whizs
whoralicious
whoralicioused
whoraliciouser
whoraliciouses
whoraliciousing
whoraliciously
whoraliciouss
whore
whorealicious
whorealicioused
whorealiciouser
whorealiciouses
whorealiciousing
whorealiciously
whorealiciouss
whored
whoreded
whoreder
whoredes
whoreding
whoredly
whoreds
whoreed
whoreer
whorees
whoreface
whorefaceed
whorefaceer
whorefacees
whorefaceing
whorefacely
whorefaces
whorehopper
whorehoppered
whorehopperer
whorehopperes
whorehoppering
whorehopperly
whorehoppers
whorehouse
whorehouseed
whorehouseer
whorehousees
whorehouseing
whorehousely
whorehouses
whoreing
whorely
whores
whoresed
whoreser
whoreses
whoresing
whoresly
whoress
whoring
whoringed
whoringer
whoringes
whoringing
whoringly
whorings
wigger
wiggered
wiggerer
wiggeres
wiggering
wiggerly
wiggers
woody
woodyed
woodyer
woodyes
woodying
woodyly
woodys
wop
woped
woper
wopes
woping
woply
wops
wtf
wtfed
wtfer
wtfes
wtfing
wtfly
wtfs
xxx
xxxed
xxxer
xxxes
xxxing
xxxly
xxxs
yeasty
yeastyed
yeastyer
yeastyes
yeastying
yeastyly
yeastys
yobbo
yobboed
yobboer
yobboes
yobboing
yobboly
yobbos
zoophile
zoophileed
zoophileer
zoophilees
zoophileing
zoophilely
zoophiles
anal
ass
ass lick
balls
ballsac
bisexual
bleach
causas
cheap
cost of miracles
cunt
display network stats
fart
fda and death
fda AND warn
fda AND warning
fda AND warns
feom
fuck
gfc
humira AND expensive
illegal
madvocate
masturbation
nuccitelli
overdose
porn
shit
snort
texarkana
Bipolar depression
Depression
adolescent depression
adolescent major depressive disorder
adolescent schizophrenia
adolescent with major depressive disorder
animals
autism
baby
brexpiprazole
child
child bipolar
child depression
child schizophrenia
children with bipolar disorder
children with depression
children with major depressive disorder
compulsive behaviors
cure
elderly bipolar
elderly depression
elderly major depressive disorder
elderly schizophrenia
elderly with dementia
first break
first episode
gambling
gaming
geriatric depression
geriatric major depressive disorder
geriatric schizophrenia
infant
kid
major depressive disorder
major depressive disorder in adolescents
major depressive disorder in children
parenting
pediatric
pediatric bipolar
pediatric depression
pediatric major depressive disorder
pediatric schizophrenia
pregnancy
pregnant
rexulti
skin care
teen
wine
section[contains(@class, 'nav-hidden')]
footer[@id='footer']
div[contains(@class, 'pane-node-field-article-topics')]
section[contains(@class, 'footer-nav-section-wrapper')]
section[contains(@class, 'content-row')]
div[contains(@class, 'panel-pane pane-article-read-next')]
A peer-reviewed clinical journal serving healthcare professionals working with the Department of Veterans Affairs, the Department of Defense, and the Public Health Service.
Seven metrics oncology practices can track to be successful
and see how they measure up against their peers.
“Once practices figure out what they want to measure, and obviously they want to measure things that they’re not doing so well, they can look for opportunities for improvement,” said Diana Berich Brieva, DHA, MBA, CPC-A, the CEO of Ambulatory Care Consultants, which partners with medical practices to optimize operations and increase revenue.
Benchmarking your practice against others shows you how your numbers stack up to other practices’ metrics by percentile – for instance, whether your revenue is in the 25th, 50th, or 75th percentile against similar practices.
The 2024 MIPS Value Pathways (MVP) for Advancing Cancer Care is a new CMS program with specific metric criteria. The voluntary program has a Nov. 30, 2023, deadline for practices to sign up. The purpose of the program is to help practices identify areas where they can improve. Also, oncology societies such as the American Society of Clinical Oncology (ASCO) have developed metrics for this specialty.
Still, for many practices, it’s essential to develop your own metrics according to your patient population and available resources, explained Dr. Brieva.
Here are seven popular oncology metrics that many practices track to measure success.
1. Productivity
Every practice may think about productivity differently depending on whether it focuses on new patients, revenue, business development, or a combination. You can measure physician productivity in many ways: by the number of new patients per full-time employee (FTE), work relative value units (wRVU) per FTE, which measures physician work, and established patient visits.
Some clinics measure for wRVU for chemotherapy administration and per-hospital visits as a percentage of total patients as well. “We’re a community-based oncology practice, so we don’t use RVUs, but we do use other production numbers,” said Emily Touloukian, DO, an oncologist-hematologist and president of Coastal Cancer Center with four locations around South Carolina. She is assistant professor of internal medicine at the University of South Carolina, Columbia.
“There are lots of quality programs out there that measure how well oncology practices are meeting guidelines. The one we’ve participated with since its inception is [the] Quality Oncology Practice Initiative (QOPI) through [the] American Society of Clinical Oncology (ASCO),” said Dr. Touloukian. “Basically, it’s a chart review and extraction of various indicators in accordance with quality measures.”
Pontchartrain Cancer Center, with four locations around Louisiana, tracks the number of new patients in hematology and oncology by location and provider. They also track follow-up patients. New and follow-up patient metrics are broken down by visit code.
“The E&M code tells me the level of acuity that the physician coded for,” said Kathy Oubre, MS, the CEO of Pontchartrain. Patients with complicated cases get a higher-paying code since clinics get paid differently for each code. Ms. Oubre tracks the codes by provider and says if they bill every patient with the same code, it can put your practice at risk for an audit, even when it’s the lowest billable code.
In the 2019 ASCO survey, the number of new patient visits reported by participants averaged 301 visits per FTE. Established patient visits averaged 3,334.
“When we talk about metrics and how we measure things and how successful our practice is, productivity also has to do with how satisfied the people working for you and with you are,” said Dr. Touloukian.
“If you’re not providing a supportive workplace for your physicians and employees, you’re not going to be successful,” she said. “You’ll end up with doctors coming and going every 2 years, employees quitting all the time, and a need for retraining.” Instead, if you can create a welcoming, sustainable environment where people are happy to come to work, physicians aren’t burnt out, and get to spend time away from the clinic to recharge, productivity will be more successful.
2. Revenue
When participating in their voluntary survey, practices can get a copy of revenue metric data annually from the Medical Group Management Association (MGMA). It collects the number of FTEs, gross revenue, net revenue, and collection rate and is broken down by specialties so your practice can benchmark against others. Total revenue, including oncologists’ salaries per FTE from the ASCO survey, was $7,323,900, but comparisons are difficult since practices differ in services.
Revenue metrics can consist of total revenue (cash collections), net medical excluding radiation services, drug revenue for infusion services, cash expenses including salaries, net accounts receivable, and gross accounts receivable minus contractual allowances and bad debt. Practices can differ on bad debt collection because of the emotional nature of cancer treatment. However, some use revenue cycle management companies with debt collection services; others find charity foundation funds for patients who can’t pay.
Pontchartrain also tracks when its clinic gets paid. Ms. Oubre said the best practice is that your claims receive payment within 21 days. They send claims out every 24 hours. “For example, most of that money is in drugs we’ve administered to patients we’ve likely already paid for.” Since there is a gap between paying their wholesaler for drugs and receiving reimbursement, they closely track claims and payment metrics.
Any claims that get sent back are refiled and sent out again within 24 hours. When claims hit 31 days without a response, the practice reaches out to learn the problem. “We’re proactive rather than waiting for the denial to come,” Ms. Oubre told this news organization. Dr. Brieva said for every revenue metric a practice tracks in which they’re not performing well, the practice has to find a solution. Are too many claims being denied? Do claim forms contain errors? Are most claims being paid in the 21-day window? Is the problem a user error, an issue with the clearinghouse, or an intake error on the other end? The key to successful metric use is to drill down for answers to these questions.
3. Patient satisfaction
Patient satisfaction may be one of the more straightforward metrics practices can track, though not specific to oncology. Dr. Brieva said most metric programs include patient satisfaction surveys against which you can benchmark your practice. You can also create your own emailed patient surveys. The metric can show how satisfied your patients are and how you compare against other practices.
Ms. Oubre said Pontchartrain also tracks metrics around participating in advanced care planning, survivorship care, and transitional care management. Even though most insurers require copays for these services, they’ve found patients who participate in them have an overall better experience.
“The Biden administration also has a Cancer Moon Shot initiative, which intends to reduce cancer deaths by 50% by 2047,” said Dr. Brieva. “They want to reduce deaths and improve the experience of patients. So, tracking survival rates will also be key for this program.”
4. Referrals
Oncology is typically a referral business. So, keeping track of the top referring physicians every quarter is the best way to ensure your referring clinicians are happy with your practice’s service. A best practice is that all new oncology referrals are seen within 48 hours. If your referral metric drops off, especially for top referrers, a physician from your practice should check in with the referrer.
Ms. Oubre runs reports out of the EMR and scrubs for referring providers, so she’s alerted to any issues. “It can be as simple as a front office staff who was rude on the phone,” she said. “We had an issue years ago with one of our schedulers who didn’t want to risk staying after 5:00 p.m., so she wouldn’t put anyone on the schedule after 3:00. But if I hadn’t called and identified that from the referring practitioner, I wouldn’t have known that they couldn’t get late-afternoon appointments at our clinic.”
5. No-show appointments
Practices track no-shows per week to determine which patients did not show up for their visits vs. those who rescheduled. If a patient on active treatment starts no-showing, practices must find out why. Is it a social or a transportation issue, or do they want to discontinue treatment? Often, you can help with the problem if you know what it is.
“Sometimes we can help from a social determinants of health perspective, helping to provide services like transportation, financial assistance, or other things, and patients appreciate that we would care enough to reach out to see if we can help,” said Ms. Oubre.
For recurring no-shows, practices should notify the referring provider. Letting the referring clinician know that the patient stopped coming is a professional courtesy that helps strengthen your referral relationships. You wouldn’t want the referring physician to think the patient is being treated for cancer only to find out later that they discontinued treatment.
6. Injections and infusions
By tracking the number of injections and infusions per location per week, clinics can assess how busy their chemo chairs are and how many injections they give. Benchmarks include the number of initial intravenous infusions/injections and the total number of drug administration services per patient per chair. Similar metrics in radiation oncology are helpful.
7. Pharmacy prescriptions
For practices with an in-house pharmacy, tracking how many prescriptions are written per week by each provider and whether they could fill them in-house or had to send them out to a specialty pharmacy because of an insurance issue tells you the volume of drugs your pharmacy is fulfilling. Point-of-care dispensing pharmacy revenue averaged $1,843,342 in the ASCO survey.
Dr. Brieva mentioned many other trackable metrics, such as the time to start treatment, adherence to treatment guidelines, rates of side effects and complications, patient retention rates, treatment completion rates, and coordination of care with other providers, which may be additional metrics your practice wants to track.
Dr. Touloukian said that practices must be careful how they measure some metrics because if you’re extracting data from the EMR and someone hasn’t entered it correctly, you won’t get accurate information. “I like programs like QOPI because while it’s a little labor intensive, my staff actually goes in and extracts the data from the charts and shows the proof.
“Comparing yourself to other [oncology] practices across the nation helps to ensure you’re achieving a certain level of success on some of these traditional metrics,” said Dr. Touloukian.
A version of this article first appeared on Medscape.com.
and see how they measure up against their peers.
“Once practices figure out what they want to measure, and obviously they want to measure things that they’re not doing so well, they can look for opportunities for improvement,” said Diana Berich Brieva, DHA, MBA, CPC-A, the CEO of Ambulatory Care Consultants, which partners with medical practices to optimize operations and increase revenue.
Benchmarking your practice against others shows you how your numbers stack up to other practices’ metrics by percentile – for instance, whether your revenue is in the 25th, 50th, or 75th percentile against similar practices.
The 2024 MIPS Value Pathways (MVP) for Advancing Cancer Care is a new CMS program with specific metric criteria. The voluntary program has a Nov. 30, 2023, deadline for practices to sign up. The purpose of the program is to help practices identify areas where they can improve. Also, oncology societies such as the American Society of Clinical Oncology (ASCO) have developed metrics for this specialty.
Still, for many practices, it’s essential to develop your own metrics according to your patient population and available resources, explained Dr. Brieva.
Here are seven popular oncology metrics that many practices track to measure success.
1. Productivity
Every practice may think about productivity differently depending on whether it focuses on new patients, revenue, business development, or a combination. You can measure physician productivity in many ways: by the number of new patients per full-time employee (FTE), work relative value units (wRVU) per FTE, which measures physician work, and established patient visits.
Some clinics measure for wRVU for chemotherapy administration and per-hospital visits as a percentage of total patients as well. “We’re a community-based oncology practice, so we don’t use RVUs, but we do use other production numbers,” said Emily Touloukian, DO, an oncologist-hematologist and president of Coastal Cancer Center with four locations around South Carolina. She is assistant professor of internal medicine at the University of South Carolina, Columbia.
“There are lots of quality programs out there that measure how well oncology practices are meeting guidelines. The one we’ve participated with since its inception is [the] Quality Oncology Practice Initiative (QOPI) through [the] American Society of Clinical Oncology (ASCO),” said Dr. Touloukian. “Basically, it’s a chart review and extraction of various indicators in accordance with quality measures.”
Pontchartrain Cancer Center, with four locations around Louisiana, tracks the number of new patients in hematology and oncology by location and provider. They also track follow-up patients. New and follow-up patient metrics are broken down by visit code.
“The E&M code tells me the level of acuity that the physician coded for,” said Kathy Oubre, MS, the CEO of Pontchartrain. Patients with complicated cases get a higher-paying code since clinics get paid differently for each code. Ms. Oubre tracks the codes by provider and says if they bill every patient with the same code, it can put your practice at risk for an audit, even when it’s the lowest billable code.
In the 2019 ASCO survey, the number of new patient visits reported by participants averaged 301 visits per FTE. Established patient visits averaged 3,334.
“When we talk about metrics and how we measure things and how successful our practice is, productivity also has to do with how satisfied the people working for you and with you are,” said Dr. Touloukian.
“If you’re not providing a supportive workplace for your physicians and employees, you’re not going to be successful,” she said. “You’ll end up with doctors coming and going every 2 years, employees quitting all the time, and a need for retraining.” Instead, if you can create a welcoming, sustainable environment where people are happy to come to work, physicians aren’t burnt out, and get to spend time away from the clinic to recharge, productivity will be more successful.
2. Revenue
When participating in their voluntary survey, practices can get a copy of revenue metric data annually from the Medical Group Management Association (MGMA). It collects the number of FTEs, gross revenue, net revenue, and collection rate and is broken down by specialties so your practice can benchmark against others. Total revenue, including oncologists’ salaries per FTE from the ASCO survey, was $7,323,900, but comparisons are difficult since practices differ in services.
Revenue metrics can consist of total revenue (cash collections), net medical excluding radiation services, drug revenue for infusion services, cash expenses including salaries, net accounts receivable, and gross accounts receivable minus contractual allowances and bad debt. Practices can differ on bad debt collection because of the emotional nature of cancer treatment. However, some use revenue cycle management companies with debt collection services; others find charity foundation funds for patients who can’t pay.
Pontchartrain also tracks when its clinic gets paid. Ms. Oubre said the best practice is that your claims receive payment within 21 days. They send claims out every 24 hours. “For example, most of that money is in drugs we’ve administered to patients we’ve likely already paid for.” Since there is a gap between paying their wholesaler for drugs and receiving reimbursement, they closely track claims and payment metrics.
Any claims that get sent back are refiled and sent out again within 24 hours. When claims hit 31 days without a response, the practice reaches out to learn the problem. “We’re proactive rather than waiting for the denial to come,” Ms. Oubre told this news organization. Dr. Brieva said for every revenue metric a practice tracks in which they’re not performing well, the practice has to find a solution. Are too many claims being denied? Do claim forms contain errors? Are most claims being paid in the 21-day window? Is the problem a user error, an issue with the clearinghouse, or an intake error on the other end? The key to successful metric use is to drill down for answers to these questions.
3. Patient satisfaction
Patient satisfaction may be one of the more straightforward metrics practices can track, though not specific to oncology. Dr. Brieva said most metric programs include patient satisfaction surveys against which you can benchmark your practice. You can also create your own emailed patient surveys. The metric can show how satisfied your patients are and how you compare against other practices.
Ms. Oubre said Pontchartrain also tracks metrics around participating in advanced care planning, survivorship care, and transitional care management. Even though most insurers require copays for these services, they’ve found patients who participate in them have an overall better experience.
“The Biden administration also has a Cancer Moon Shot initiative, which intends to reduce cancer deaths by 50% by 2047,” said Dr. Brieva. “They want to reduce deaths and improve the experience of patients. So, tracking survival rates will also be key for this program.”
4. Referrals
Oncology is typically a referral business. So, keeping track of the top referring physicians every quarter is the best way to ensure your referring clinicians are happy with your practice’s service. A best practice is that all new oncology referrals are seen within 48 hours. If your referral metric drops off, especially for top referrers, a physician from your practice should check in with the referrer.
Ms. Oubre runs reports out of the EMR and scrubs for referring providers, so she’s alerted to any issues. “It can be as simple as a front office staff who was rude on the phone,” she said. “We had an issue years ago with one of our schedulers who didn’t want to risk staying after 5:00 p.m., so she wouldn’t put anyone on the schedule after 3:00. But if I hadn’t called and identified that from the referring practitioner, I wouldn’t have known that they couldn’t get late-afternoon appointments at our clinic.”
5. No-show appointments
Practices track no-shows per week to determine which patients did not show up for their visits vs. those who rescheduled. If a patient on active treatment starts no-showing, practices must find out why. Is it a social or a transportation issue, or do they want to discontinue treatment? Often, you can help with the problem if you know what it is.
“Sometimes we can help from a social determinants of health perspective, helping to provide services like transportation, financial assistance, or other things, and patients appreciate that we would care enough to reach out to see if we can help,” said Ms. Oubre.
For recurring no-shows, practices should notify the referring provider. Letting the referring clinician know that the patient stopped coming is a professional courtesy that helps strengthen your referral relationships. You wouldn’t want the referring physician to think the patient is being treated for cancer only to find out later that they discontinued treatment.
6. Injections and infusions
By tracking the number of injections and infusions per location per week, clinics can assess how busy their chemo chairs are and how many injections they give. Benchmarks include the number of initial intravenous infusions/injections and the total number of drug administration services per patient per chair. Similar metrics in radiation oncology are helpful.
7. Pharmacy prescriptions
For practices with an in-house pharmacy, tracking how many prescriptions are written per week by each provider and whether they could fill them in-house or had to send them out to a specialty pharmacy because of an insurance issue tells you the volume of drugs your pharmacy is fulfilling. Point-of-care dispensing pharmacy revenue averaged $1,843,342 in the ASCO survey.
Dr. Brieva mentioned many other trackable metrics, such as the time to start treatment, adherence to treatment guidelines, rates of side effects and complications, patient retention rates, treatment completion rates, and coordination of care with other providers, which may be additional metrics your practice wants to track.
Dr. Touloukian said that practices must be careful how they measure some metrics because if you’re extracting data from the EMR and someone hasn’t entered it correctly, you won’t get accurate information. “I like programs like QOPI because while it’s a little labor intensive, my staff actually goes in and extracts the data from the charts and shows the proof.
“Comparing yourself to other [oncology] practices across the nation helps to ensure you’re achieving a certain level of success on some of these traditional metrics,” said Dr. Touloukian.
A version of this article first appeared on Medscape.com.
and see how they measure up against their peers.
“Once practices figure out what they want to measure, and obviously they want to measure things that they’re not doing so well, they can look for opportunities for improvement,” said Diana Berich Brieva, DHA, MBA, CPC-A, the CEO of Ambulatory Care Consultants, which partners with medical practices to optimize operations and increase revenue.
Benchmarking your practice against others shows you how your numbers stack up to other practices’ metrics by percentile – for instance, whether your revenue is in the 25th, 50th, or 75th percentile against similar practices.
The 2024 MIPS Value Pathways (MVP) for Advancing Cancer Care is a new CMS program with specific metric criteria. The voluntary program has a Nov. 30, 2023, deadline for practices to sign up. The purpose of the program is to help practices identify areas where they can improve. Also, oncology societies such as the American Society of Clinical Oncology (ASCO) have developed metrics for this specialty.
Still, for many practices, it’s essential to develop your own metrics according to your patient population and available resources, explained Dr. Brieva.
Here are seven popular oncology metrics that many practices track to measure success.
1. Productivity
Every practice may think about productivity differently depending on whether it focuses on new patients, revenue, business development, or a combination. You can measure physician productivity in many ways: by the number of new patients per full-time employee (FTE), work relative value units (wRVU) per FTE, which measures physician work, and established patient visits.
Some clinics measure for wRVU for chemotherapy administration and per-hospital visits as a percentage of total patients as well. “We’re a community-based oncology practice, so we don’t use RVUs, but we do use other production numbers,” said Emily Touloukian, DO, an oncologist-hematologist and president of Coastal Cancer Center with four locations around South Carolina. She is assistant professor of internal medicine at the University of South Carolina, Columbia.
“There are lots of quality programs out there that measure how well oncology practices are meeting guidelines. The one we’ve participated with since its inception is [the] Quality Oncology Practice Initiative (QOPI) through [the] American Society of Clinical Oncology (ASCO),” said Dr. Touloukian. “Basically, it’s a chart review and extraction of various indicators in accordance with quality measures.”
Pontchartrain Cancer Center, with four locations around Louisiana, tracks the number of new patients in hematology and oncology by location and provider. They also track follow-up patients. New and follow-up patient metrics are broken down by visit code.
“The E&M code tells me the level of acuity that the physician coded for,” said Kathy Oubre, MS, the CEO of Pontchartrain. Patients with complicated cases get a higher-paying code since clinics get paid differently for each code. Ms. Oubre tracks the codes by provider and says if they bill every patient with the same code, it can put your practice at risk for an audit, even when it’s the lowest billable code.
In the 2019 ASCO survey, the number of new patient visits reported by participants averaged 301 visits per FTE. Established patient visits averaged 3,334.
“When we talk about metrics and how we measure things and how successful our practice is, productivity also has to do with how satisfied the people working for you and with you are,” said Dr. Touloukian.
“If you’re not providing a supportive workplace for your physicians and employees, you’re not going to be successful,” she said. “You’ll end up with doctors coming and going every 2 years, employees quitting all the time, and a need for retraining.” Instead, if you can create a welcoming, sustainable environment where people are happy to come to work, physicians aren’t burnt out, and get to spend time away from the clinic to recharge, productivity will be more successful.
2. Revenue
When participating in their voluntary survey, practices can get a copy of revenue metric data annually from the Medical Group Management Association (MGMA). It collects the number of FTEs, gross revenue, net revenue, and collection rate and is broken down by specialties so your practice can benchmark against others. Total revenue, including oncologists’ salaries per FTE from the ASCO survey, was $7,323,900, but comparisons are difficult since practices differ in services.
Revenue metrics can consist of total revenue (cash collections), net medical excluding radiation services, drug revenue for infusion services, cash expenses including salaries, net accounts receivable, and gross accounts receivable minus contractual allowances and bad debt. Practices can differ on bad debt collection because of the emotional nature of cancer treatment. However, some use revenue cycle management companies with debt collection services; others find charity foundation funds for patients who can’t pay.
Pontchartrain also tracks when its clinic gets paid. Ms. Oubre said the best practice is that your claims receive payment within 21 days. They send claims out every 24 hours. “For example, most of that money is in drugs we’ve administered to patients we’ve likely already paid for.” Since there is a gap between paying their wholesaler for drugs and receiving reimbursement, they closely track claims and payment metrics.
Any claims that get sent back are refiled and sent out again within 24 hours. When claims hit 31 days without a response, the practice reaches out to learn the problem. “We’re proactive rather than waiting for the denial to come,” Ms. Oubre told this news organization. Dr. Brieva said for every revenue metric a practice tracks in which they’re not performing well, the practice has to find a solution. Are too many claims being denied? Do claim forms contain errors? Are most claims being paid in the 21-day window? Is the problem a user error, an issue with the clearinghouse, or an intake error on the other end? The key to successful metric use is to drill down for answers to these questions.
3. Patient satisfaction
Patient satisfaction may be one of the more straightforward metrics practices can track, though not specific to oncology. Dr. Brieva said most metric programs include patient satisfaction surveys against which you can benchmark your practice. You can also create your own emailed patient surveys. The metric can show how satisfied your patients are and how you compare against other practices.
Ms. Oubre said Pontchartrain also tracks metrics around participating in advanced care planning, survivorship care, and transitional care management. Even though most insurers require copays for these services, they’ve found patients who participate in them have an overall better experience.
“The Biden administration also has a Cancer Moon Shot initiative, which intends to reduce cancer deaths by 50% by 2047,” said Dr. Brieva. “They want to reduce deaths and improve the experience of patients. So, tracking survival rates will also be key for this program.”
4. Referrals
Oncology is typically a referral business. So, keeping track of the top referring physicians every quarter is the best way to ensure your referring clinicians are happy with your practice’s service. A best practice is that all new oncology referrals are seen within 48 hours. If your referral metric drops off, especially for top referrers, a physician from your practice should check in with the referrer.
Ms. Oubre runs reports out of the EMR and scrubs for referring providers, so she’s alerted to any issues. “It can be as simple as a front office staff who was rude on the phone,” she said. “We had an issue years ago with one of our schedulers who didn’t want to risk staying after 5:00 p.m., so she wouldn’t put anyone on the schedule after 3:00. But if I hadn’t called and identified that from the referring practitioner, I wouldn’t have known that they couldn’t get late-afternoon appointments at our clinic.”
5. No-show appointments
Practices track no-shows per week to determine which patients did not show up for their visits vs. those who rescheduled. If a patient on active treatment starts no-showing, practices must find out why. Is it a social or a transportation issue, or do they want to discontinue treatment? Often, you can help with the problem if you know what it is.
“Sometimes we can help from a social determinants of health perspective, helping to provide services like transportation, financial assistance, or other things, and patients appreciate that we would care enough to reach out to see if we can help,” said Ms. Oubre.
For recurring no-shows, practices should notify the referring provider. Letting the referring clinician know that the patient stopped coming is a professional courtesy that helps strengthen your referral relationships. You wouldn’t want the referring physician to think the patient is being treated for cancer only to find out later that they discontinued treatment.
6. Injections and infusions
By tracking the number of injections and infusions per location per week, clinics can assess how busy their chemo chairs are and how many injections they give. Benchmarks include the number of initial intravenous infusions/injections and the total number of drug administration services per patient per chair. Similar metrics in radiation oncology are helpful.
7. Pharmacy prescriptions
For practices with an in-house pharmacy, tracking how many prescriptions are written per week by each provider and whether they could fill them in-house or had to send them out to a specialty pharmacy because of an insurance issue tells you the volume of drugs your pharmacy is fulfilling. Point-of-care dispensing pharmacy revenue averaged $1,843,342 in the ASCO survey.
Dr. Brieva mentioned many other trackable metrics, such as the time to start treatment, adherence to treatment guidelines, rates of side effects and complications, patient retention rates, treatment completion rates, and coordination of care with other providers, which may be additional metrics your practice wants to track.
Dr. Touloukian said that practices must be careful how they measure some metrics because if you’re extracting data from the EMR and someone hasn’t entered it correctly, you won’t get accurate information. “I like programs like QOPI because while it’s a little labor intensive, my staff actually goes in and extracts the data from the charts and shows the proof.
“Comparing yourself to other [oncology] practices across the nation helps to ensure you’re achieving a certain level of success on some of these traditional metrics,” said Dr. Touloukian.
A version of this article first appeared on Medscape.com.
Portfolio diet tied to lower risk for CVD, stroke
TOPLINE:
Close adherence to the Portfolio dietary pattern, including foods that have been shown to actively lower cholesterol (for example, plant proteins, nuts, viscous fiber, phytosterols, and plant monounsaturated fats) is associated with a 14% lower risk for total cardiovascular disease (CVD), coronary heart disease (CHD), and stroke, pooled results from three large observational studies suggest.
METHODOLOGY:
- The study included 73,924 women from the Nurses’ Health Study (NHS), 92,346 women from the Nurses’ Health Study II (NHSII), and 43,970 men from the Health Professionals Follow-Up Study (HPFS) without CVD at baseline who were followed biennially on lifestyle, medical history, and other health-related factors.
- From food-frequency questionnaires (FFQs) completed every 4 years, researchers categorized foods into the six components of the Portfolio diet:
- Plant protein such as legumes, beans, tofu, peas, and soymilk
- Nuts and seeds
- Fiber sources such as bran, oats, berries, and eggplant
- Phytosterols
- Monounsaturated fat (MUFA) sources such as olive oil and avocado
- High saturated fat and cholesterol sources such as whole-fat dairy and red and processed meats
- They scored each from 1 (least adherent) to 5 (most adherent), with a higher score indicating higher consumption.
- Researchers examined the association of this Portfolio Diet Score (PDS) with total CVD, CHD, and stroke, in the three cohorts, and associations with plasma levels of lipid and inflammatory biomarkers in a subpopulation of the cohorts.
TAKEAWAY:
- During up to 30 years of follow-up, there were 16,917 incident CVD cases, including 10,666 CHD cases and 6,473 strokes.
- In a pooled analysis of the three cohorts, the fully adjusted hazard ratio for total CVD comparing the highest with the lowest quintile of the PDS was 0.86 (95% confidence interval, 0.81-0.92; P for trend < .001).
- Also comparing extreme quintiles, the pooled HR for CHD was 0.86 (95% CI, 0.80-0.93; P for trend = .0001) and for stroke, it was 0.86 (95% CI, 0.78-0.95; P for trend = .0003).
- A higher PDS was also associated with a more favorable lipid profile and lower levels of inflammation.
IN PRACTICE:
which aligns with American Heart Association guidelines promoting consumption of whole grains, fruits and vegetables, plant-based proteins, minimally processed foods, and healthy unsaturated plant oils, the authors conclude.
SOURCE:
The study was conducted by Andrea J. Glenn, PhD, RD, department of nutrition, Harvard T.H. Chan School of Public Health, Boston, and colleagues. It was published online in the journal Circulation.
LIMITATIONS:
As the study was observational, residual confounding can’t be ruled out. Diet was self-reported, which may have resulted in measurement errors. Consumption of some recommended foods was low, even in the top quintiles, so the association with CVD risk may be underestimated. Information on a few key Portfolio diet foods, including barley and okra, was unavailable, potentially leading to underestimation of intake, which may also attenuate the findings.
DISCLOSURES:
The study was supported by the Diabetes Canada End Diabetes 100 Award. The NH and HPFS studies are supported by the National Institutes of Health. Dr. Glenn is supported by a Canadian Institutes of Health Research fellowship; she has received honoraria or travel support from the Soy Nutrition Institute Global, Vinasoy, and the Academy of Nutrition and Dietetics. See original article for disclosures of other authors.
A version of this article first appeared on Medscape.com.
TOPLINE:
Close adherence to the Portfolio dietary pattern, including foods that have been shown to actively lower cholesterol (for example, plant proteins, nuts, viscous fiber, phytosterols, and plant monounsaturated fats) is associated with a 14% lower risk for total cardiovascular disease (CVD), coronary heart disease (CHD), and stroke, pooled results from three large observational studies suggest.
METHODOLOGY:
- The study included 73,924 women from the Nurses’ Health Study (NHS), 92,346 women from the Nurses’ Health Study II (NHSII), and 43,970 men from the Health Professionals Follow-Up Study (HPFS) without CVD at baseline who were followed biennially on lifestyle, medical history, and other health-related factors.
- From food-frequency questionnaires (FFQs) completed every 4 years, researchers categorized foods into the six components of the Portfolio diet:
- Plant protein such as legumes, beans, tofu, peas, and soymilk
- Nuts and seeds
- Fiber sources such as bran, oats, berries, and eggplant
- Phytosterols
- Monounsaturated fat (MUFA) sources such as olive oil and avocado
- High saturated fat and cholesterol sources such as whole-fat dairy and red and processed meats
- They scored each from 1 (least adherent) to 5 (most adherent), with a higher score indicating higher consumption.
- Researchers examined the association of this Portfolio Diet Score (PDS) with total CVD, CHD, and stroke, in the three cohorts, and associations with plasma levels of lipid and inflammatory biomarkers in a subpopulation of the cohorts.
TAKEAWAY:
- During up to 30 years of follow-up, there were 16,917 incident CVD cases, including 10,666 CHD cases and 6,473 strokes.
- In a pooled analysis of the three cohorts, the fully adjusted hazard ratio for total CVD comparing the highest with the lowest quintile of the PDS was 0.86 (95% confidence interval, 0.81-0.92; P for trend < .001).
- Also comparing extreme quintiles, the pooled HR for CHD was 0.86 (95% CI, 0.80-0.93; P for trend = .0001) and for stroke, it was 0.86 (95% CI, 0.78-0.95; P for trend = .0003).
- A higher PDS was also associated with a more favorable lipid profile and lower levels of inflammation.
IN PRACTICE:
which aligns with American Heart Association guidelines promoting consumption of whole grains, fruits and vegetables, plant-based proteins, minimally processed foods, and healthy unsaturated plant oils, the authors conclude.
SOURCE:
The study was conducted by Andrea J. Glenn, PhD, RD, department of nutrition, Harvard T.H. Chan School of Public Health, Boston, and colleagues. It was published online in the journal Circulation.
LIMITATIONS:
As the study was observational, residual confounding can’t be ruled out. Diet was self-reported, which may have resulted in measurement errors. Consumption of some recommended foods was low, even in the top quintiles, so the association with CVD risk may be underestimated. Information on a few key Portfolio diet foods, including barley and okra, was unavailable, potentially leading to underestimation of intake, which may also attenuate the findings.
DISCLOSURES:
The study was supported by the Diabetes Canada End Diabetes 100 Award. The NH and HPFS studies are supported by the National Institutes of Health. Dr. Glenn is supported by a Canadian Institutes of Health Research fellowship; she has received honoraria or travel support from the Soy Nutrition Institute Global, Vinasoy, and the Academy of Nutrition and Dietetics. See original article for disclosures of other authors.
A version of this article first appeared on Medscape.com.
TOPLINE:
Close adherence to the Portfolio dietary pattern, including foods that have been shown to actively lower cholesterol (for example, plant proteins, nuts, viscous fiber, phytosterols, and plant monounsaturated fats) is associated with a 14% lower risk for total cardiovascular disease (CVD), coronary heart disease (CHD), and stroke, pooled results from three large observational studies suggest.
METHODOLOGY:
- The study included 73,924 women from the Nurses’ Health Study (NHS), 92,346 women from the Nurses’ Health Study II (NHSII), and 43,970 men from the Health Professionals Follow-Up Study (HPFS) without CVD at baseline who were followed biennially on lifestyle, medical history, and other health-related factors.
- From food-frequency questionnaires (FFQs) completed every 4 years, researchers categorized foods into the six components of the Portfolio diet:
- Plant protein such as legumes, beans, tofu, peas, and soymilk
- Nuts and seeds
- Fiber sources such as bran, oats, berries, and eggplant
- Phytosterols
- Monounsaturated fat (MUFA) sources such as olive oil and avocado
- High saturated fat and cholesterol sources such as whole-fat dairy and red and processed meats
- They scored each from 1 (least adherent) to 5 (most adherent), with a higher score indicating higher consumption.
- Researchers examined the association of this Portfolio Diet Score (PDS) with total CVD, CHD, and stroke, in the three cohorts, and associations with plasma levels of lipid and inflammatory biomarkers in a subpopulation of the cohorts.
TAKEAWAY:
- During up to 30 years of follow-up, there were 16,917 incident CVD cases, including 10,666 CHD cases and 6,473 strokes.
- In a pooled analysis of the three cohorts, the fully adjusted hazard ratio for total CVD comparing the highest with the lowest quintile of the PDS was 0.86 (95% confidence interval, 0.81-0.92; P for trend < .001).
- Also comparing extreme quintiles, the pooled HR for CHD was 0.86 (95% CI, 0.80-0.93; P for trend = .0001) and for stroke, it was 0.86 (95% CI, 0.78-0.95; P for trend = .0003).
- A higher PDS was also associated with a more favorable lipid profile and lower levels of inflammation.
IN PRACTICE:
which aligns with American Heart Association guidelines promoting consumption of whole grains, fruits and vegetables, plant-based proteins, minimally processed foods, and healthy unsaturated plant oils, the authors conclude.
SOURCE:
The study was conducted by Andrea J. Glenn, PhD, RD, department of nutrition, Harvard T.H. Chan School of Public Health, Boston, and colleagues. It was published online in the journal Circulation.
LIMITATIONS:
As the study was observational, residual confounding can’t be ruled out. Diet was self-reported, which may have resulted in measurement errors. Consumption of some recommended foods was low, even in the top quintiles, so the association with CVD risk may be underestimated. Information on a few key Portfolio diet foods, including barley and okra, was unavailable, potentially leading to underestimation of intake, which may also attenuate the findings.
DISCLOSURES:
The study was supported by the Diabetes Canada End Diabetes 100 Award. The NH and HPFS studies are supported by the National Institutes of Health. Dr. Glenn is supported by a Canadian Institutes of Health Research fellowship; she has received honoraria or travel support from the Soy Nutrition Institute Global, Vinasoy, and the Academy of Nutrition and Dietetics. See original article for disclosures of other authors.
A version of this article first appeared on Medscape.com.
Experts question finding that 70% cancer deaths are preventable
A new global analysis highlights the substantial burden of premature deaths from cancer around the world – a burden that could potentially be averted through prevention, early detection, and timely treatment.
According to the analysis, in 2020, over half of all cancer deaths – 5.28 million of 9.96 million – occurred prematurely (before age 70), leading to a loss of roughly 183 million life-years from the disease worldwide.
More than two-thirds of premature cancer-related deaths – 3.6 million, or 68% – were potentially preventable through lifestyle changes or early detection efforts, such as cancer screening, dietary changes, or smoking cessation, and about one-third – 1.65 million, or 31% – may have been treatable.
But two biostatisticians not involved in the study who took a deep dive into it urged caution in interpreting the data.
Nilanjan Chatterjee, PhD, Bloomberg Distinguished Professor, Bloomberg School of Public Health at Johns Hopkins University, Baltimore, said the study does a “great job in bringing a lot of diverse data together to show there is very high potential for preventing premature deaths due to cancer worldwide.”
However, for a variety of reasons, Dr. Chatterjee explained, one should not “overinterpret” the high percentage of potentially preventable cancer deaths.
Gideon Meyerowitz-Katz, PhD, an epidemiologist at the University of Wollongong, in Australia, agreed.
“It’s likely many cancer deaths are, in theory, preventable, but the numbers around just how many are necessarily vague,” said Dr. Meyerowitz-Katz. “Also, ‘in theory preventable’ doesn’t necessarily mean that we can actually do it in practice.”
Invest in cancer prevention
The study, led by researchers from the World Health Organization’s International Agency for Research on Cancer and partners, provides estimates of premature deaths from 36 cancers across 185 countries.
The findings, published in The Lancet Global Health along with a new Lancet Commission report – “Women, Power, and Cancer” also highlighted the “underrecognized” cancer burden among women around the world.
Cancer ranks in the top three causes of premature mortality among women in almost all countries worldwide, but it is often “deprioritized,” the Lancet Commission report explained.
Of the nearly 5.3 million premature cancer deaths in 2020, 2.9 million occurred in men, and 2.3 million occurred in women, the investigators found. Of the premature deaths among women, 1.5 million could have potentially been avoided through prevention or detection efforts, while the remaining 800,000 might have been averted “if all women everywhere could access optimal cancer care,” the authors said.
Lung cancer was the leading contributor to preventable premature years of life lost in countries that have medium to very high scores on the Human Development Index (HDI), whereas cervical cancer was the leading contributor in low-HDI countries. HDI rankings are based on life expectancy, education, and gross national income.
Among women, as many as 72% of cancer death were premature in low-HDI countries, vs 36% in very high-HDI countries.
Overall, across all four tiers of HDI, colorectal and breast cancers represented the major treatable cancers.
Reducing exposure to four main risk factors – tobacco smoking, alcohol consumption, high body weight, and infections – would go a long way toward reducing potentially preventable premature cancer-related deaths, the authors said.
“Globally, there are marked inequalities between countries in reaching the target of reducing premature mortality from noncommunicable diseases, including cancer,” author Isabelle Soerjomataram, MD, PhD, deputy head of cancer surveillance at the International Agency for Research on Cancer, said in a press release.
“Greater investments in cancer prevention programs can reduce the prevalence of key risk factors for cancer, and increased coverage of vaccination alongside early diagnosis and screening linked to timely treatment can and must address the current cancer inequalities that are seen worldwide,” she added.
Caveats and cautionary notes
The authors acknowledge that the study has limitations related to its methodology and underlying assumptions. For instance, some premature cancer deaths that were classified as preventable may have been averted through curative therapy as well.
The findings also represent a snapshot of premature mortality in 2020 but do not necessarily predict progress in cancer control over time.
In Dr. Chatterjee’s view, this is “an excellent descriptive study that gives a good overall picture about the potential for saving a very large fraction of premature death due to cancer by implementing what is now known about primary and secondary interventions, and treatments.”
However, estimates for the effects of various risk factors and interventions are often derived from observational nonrandomized studies, which can have various types of biases, he said.
“Additionally, availability of data, observational or randomized, are often limited from many countries in Africa, Latin America, and Asia, where the cancer burdens are increasing,” Dr. Chatterjee told this news organization. “Therefore, extrapolating evidence generated mostly from North America and European countries to other understudied settings could be problematic due to difference in background in genetics, environment, socioeconomic, and cultural differences.”
Dr. Meyerowitz-Katz said the issue with this “very complex” article is that it includes “models built upon models, all of which include layers of assumptions that aren’t always obvious and may be wrong.”
On top of that, he said, “there are questions over whether the modifiable risk factors are really modifiable. Can we really get rid of 100% of ‘lack of physical exercise’? What would that even look like?”
Overall, Dr. Meyerowitz-Katz noted, “Yes, some proportion of these cancers could be prevented, and that percentage may be large, but the exact 70% estimate is very uncertain in my opinion.”
The study had no commercial funding. The authors, Dr. Chatterjee, and Dr. Meyerowitz-Katz have disclosed no relevant financial relationships.
A version of this article first appeared on Medscape.com.
A new global analysis highlights the substantial burden of premature deaths from cancer around the world – a burden that could potentially be averted through prevention, early detection, and timely treatment.
According to the analysis, in 2020, over half of all cancer deaths – 5.28 million of 9.96 million – occurred prematurely (before age 70), leading to a loss of roughly 183 million life-years from the disease worldwide.
More than two-thirds of premature cancer-related deaths – 3.6 million, or 68% – were potentially preventable through lifestyle changes or early detection efforts, such as cancer screening, dietary changes, or smoking cessation, and about one-third – 1.65 million, or 31% – may have been treatable.
But two biostatisticians not involved in the study who took a deep dive into it urged caution in interpreting the data.
Nilanjan Chatterjee, PhD, Bloomberg Distinguished Professor, Bloomberg School of Public Health at Johns Hopkins University, Baltimore, said the study does a “great job in bringing a lot of diverse data together to show there is very high potential for preventing premature deaths due to cancer worldwide.”
However, for a variety of reasons, Dr. Chatterjee explained, one should not “overinterpret” the high percentage of potentially preventable cancer deaths.
Gideon Meyerowitz-Katz, PhD, an epidemiologist at the University of Wollongong, in Australia, agreed.
“It’s likely many cancer deaths are, in theory, preventable, but the numbers around just how many are necessarily vague,” said Dr. Meyerowitz-Katz. “Also, ‘in theory preventable’ doesn’t necessarily mean that we can actually do it in practice.”
Invest in cancer prevention
The study, led by researchers from the World Health Organization’s International Agency for Research on Cancer and partners, provides estimates of premature deaths from 36 cancers across 185 countries.
The findings, published in The Lancet Global Health along with a new Lancet Commission report – “Women, Power, and Cancer” also highlighted the “underrecognized” cancer burden among women around the world.
Cancer ranks in the top three causes of premature mortality among women in almost all countries worldwide, but it is often “deprioritized,” the Lancet Commission report explained.
Of the nearly 5.3 million premature cancer deaths in 2020, 2.9 million occurred in men, and 2.3 million occurred in women, the investigators found. Of the premature deaths among women, 1.5 million could have potentially been avoided through prevention or detection efforts, while the remaining 800,000 might have been averted “if all women everywhere could access optimal cancer care,” the authors said.
Lung cancer was the leading contributor to preventable premature years of life lost in countries that have medium to very high scores on the Human Development Index (HDI), whereas cervical cancer was the leading contributor in low-HDI countries. HDI rankings are based on life expectancy, education, and gross national income.
Among women, as many as 72% of cancer death were premature in low-HDI countries, vs 36% in very high-HDI countries.
Overall, across all four tiers of HDI, colorectal and breast cancers represented the major treatable cancers.
Reducing exposure to four main risk factors – tobacco smoking, alcohol consumption, high body weight, and infections – would go a long way toward reducing potentially preventable premature cancer-related deaths, the authors said.
“Globally, there are marked inequalities between countries in reaching the target of reducing premature mortality from noncommunicable diseases, including cancer,” author Isabelle Soerjomataram, MD, PhD, deputy head of cancer surveillance at the International Agency for Research on Cancer, said in a press release.
“Greater investments in cancer prevention programs can reduce the prevalence of key risk factors for cancer, and increased coverage of vaccination alongside early diagnosis and screening linked to timely treatment can and must address the current cancer inequalities that are seen worldwide,” she added.
Caveats and cautionary notes
The authors acknowledge that the study has limitations related to its methodology and underlying assumptions. For instance, some premature cancer deaths that were classified as preventable may have been averted through curative therapy as well.
The findings also represent a snapshot of premature mortality in 2020 but do not necessarily predict progress in cancer control over time.
In Dr. Chatterjee’s view, this is “an excellent descriptive study that gives a good overall picture about the potential for saving a very large fraction of premature death due to cancer by implementing what is now known about primary and secondary interventions, and treatments.”
However, estimates for the effects of various risk factors and interventions are often derived from observational nonrandomized studies, which can have various types of biases, he said.
“Additionally, availability of data, observational or randomized, are often limited from many countries in Africa, Latin America, and Asia, where the cancer burdens are increasing,” Dr. Chatterjee told this news organization. “Therefore, extrapolating evidence generated mostly from North America and European countries to other understudied settings could be problematic due to difference in background in genetics, environment, socioeconomic, and cultural differences.”
Dr. Meyerowitz-Katz said the issue with this “very complex” article is that it includes “models built upon models, all of which include layers of assumptions that aren’t always obvious and may be wrong.”
On top of that, he said, “there are questions over whether the modifiable risk factors are really modifiable. Can we really get rid of 100% of ‘lack of physical exercise’? What would that even look like?”
Overall, Dr. Meyerowitz-Katz noted, “Yes, some proportion of these cancers could be prevented, and that percentage may be large, but the exact 70% estimate is very uncertain in my opinion.”
The study had no commercial funding. The authors, Dr. Chatterjee, and Dr. Meyerowitz-Katz have disclosed no relevant financial relationships.
A version of this article first appeared on Medscape.com.
A new global analysis highlights the substantial burden of premature deaths from cancer around the world – a burden that could potentially be averted through prevention, early detection, and timely treatment.
According to the analysis, in 2020, over half of all cancer deaths – 5.28 million of 9.96 million – occurred prematurely (before age 70), leading to a loss of roughly 183 million life-years from the disease worldwide.
More than two-thirds of premature cancer-related deaths – 3.6 million, or 68% – were potentially preventable through lifestyle changes or early detection efforts, such as cancer screening, dietary changes, or smoking cessation, and about one-third – 1.65 million, or 31% – may have been treatable.
But two biostatisticians not involved in the study who took a deep dive into it urged caution in interpreting the data.
Nilanjan Chatterjee, PhD, Bloomberg Distinguished Professor, Bloomberg School of Public Health at Johns Hopkins University, Baltimore, said the study does a “great job in bringing a lot of diverse data together to show there is very high potential for preventing premature deaths due to cancer worldwide.”
However, for a variety of reasons, Dr. Chatterjee explained, one should not “overinterpret” the high percentage of potentially preventable cancer deaths.
Gideon Meyerowitz-Katz, PhD, an epidemiologist at the University of Wollongong, in Australia, agreed.
“It’s likely many cancer deaths are, in theory, preventable, but the numbers around just how many are necessarily vague,” said Dr. Meyerowitz-Katz. “Also, ‘in theory preventable’ doesn’t necessarily mean that we can actually do it in practice.”
Invest in cancer prevention
The study, led by researchers from the World Health Organization’s International Agency for Research on Cancer and partners, provides estimates of premature deaths from 36 cancers across 185 countries.
The findings, published in The Lancet Global Health along with a new Lancet Commission report – “Women, Power, and Cancer” also highlighted the “underrecognized” cancer burden among women around the world.
Cancer ranks in the top three causes of premature mortality among women in almost all countries worldwide, but it is often “deprioritized,” the Lancet Commission report explained.
Of the nearly 5.3 million premature cancer deaths in 2020, 2.9 million occurred in men, and 2.3 million occurred in women, the investigators found. Of the premature deaths among women, 1.5 million could have potentially been avoided through prevention or detection efforts, while the remaining 800,000 might have been averted “if all women everywhere could access optimal cancer care,” the authors said.
Lung cancer was the leading contributor to preventable premature years of life lost in countries that have medium to very high scores on the Human Development Index (HDI), whereas cervical cancer was the leading contributor in low-HDI countries. HDI rankings are based on life expectancy, education, and gross national income.
Among women, as many as 72% of cancer death were premature in low-HDI countries, vs 36% in very high-HDI countries.
Overall, across all four tiers of HDI, colorectal and breast cancers represented the major treatable cancers.
Reducing exposure to four main risk factors – tobacco smoking, alcohol consumption, high body weight, and infections – would go a long way toward reducing potentially preventable premature cancer-related deaths, the authors said.
“Globally, there are marked inequalities between countries in reaching the target of reducing premature mortality from noncommunicable diseases, including cancer,” author Isabelle Soerjomataram, MD, PhD, deputy head of cancer surveillance at the International Agency for Research on Cancer, said in a press release.
“Greater investments in cancer prevention programs can reduce the prevalence of key risk factors for cancer, and increased coverage of vaccination alongside early diagnosis and screening linked to timely treatment can and must address the current cancer inequalities that are seen worldwide,” she added.
Caveats and cautionary notes
The authors acknowledge that the study has limitations related to its methodology and underlying assumptions. For instance, some premature cancer deaths that were classified as preventable may have been averted through curative therapy as well.
The findings also represent a snapshot of premature mortality in 2020 but do not necessarily predict progress in cancer control over time.
In Dr. Chatterjee’s view, this is “an excellent descriptive study that gives a good overall picture about the potential for saving a very large fraction of premature death due to cancer by implementing what is now known about primary and secondary interventions, and treatments.”
However, estimates for the effects of various risk factors and interventions are often derived from observational nonrandomized studies, which can have various types of biases, he said.
“Additionally, availability of data, observational or randomized, are often limited from many countries in Africa, Latin America, and Asia, where the cancer burdens are increasing,” Dr. Chatterjee told this news organization. “Therefore, extrapolating evidence generated mostly from North America and European countries to other understudied settings could be problematic due to difference in background in genetics, environment, socioeconomic, and cultural differences.”
Dr. Meyerowitz-Katz said the issue with this “very complex” article is that it includes “models built upon models, all of which include layers of assumptions that aren’t always obvious and may be wrong.”
On top of that, he said, “there are questions over whether the modifiable risk factors are really modifiable. Can we really get rid of 100% of ‘lack of physical exercise’? What would that even look like?”
Overall, Dr. Meyerowitz-Katz noted, “Yes, some proportion of these cancers could be prevented, and that percentage may be large, but the exact 70% estimate is very uncertain in my opinion.”
The study had no commercial funding. The authors, Dr. Chatterjee, and Dr. Meyerowitz-Katz have disclosed no relevant financial relationships.
A version of this article first appeared on Medscape.com.
FROM THE LANCET GLOBAL HEALTH
How VA Innovative Partnerships and Health Care Systems Can Respond to National Needs: NOSE Trial Example
Traditional manufacturing concentrates capacity into a few discrete locations while applying lean and just-in-time philosophies to maximize profit during times of somewhat predictable supply and demand. This approach exposed nationwide vulnerabilities even during local crises, such as the United States saline shortages following closure of a single plant in Puerto Rico following Hurricane Maria in 2017.1 Interruptions to the supply chain due to pandemic plant closure, weather, politics, or surge demand can cause immediate and lasting shortages. Nasal swabs were a clear example.
At the onset of COVID-19, 2 companies—Puritan in Guilford, Maine, and Copan in Italy—manufactured nearly all of the highly specialized nasopharyngeal (NP) swabs singled out by the Centers for Disease Control and Prevention (CDC) and the US Food and Drug Administration (FDA) to test patients for COVID-19. Demand for swabs skyrocketed as the virus spread, and they became unattainable. The lack of swabs meant patients went undiagnosed. Without knowing who was positive, people with symptoms and known contacts were presumed positive and quarantined, impacting isolated patients, the health care professionals treating them, and the entire US economy.
3-Dimensional Printing Solutions
Manufacturing NP swabs is not trivial. Their simple shape conceals complexity and requires highly specialized equipment. The lead time for one non-US machine manufacturer was > 6 months at the start of the pandemic.
Digital manufacturing/3-dimensional (3D) printing represented a potential solution to the supply chain crisis.2 Designers created digital blueprints for 3D-printed goods, face masks, face shields, and ventilator splitters were rapidly created and shared.3,4 Scrambling to fill the critical need for NP swabs, many hospitals, businesses, and academic centers began 3D printing swabs. This effort was spearheaded by University of South Florida (USF) and Northwell Health researchers and clinicians, who designed and tested a 3D-printed NP swab from photocurable resin that was printable on 2 models of Formlabs printers.5 Several other 3D-printed NP swab designs soon followed. This innovation and problem-solving renaissance faced several challenges well known to traditional manufacturers of regulated products but novel to newcomers.
The first challlenge was that these NP swabs predate FDA oversight of medical device development and manufacturing and no testing standards existed. Designers began casting prototypes out without guidance about the critical features and clinical functions required. Many of these designs did not have a clinical evaluation pathway to test safety and efficacy.
The second challlenge was that these swabs were being produced by facilities not registered with the FDA. This raised concerns about the quality of unlisted medical products developed and manufactured at novel facilities.
The third challenge was that small-scale novel approaches may offset local shortages but could not address national needs. The self-organized infrastructure for this crisis was ad hoc, local, and lacked coordinated federal support. This led to rolling shortages of these materials for years.
Two studies were performed early in the pandemic. The first study evaluated 4 prototypes of different manufacturer designs, finding excellent concordance among them and their control swab.6 A second study demonstrated the USF swab to be noninferior to the standard of care.7 Both studies acknowledged and addressed the first challenge for their designs.
COLLABORATIONS
Interagency
Before the pandemic, the US Department of Veterans Affairs (VA) had been coordinating with the FDA, the National Institutes of Health (NIH), and the nonprofit America Makes to bring medical product development and manufacturing closer to the point of care.
At the outset of the COVID-19 pandemic, the collaboration was formalized to address new challenges.8 The objectives of this collaboration were the following: (1) host a digital repository for 3D-printed digital designs for personal protectice equipment and other medical supplies in or at risk of shortage; (2) provide scientifically based ratings for designs according to clinical and field testing; and (3) offer education to health care workers and the public about the digital manufacturing of medical goods and devices.4,9
A key output of this collaboration was the COVID 3D Trusted Repository For Users And Suppliers Through Testing (COVID 3D TRUST), a curated archive of designs. In most cases, existing FDA standards and guidance formed the basis of testing strategies with deviations due to limited access to traditional testing facilities and reagents.
To address novel NP swabs, working with its COVID 3D TRUST partners, the VA gathered a combined list of clinical- and engineering-informed customer requirements and performed a hazard analysis. The result was a list of design inputs for NP swabs and 8 standard test protocols to evaluate key functions (Table).10 These protocols are meant to benchmark novel 3D-printed swabs against the key functions of established, traditionally manufactured swabs, which have a long record of safety and efficacy. The protocols, developed by the VA and undergoing validation by the US Army, empower and inform consumers and provide performance metrics to swab designers and manufacturers. The testing protocols and preliminary test results developed by the VA are publicly available at the NIH.11
Intra-agency
The use of the inputs and verification tests noted in the Table may reduce the risk of poor design but were inadequate to evaluate the clinical safety and efficacy of novel swabs. Recognizing this, the VA Office of Healthcare Innovation and Learning (OHIL) and the Office of Research and Development (ORD) launched the Nasal Swab Objective and Statistical Evaluation (NOSE) study to formally evaluate the safety and efficacy of 3D-printed swabs in the field. This multisite clinical study was a close collaboration between the OHIL and ORD. The OHIL provided the quality system and manufacturing oversight and delivery of the swabs, and the ORD provided scientific review, research infrastructure, human subjects oversight, administrative support, and funding and fiscal oversight. The OHIL/ORD collaboration resulted in the successful completion of the NOSE study.
This study (manuscript under preparation) yielded two 3D-printing production processes and swab designs that had comparable performance to the standard of care, were manufacturable compliant with FDA guidelines, and could be produced at scale in a distributed manner. This approach directly addressed the 3 challenges described earlier.
LESSONS LEARNED
Swabs were an example of supply challenges in the pandemic, but advanced manufacturing (notably, digital designs leading to 3D-printed solutions) also served as a temporary solution to device and product shortages during the COVID-19 pandemic. Digital designs and 3D printing as manufacturing techniques have the following key advantages: (1) they are distributed in nature, both in the breadth of locations that have access to these manufacturing platforms and in the depth of material choice that can be used to fabricate products, which alleviates the threat of a disaster impacting manufacturing capacity or a material stream; (2) they do not require retooling of machinery so new products can deploy rapidly and on demand; and (3) the speed of digital iteration, printing, and revision allows for rapid product development and production.
There also are notable disadvantages to these techniques. First, because 3D printing is a newer technology, there is less general depth of knowledge regarding design and material choice for additive manufacturing. Second, the flexibility of 3D printing means that operators must increase awareness of the factors that might cause the fabrication of a part to fail in either printing or postprocessing. Third, there are significant gaps in understanding how materials and manufacturing processes will perform in high-stakes settings such as health care, where performance and biocompatibility may be critical to support life-sustaining functions. Fourth, digital files are vulnerable to intentional or unintentional alteration. These alterations might weaken design integrity and be imperceptible to the manufacturer or end user. This is a prevalent challenge in all open-source designs.
The pandemic materialized quickly and created vast supply chain challenges. To address this crisis, it was clear that the average 17-year interval between research and translation in the US was unacceptable. The VA was able to accelerate swiftly many existing processes to meet this need, build new capabilities, and establish new practices for the rapid evaluation and deployment of health care products and guidance. This agile and innovative cooperation was critical in the success of the VA’s national support for pandemic solutions.
Finally, although COVID 3D TRUST was able to provide testing of submitted designs, this collaboration was not a substitute for the “peacetime” process of manufacturing site registration with the FDA and product listing. COVID 3D TRUST could evaluate designs only, not the production process, safety, and efficacy.
CALLS TO ACTION
The pandemic's impact on medical supply chain security persists, as does the need for greater foresight and crisis preparation. We must act now to avoid experiencing again the magnitude of fatalities (civilian and veteran) and the devastation to the US economy and livelihoods that occurred during this single biological event. To this end, creating a digital stockpile of federally curated, crisis-ready designs for as-needed distribution across our US industrial base would offer a second line of defense against life-threatening supply chain interruptions. The realization of such a digital stockpile requires calls to action among multiple contributors.
Collaborations
The VA’s Fourth Mission is to improve the nation’s preparedness for response to war, terrorism, national emergencies, and natural disasters. The VA does this by developing plans and taking actions to ensure continued service to veterans, as well as to support national, state, and local emergency management, public health, safety, and homeland security efforts.
The VA partnership with the FDA and NIH during the pandemic enabled successful coordination among federal agencies. Numerous other agencies, including the US Department of Defense (DoD), the Biomedical Advanced Research and Development Authority (BARDA), and the Defense Advanced Research Projects Agency (DARPA), also developed and executed successful initiatives.12-14 The joint awareness and management of these efforts, however, could be strengthened through more formal agreements and processes in peacetime. The VA/FDA/NIH Memorandum of Understanding is a prototype example of each agency lending its subject matter expertise to address a host of pandemic challenges collectively, cooperatively, and efficiently.8
Public-private partnerships (eg, VA/FDA/NIH and America Makes) led to coordinated responses for crisis readiness. The Advanced Manufacturing Crisis Product Response Program, a multipartner collaboration that included VA, addressed 7 crisis scenarios, 3 of which were specifically related to COVID-19.15 In addition, both BARDA and DARPA had successful public-private collaborations, and the DoD supported national logistics and other efforts.12-14 Clearly, industry and government both recognize complementary synergies: (1) the depth of resources of US industry; and (2) the national resources, coordination, and clinical insight available through federal agencies that can address the challenges of future crises quickly and efficiently.
When traditional supply chains and manufacturing processes failed during the pandemic, new techniques were exploited to fill the unmet material needs. Novel techniques and product pathways, however, are untested or undeveloped. The collaboration between the ORD and OHIL in support of NP swab testing and production is an example of bringing research insight, regulated product development, and manufacturing together to support a complete product life cycle.
Joint Awareness and Management
The VA continues to refine the joint awareness and management (JAM) process of products from ideation to translation, to shorten the time from research to product delivery. JAM is a VA collaborative committee of partners from ORD research offices and technology transfer program, and the OHIL Office of Advanced Manufacturing, which seeks additional support and guidance from VHA clinical service lines, VA Office of General Council, and VA Office of Acquisitions, Logistics, and Construction.
This team enables the rapid identification of unmet veteran health care product needs. In addition, JAM leverages the resources of each group to support products from problem identification to solution ideation, regulated development, production, and delivery into clinical service lines. While the concept of JAM arose to meet the crisis needs of the pandemic, it persists in delivering advanced health care solutions to veterans.
A Proposed Plan
The next national crisis is likely to involve and threaten national health care security. We propose that federal agencies be brought together to form a federally supported digital stockpile. This digital stockpile must encompass, at minimum, the following features: (1) preservation of novel, scalable medical supplies and products generated during the COVID-19 pandemic, to avoid the loss of this work; (2) clinical maturation of those existing supplies and products to refine their features and functions under the guidance of clinical, regulatory, and manufacturing experts—and validate those outputs with clinical evidence; (3) manufacturing maturation of those existing supplies and products, such that complete design and production processes are developed with the intent to distribute to multiple public manufacturers during the next crisis; (4) a call for new designs/intake portal for new designs to be matured and curated as vulnerabilities are identified; (5) supply chain crisis drills executed to test public-private preparedness to ensure design transfer is turnkey and can be engaged quickly during the next crisis; and (6) public-private engagement to develop strategy, scenarios, and policy to ensure that when supply chains next fail, additional surge capacity can be quickly added to protect American lives and health care, and that when supply chains resume, surge capacity can be redirected or stood down to protect the competitive markets.
This digital stockpile can complement and be part of the Strategic National Stockpile. Whereas the Strategic National Stockpile is a reserve of physical products that may offset product shortages, the digital stockpile is a reserve of turnkey, transferable designs that may offset supply chain disruptions and production-capacity shortages.
CONCLUSIONS
The success of 3D-printed NP swabs is a specific example of the importance of collaborations across industry, government, innovators, and researchers. More important than a sole product, however, these collaborations demonstrated the potential for game-changing approaches to how public-private partnerships support the continuity of health care operations nationally and prevent the potential for unnecessary loss of life due to capacity and supply chain disruptions.
As the largest health care system in the US, the VA has a unique capability to lead in the assessment of other novel 3D-printed medical devices in partnership with the FDA. The VA has a unique patient-centered perspective on medical device efficacy, and as a government institution, it is a trusted independent source for medical device evaluation. The VA’s role in the evaluation of 3D-printed medical devices will benefit veterans and their families, clinicians, hospitals, and the broader public by providing a gold-standard evaluation for the growing medical 3D-printing industry to follow. By creating new pathways and expectations for how federal agencies maintain crisis preparedness—such as establishing a digital stockpile—we can be equipped to serve the US health care system and minimize the effects of supply chain disruptions.
1. Sacks CA, Kesselheim AS, Fralick M. The shortage of normal saline in the wake of Hurricane Maria. JAMA Intern Med. 2018;178(7):885–886. doi:10.1001/jamainternmed.2018.1936
2. Bauchner H, Fontanarosa PB, Livingston EH. Conserving supply of personal protective equipment–a call for ideas. JAMA. 2020;323(19):1911. doi:10.1001/jama.2020.4770
3. Sinha MS, Bourgeois FT, Sorger PK. Personal protective equipment for COVID-19: distributed fabrication and additive manufacturing. Am J Public Health. 2020;110(8):1162-1164. doi:10.2105/AJPH.2020.305753
4. McCarthy MC, Di Prima M, Cruz P, et al. Trust in the time of Covid-19: 3D printing and additive manufacturing (3DP/AM) as a solution to supply chain gaps. NEJM Catalyst. 2021;2(6). doi:10.1056/CAT.21.0321
5. Ford J, Goldstein T, Trahan S, Neuwirth A, Tatoris K, Decker S. A 3D-printed nasopharyngeal swab for COVID-19 diagnostic testing. 3D Print Med. 2020;6(1):21. Published 2020 Aug 15. doi:10.1186/s41205-020-00076-3
6. Callahan CJ, Lee R, Zulauf K, et al. Open development and clinical validation of multiple 3D-printed sample-collection swabs: rapid resolution of a critical COVID-19 testing bottleneck. Preprint. medRxiv. 2020;2020.04.14.20065094. Published 2020 Apr 17. doi:10.1101/2020.04.14.20065094
7. Decker SJ, Goldstein TA, Ford JM, et al. 3-dimensional printed alternative to the standard synthetic flocked nasopharyngeal swabs used for coronavirus disease 2019 testing. Clin Infect Dis. 2021;73(9):e3027-e3032. doi:10.1093/cid/ciaa1366
8. US Food and Drug Administration. Memorandum of understanding: rapid response to Covid-19 using 3d printing between National Institutes of Health within U.S. Department of Health and Human Services and Food and Drug Administration, U.S. Department of Health and Human Services and Veterans Health Administration within the U.S. Department of Veterans Affairs. March 26, 2020. Accessed August 31, 2023. https://www.fda.gov/about-fda/domestic-mous/mou-225-20-008
9. National Institutes of Health, NIH 3D Print Exchange. Covid 3D trust: trusted repository for users and suppliers through testing. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=search
10. National Institutes of Health, NIH 3D Print Exchange. 3D printed nasal swabs - assessment criteria. August 17, 2020. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=swabassessment
11. National Institutes of Health, NIH 3D Print Exchange. 3D printed nasal swabs - general information. August 17, 2020. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=swabinfo
12. US Department of Defense. Coronavirus: DOD response. December 20, 2022. Accessed August 31, 2023. https://www.defense.gov/Spotlights/Coronavirus-DoD-Response
13. US Department of Health and Human Services, Biomedical Advanced Research and Development Authority. BARDA COVID-19 response. Updated May 25, 2023. Accessed August 31, 2023. https://www.medicalcountermeasures.gov/barda/barda-covid-19-response
14. Green S. Pandemic prevention platform (P3). Accessed August 31, 2023. https://www.darpa.mil/program/pandemic-prevention-platform
15. America Makes. America makes completes successful scenario testing for crisis response program [press release]. May 25, 2021. Accessed August 31, 2023. https://www.americamakes.us/america-makes-completes-successful-scenario-testing-for-crisis-response-program
Traditional manufacturing concentrates capacity into a few discrete locations while applying lean and just-in-time philosophies to maximize profit during times of somewhat predictable supply and demand. This approach exposed nationwide vulnerabilities even during local crises, such as the United States saline shortages following closure of a single plant in Puerto Rico following Hurricane Maria in 2017.1 Interruptions to the supply chain due to pandemic plant closure, weather, politics, or surge demand can cause immediate and lasting shortages. Nasal swabs were a clear example.
At the onset of COVID-19, 2 companies—Puritan in Guilford, Maine, and Copan in Italy—manufactured nearly all of the highly specialized nasopharyngeal (NP) swabs singled out by the Centers for Disease Control and Prevention (CDC) and the US Food and Drug Administration (FDA) to test patients for COVID-19. Demand for swabs skyrocketed as the virus spread, and they became unattainable. The lack of swabs meant patients went undiagnosed. Without knowing who was positive, people with symptoms and known contacts were presumed positive and quarantined, impacting isolated patients, the health care professionals treating them, and the entire US economy.
3-Dimensional Printing Solutions
Manufacturing NP swabs is not trivial. Their simple shape conceals complexity and requires highly specialized equipment. The lead time for one non-US machine manufacturer was > 6 months at the start of the pandemic.
Digital manufacturing/3-dimensional (3D) printing represented a potential solution to the supply chain crisis.2 Designers created digital blueprints for 3D-printed goods, face masks, face shields, and ventilator splitters were rapidly created and shared.3,4 Scrambling to fill the critical need for NP swabs, many hospitals, businesses, and academic centers began 3D printing swabs. This effort was spearheaded by University of South Florida (USF) and Northwell Health researchers and clinicians, who designed and tested a 3D-printed NP swab from photocurable resin that was printable on 2 models of Formlabs printers.5 Several other 3D-printed NP swab designs soon followed. This innovation and problem-solving renaissance faced several challenges well known to traditional manufacturers of regulated products but novel to newcomers.
The first challlenge was that these NP swabs predate FDA oversight of medical device development and manufacturing and no testing standards existed. Designers began casting prototypes out without guidance about the critical features and clinical functions required. Many of these designs did not have a clinical evaluation pathway to test safety and efficacy.
The second challlenge was that these swabs were being produced by facilities not registered with the FDA. This raised concerns about the quality of unlisted medical products developed and manufactured at novel facilities.
The third challenge was that small-scale novel approaches may offset local shortages but could not address national needs. The self-organized infrastructure for this crisis was ad hoc, local, and lacked coordinated federal support. This led to rolling shortages of these materials for years.
Two studies were performed early in the pandemic. The first study evaluated 4 prototypes of different manufacturer designs, finding excellent concordance among them and their control swab.6 A second study demonstrated the USF swab to be noninferior to the standard of care.7 Both studies acknowledged and addressed the first challenge for their designs.
COLLABORATIONS
Interagency
Before the pandemic, the US Department of Veterans Affairs (VA) had been coordinating with the FDA, the National Institutes of Health (NIH), and the nonprofit America Makes to bring medical product development and manufacturing closer to the point of care.
At the outset of the COVID-19 pandemic, the collaboration was formalized to address new challenges.8 The objectives of this collaboration were the following: (1) host a digital repository for 3D-printed digital designs for personal protectice equipment and other medical supplies in or at risk of shortage; (2) provide scientifically based ratings for designs according to clinical and field testing; and (3) offer education to health care workers and the public about the digital manufacturing of medical goods and devices.4,9
A key output of this collaboration was the COVID 3D Trusted Repository For Users And Suppliers Through Testing (COVID 3D TRUST), a curated archive of designs. In most cases, existing FDA standards and guidance formed the basis of testing strategies with deviations due to limited access to traditional testing facilities and reagents.
To address novel NP swabs, working with its COVID 3D TRUST partners, the VA gathered a combined list of clinical- and engineering-informed customer requirements and performed a hazard analysis. The result was a list of design inputs for NP swabs and 8 standard test protocols to evaluate key functions (Table).10 These protocols are meant to benchmark novel 3D-printed swabs against the key functions of established, traditionally manufactured swabs, which have a long record of safety and efficacy. The protocols, developed by the VA and undergoing validation by the US Army, empower and inform consumers and provide performance metrics to swab designers and manufacturers. The testing protocols and preliminary test results developed by the VA are publicly available at the NIH.11
Intra-agency
The use of the inputs and verification tests noted in the Table may reduce the risk of poor design but were inadequate to evaluate the clinical safety and efficacy of novel swabs. Recognizing this, the VA Office of Healthcare Innovation and Learning (OHIL) and the Office of Research and Development (ORD) launched the Nasal Swab Objective and Statistical Evaluation (NOSE) study to formally evaluate the safety and efficacy of 3D-printed swabs in the field. This multisite clinical study was a close collaboration between the OHIL and ORD. The OHIL provided the quality system and manufacturing oversight and delivery of the swabs, and the ORD provided scientific review, research infrastructure, human subjects oversight, administrative support, and funding and fiscal oversight. The OHIL/ORD collaboration resulted in the successful completion of the NOSE study.
This study (manuscript under preparation) yielded two 3D-printing production processes and swab designs that had comparable performance to the standard of care, were manufacturable compliant with FDA guidelines, and could be produced at scale in a distributed manner. This approach directly addressed the 3 challenges described earlier.
LESSONS LEARNED
Swabs were an example of supply challenges in the pandemic, but advanced manufacturing (notably, digital designs leading to 3D-printed solutions) also served as a temporary solution to device and product shortages during the COVID-19 pandemic. Digital designs and 3D printing as manufacturing techniques have the following key advantages: (1) they are distributed in nature, both in the breadth of locations that have access to these manufacturing platforms and in the depth of material choice that can be used to fabricate products, which alleviates the threat of a disaster impacting manufacturing capacity or a material stream; (2) they do not require retooling of machinery so new products can deploy rapidly and on demand; and (3) the speed of digital iteration, printing, and revision allows for rapid product development and production.
There also are notable disadvantages to these techniques. First, because 3D printing is a newer technology, there is less general depth of knowledge regarding design and material choice for additive manufacturing. Second, the flexibility of 3D printing means that operators must increase awareness of the factors that might cause the fabrication of a part to fail in either printing or postprocessing. Third, there are significant gaps in understanding how materials and manufacturing processes will perform in high-stakes settings such as health care, where performance and biocompatibility may be critical to support life-sustaining functions. Fourth, digital files are vulnerable to intentional or unintentional alteration. These alterations might weaken design integrity and be imperceptible to the manufacturer or end user. This is a prevalent challenge in all open-source designs.
The pandemic materialized quickly and created vast supply chain challenges. To address this crisis, it was clear that the average 17-year interval between research and translation in the US was unacceptable. The VA was able to accelerate swiftly many existing processes to meet this need, build new capabilities, and establish new practices for the rapid evaluation and deployment of health care products and guidance. This agile and innovative cooperation was critical in the success of the VA’s national support for pandemic solutions.
Finally, although COVID 3D TRUST was able to provide testing of submitted designs, this collaboration was not a substitute for the “peacetime” process of manufacturing site registration with the FDA and product listing. COVID 3D TRUST could evaluate designs only, not the production process, safety, and efficacy.
CALLS TO ACTION
The pandemic's impact on medical supply chain security persists, as does the need for greater foresight and crisis preparation. We must act now to avoid experiencing again the magnitude of fatalities (civilian and veteran) and the devastation to the US economy and livelihoods that occurred during this single biological event. To this end, creating a digital stockpile of federally curated, crisis-ready designs for as-needed distribution across our US industrial base would offer a second line of defense against life-threatening supply chain interruptions. The realization of such a digital stockpile requires calls to action among multiple contributors.
Collaborations
The VA’s Fourth Mission is to improve the nation’s preparedness for response to war, terrorism, national emergencies, and natural disasters. The VA does this by developing plans and taking actions to ensure continued service to veterans, as well as to support national, state, and local emergency management, public health, safety, and homeland security efforts.
The VA partnership with the FDA and NIH during the pandemic enabled successful coordination among federal agencies. Numerous other agencies, including the US Department of Defense (DoD), the Biomedical Advanced Research and Development Authority (BARDA), and the Defense Advanced Research Projects Agency (DARPA), also developed and executed successful initiatives.12-14 The joint awareness and management of these efforts, however, could be strengthened through more formal agreements and processes in peacetime. The VA/FDA/NIH Memorandum of Understanding is a prototype example of each agency lending its subject matter expertise to address a host of pandemic challenges collectively, cooperatively, and efficiently.8
Public-private partnerships (eg, VA/FDA/NIH and America Makes) led to coordinated responses for crisis readiness. The Advanced Manufacturing Crisis Product Response Program, a multipartner collaboration that included VA, addressed 7 crisis scenarios, 3 of which were specifically related to COVID-19.15 In addition, both BARDA and DARPA had successful public-private collaborations, and the DoD supported national logistics and other efforts.12-14 Clearly, industry and government both recognize complementary synergies: (1) the depth of resources of US industry; and (2) the national resources, coordination, and clinical insight available through federal agencies that can address the challenges of future crises quickly and efficiently.
When traditional supply chains and manufacturing processes failed during the pandemic, new techniques were exploited to fill the unmet material needs. Novel techniques and product pathways, however, are untested or undeveloped. The collaboration between the ORD and OHIL in support of NP swab testing and production is an example of bringing research insight, regulated product development, and manufacturing together to support a complete product life cycle.
Joint Awareness and Management
The VA continues to refine the joint awareness and management (JAM) process of products from ideation to translation, to shorten the time from research to product delivery. JAM is a VA collaborative committee of partners from ORD research offices and technology transfer program, and the OHIL Office of Advanced Manufacturing, which seeks additional support and guidance from VHA clinical service lines, VA Office of General Council, and VA Office of Acquisitions, Logistics, and Construction.
This team enables the rapid identification of unmet veteran health care product needs. In addition, JAM leverages the resources of each group to support products from problem identification to solution ideation, regulated development, production, and delivery into clinical service lines. While the concept of JAM arose to meet the crisis needs of the pandemic, it persists in delivering advanced health care solutions to veterans.
A Proposed Plan
The next national crisis is likely to involve and threaten national health care security. We propose that federal agencies be brought together to form a federally supported digital stockpile. This digital stockpile must encompass, at minimum, the following features: (1) preservation of novel, scalable medical supplies and products generated during the COVID-19 pandemic, to avoid the loss of this work; (2) clinical maturation of those existing supplies and products to refine their features and functions under the guidance of clinical, regulatory, and manufacturing experts—and validate those outputs with clinical evidence; (3) manufacturing maturation of those existing supplies and products, such that complete design and production processes are developed with the intent to distribute to multiple public manufacturers during the next crisis; (4) a call for new designs/intake portal for new designs to be matured and curated as vulnerabilities are identified; (5) supply chain crisis drills executed to test public-private preparedness to ensure design transfer is turnkey and can be engaged quickly during the next crisis; and (6) public-private engagement to develop strategy, scenarios, and policy to ensure that when supply chains next fail, additional surge capacity can be quickly added to protect American lives and health care, and that when supply chains resume, surge capacity can be redirected or stood down to protect the competitive markets.
This digital stockpile can complement and be part of the Strategic National Stockpile. Whereas the Strategic National Stockpile is a reserve of physical products that may offset product shortages, the digital stockpile is a reserve of turnkey, transferable designs that may offset supply chain disruptions and production-capacity shortages.
CONCLUSIONS
The success of 3D-printed NP swabs is a specific example of the importance of collaborations across industry, government, innovators, and researchers. More important than a sole product, however, these collaborations demonstrated the potential for game-changing approaches to how public-private partnerships support the continuity of health care operations nationally and prevent the potential for unnecessary loss of life due to capacity and supply chain disruptions.
As the largest health care system in the US, the VA has a unique capability to lead in the assessment of other novel 3D-printed medical devices in partnership with the FDA. The VA has a unique patient-centered perspective on medical device efficacy, and as a government institution, it is a trusted independent source for medical device evaluation. The VA’s role in the evaluation of 3D-printed medical devices will benefit veterans and their families, clinicians, hospitals, and the broader public by providing a gold-standard evaluation for the growing medical 3D-printing industry to follow. By creating new pathways and expectations for how federal agencies maintain crisis preparedness—such as establishing a digital stockpile—we can be equipped to serve the US health care system and minimize the effects of supply chain disruptions.
Traditional manufacturing concentrates capacity into a few discrete locations while applying lean and just-in-time philosophies to maximize profit during times of somewhat predictable supply and demand. This approach exposed nationwide vulnerabilities even during local crises, such as the United States saline shortages following closure of a single plant in Puerto Rico following Hurricane Maria in 2017.1 Interruptions to the supply chain due to pandemic plant closure, weather, politics, or surge demand can cause immediate and lasting shortages. Nasal swabs were a clear example.
At the onset of COVID-19, 2 companies—Puritan in Guilford, Maine, and Copan in Italy—manufactured nearly all of the highly specialized nasopharyngeal (NP) swabs singled out by the Centers for Disease Control and Prevention (CDC) and the US Food and Drug Administration (FDA) to test patients for COVID-19. Demand for swabs skyrocketed as the virus spread, and they became unattainable. The lack of swabs meant patients went undiagnosed. Without knowing who was positive, people with symptoms and known contacts were presumed positive and quarantined, impacting isolated patients, the health care professionals treating them, and the entire US economy.
3-Dimensional Printing Solutions
Manufacturing NP swabs is not trivial. Their simple shape conceals complexity and requires highly specialized equipment. The lead time for one non-US machine manufacturer was > 6 months at the start of the pandemic.
Digital manufacturing/3-dimensional (3D) printing represented a potential solution to the supply chain crisis.2 Designers created digital blueprints for 3D-printed goods, face masks, face shields, and ventilator splitters were rapidly created and shared.3,4 Scrambling to fill the critical need for NP swabs, many hospitals, businesses, and academic centers began 3D printing swabs. This effort was spearheaded by University of South Florida (USF) and Northwell Health researchers and clinicians, who designed and tested a 3D-printed NP swab from photocurable resin that was printable on 2 models of Formlabs printers.5 Several other 3D-printed NP swab designs soon followed. This innovation and problem-solving renaissance faced several challenges well known to traditional manufacturers of regulated products but novel to newcomers.
The first challlenge was that these NP swabs predate FDA oversight of medical device development and manufacturing and no testing standards existed. Designers began casting prototypes out without guidance about the critical features and clinical functions required. Many of these designs did not have a clinical evaluation pathway to test safety and efficacy.
The second challlenge was that these swabs were being produced by facilities not registered with the FDA. This raised concerns about the quality of unlisted medical products developed and manufactured at novel facilities.
The third challenge was that small-scale novel approaches may offset local shortages but could not address national needs. The self-organized infrastructure for this crisis was ad hoc, local, and lacked coordinated federal support. This led to rolling shortages of these materials for years.
Two studies were performed early in the pandemic. The first study evaluated 4 prototypes of different manufacturer designs, finding excellent concordance among them and their control swab.6 A second study demonstrated the USF swab to be noninferior to the standard of care.7 Both studies acknowledged and addressed the first challenge for their designs.
COLLABORATIONS
Interagency
Before the pandemic, the US Department of Veterans Affairs (VA) had been coordinating with the FDA, the National Institutes of Health (NIH), and the nonprofit America Makes to bring medical product development and manufacturing closer to the point of care.
At the outset of the COVID-19 pandemic, the collaboration was formalized to address new challenges.8 The objectives of this collaboration were the following: (1) host a digital repository for 3D-printed digital designs for personal protectice equipment and other medical supplies in or at risk of shortage; (2) provide scientifically based ratings for designs according to clinical and field testing; and (3) offer education to health care workers and the public about the digital manufacturing of medical goods and devices.4,9
A key output of this collaboration was the COVID 3D Trusted Repository For Users And Suppliers Through Testing (COVID 3D TRUST), a curated archive of designs. In most cases, existing FDA standards and guidance formed the basis of testing strategies with deviations due to limited access to traditional testing facilities and reagents.
To address novel NP swabs, working with its COVID 3D TRUST partners, the VA gathered a combined list of clinical- and engineering-informed customer requirements and performed a hazard analysis. The result was a list of design inputs for NP swabs and 8 standard test protocols to evaluate key functions (Table).10 These protocols are meant to benchmark novel 3D-printed swabs against the key functions of established, traditionally manufactured swabs, which have a long record of safety and efficacy. The protocols, developed by the VA and undergoing validation by the US Army, empower and inform consumers and provide performance metrics to swab designers and manufacturers. The testing protocols and preliminary test results developed by the VA are publicly available at the NIH.11
Intra-agency
The use of the inputs and verification tests noted in the Table may reduce the risk of poor design but were inadequate to evaluate the clinical safety and efficacy of novel swabs. Recognizing this, the VA Office of Healthcare Innovation and Learning (OHIL) and the Office of Research and Development (ORD) launched the Nasal Swab Objective and Statistical Evaluation (NOSE) study to formally evaluate the safety and efficacy of 3D-printed swabs in the field. This multisite clinical study was a close collaboration between the OHIL and ORD. The OHIL provided the quality system and manufacturing oversight and delivery of the swabs, and the ORD provided scientific review, research infrastructure, human subjects oversight, administrative support, and funding and fiscal oversight. The OHIL/ORD collaboration resulted in the successful completion of the NOSE study.
This study (manuscript under preparation) yielded two 3D-printing production processes and swab designs that had comparable performance to the standard of care, were manufacturable compliant with FDA guidelines, and could be produced at scale in a distributed manner. This approach directly addressed the 3 challenges described earlier.
LESSONS LEARNED
Swabs were an example of supply challenges in the pandemic, but advanced manufacturing (notably, digital designs leading to 3D-printed solutions) also served as a temporary solution to device and product shortages during the COVID-19 pandemic. Digital designs and 3D printing as manufacturing techniques have the following key advantages: (1) they are distributed in nature, both in the breadth of locations that have access to these manufacturing platforms and in the depth of material choice that can be used to fabricate products, which alleviates the threat of a disaster impacting manufacturing capacity or a material stream; (2) they do not require retooling of machinery so new products can deploy rapidly and on demand; and (3) the speed of digital iteration, printing, and revision allows for rapid product development and production.
There also are notable disadvantages to these techniques. First, because 3D printing is a newer technology, there is less general depth of knowledge regarding design and material choice for additive manufacturing. Second, the flexibility of 3D printing means that operators must increase awareness of the factors that might cause the fabrication of a part to fail in either printing or postprocessing. Third, there are significant gaps in understanding how materials and manufacturing processes will perform in high-stakes settings such as health care, where performance and biocompatibility may be critical to support life-sustaining functions. Fourth, digital files are vulnerable to intentional or unintentional alteration. These alterations might weaken design integrity and be imperceptible to the manufacturer or end user. This is a prevalent challenge in all open-source designs.
The pandemic materialized quickly and created vast supply chain challenges. To address this crisis, it was clear that the average 17-year interval between research and translation in the US was unacceptable. The VA was able to accelerate swiftly many existing processes to meet this need, build new capabilities, and establish new practices for the rapid evaluation and deployment of health care products and guidance. This agile and innovative cooperation was critical in the success of the VA’s national support for pandemic solutions.
Finally, although COVID 3D TRUST was able to provide testing of submitted designs, this collaboration was not a substitute for the “peacetime” process of manufacturing site registration with the FDA and product listing. COVID 3D TRUST could evaluate designs only, not the production process, safety, and efficacy.
CALLS TO ACTION
The pandemic's impact on medical supply chain security persists, as does the need for greater foresight and crisis preparation. We must act now to avoid experiencing again the magnitude of fatalities (civilian and veteran) and the devastation to the US economy and livelihoods that occurred during this single biological event. To this end, creating a digital stockpile of federally curated, crisis-ready designs for as-needed distribution across our US industrial base would offer a second line of defense against life-threatening supply chain interruptions. The realization of such a digital stockpile requires calls to action among multiple contributors.
Collaborations
The VA’s Fourth Mission is to improve the nation’s preparedness for response to war, terrorism, national emergencies, and natural disasters. The VA does this by developing plans and taking actions to ensure continued service to veterans, as well as to support national, state, and local emergency management, public health, safety, and homeland security efforts.
The VA partnership with the FDA and NIH during the pandemic enabled successful coordination among federal agencies. Numerous other agencies, including the US Department of Defense (DoD), the Biomedical Advanced Research and Development Authority (BARDA), and the Defense Advanced Research Projects Agency (DARPA), also developed and executed successful initiatives.12-14 The joint awareness and management of these efforts, however, could be strengthened through more formal agreements and processes in peacetime. The VA/FDA/NIH Memorandum of Understanding is a prototype example of each agency lending its subject matter expertise to address a host of pandemic challenges collectively, cooperatively, and efficiently.8
Public-private partnerships (eg, VA/FDA/NIH and America Makes) led to coordinated responses for crisis readiness. The Advanced Manufacturing Crisis Product Response Program, a multipartner collaboration that included VA, addressed 7 crisis scenarios, 3 of which were specifically related to COVID-19.15 In addition, both BARDA and DARPA had successful public-private collaborations, and the DoD supported national logistics and other efforts.12-14 Clearly, industry and government both recognize complementary synergies: (1) the depth of resources of US industry; and (2) the national resources, coordination, and clinical insight available through federal agencies that can address the challenges of future crises quickly and efficiently.
When traditional supply chains and manufacturing processes failed during the pandemic, new techniques were exploited to fill the unmet material needs. Novel techniques and product pathways, however, are untested or undeveloped. The collaboration between the ORD and OHIL in support of NP swab testing and production is an example of bringing research insight, regulated product development, and manufacturing together to support a complete product life cycle.
Joint Awareness and Management
The VA continues to refine the joint awareness and management (JAM) process of products from ideation to translation, to shorten the time from research to product delivery. JAM is a VA collaborative committee of partners from ORD research offices and technology transfer program, and the OHIL Office of Advanced Manufacturing, which seeks additional support and guidance from VHA clinical service lines, VA Office of General Council, and VA Office of Acquisitions, Logistics, and Construction.
This team enables the rapid identification of unmet veteran health care product needs. In addition, JAM leverages the resources of each group to support products from problem identification to solution ideation, regulated development, production, and delivery into clinical service lines. While the concept of JAM arose to meet the crisis needs of the pandemic, it persists in delivering advanced health care solutions to veterans.
A Proposed Plan
The next national crisis is likely to involve and threaten national health care security. We propose that federal agencies be brought together to form a federally supported digital stockpile. This digital stockpile must encompass, at minimum, the following features: (1) preservation of novel, scalable medical supplies and products generated during the COVID-19 pandemic, to avoid the loss of this work; (2) clinical maturation of those existing supplies and products to refine their features and functions under the guidance of clinical, regulatory, and manufacturing experts—and validate those outputs with clinical evidence; (3) manufacturing maturation of those existing supplies and products, such that complete design and production processes are developed with the intent to distribute to multiple public manufacturers during the next crisis; (4) a call for new designs/intake portal for new designs to be matured and curated as vulnerabilities are identified; (5) supply chain crisis drills executed to test public-private preparedness to ensure design transfer is turnkey and can be engaged quickly during the next crisis; and (6) public-private engagement to develop strategy, scenarios, and policy to ensure that when supply chains next fail, additional surge capacity can be quickly added to protect American lives and health care, and that when supply chains resume, surge capacity can be redirected or stood down to protect the competitive markets.
This digital stockpile can complement and be part of the Strategic National Stockpile. Whereas the Strategic National Stockpile is a reserve of physical products that may offset product shortages, the digital stockpile is a reserve of turnkey, transferable designs that may offset supply chain disruptions and production-capacity shortages.
CONCLUSIONS
The success of 3D-printed NP swabs is a specific example of the importance of collaborations across industry, government, innovators, and researchers. More important than a sole product, however, these collaborations demonstrated the potential for game-changing approaches to how public-private partnerships support the continuity of health care operations nationally and prevent the potential for unnecessary loss of life due to capacity and supply chain disruptions.
As the largest health care system in the US, the VA has a unique capability to lead in the assessment of other novel 3D-printed medical devices in partnership with the FDA. The VA has a unique patient-centered perspective on medical device efficacy, and as a government institution, it is a trusted independent source for medical device evaluation. The VA’s role in the evaluation of 3D-printed medical devices will benefit veterans and their families, clinicians, hospitals, and the broader public by providing a gold-standard evaluation for the growing medical 3D-printing industry to follow. By creating new pathways and expectations for how federal agencies maintain crisis preparedness—such as establishing a digital stockpile—we can be equipped to serve the US health care system and minimize the effects of supply chain disruptions.
1. Sacks CA, Kesselheim AS, Fralick M. The shortage of normal saline in the wake of Hurricane Maria. JAMA Intern Med. 2018;178(7):885–886. doi:10.1001/jamainternmed.2018.1936
2. Bauchner H, Fontanarosa PB, Livingston EH. Conserving supply of personal protective equipment–a call for ideas. JAMA. 2020;323(19):1911. doi:10.1001/jama.2020.4770
3. Sinha MS, Bourgeois FT, Sorger PK. Personal protective equipment for COVID-19: distributed fabrication and additive manufacturing. Am J Public Health. 2020;110(8):1162-1164. doi:10.2105/AJPH.2020.305753
4. McCarthy MC, Di Prima M, Cruz P, et al. Trust in the time of Covid-19: 3D printing and additive manufacturing (3DP/AM) as a solution to supply chain gaps. NEJM Catalyst. 2021;2(6). doi:10.1056/CAT.21.0321
5. Ford J, Goldstein T, Trahan S, Neuwirth A, Tatoris K, Decker S. A 3D-printed nasopharyngeal swab for COVID-19 diagnostic testing. 3D Print Med. 2020;6(1):21. Published 2020 Aug 15. doi:10.1186/s41205-020-00076-3
6. Callahan CJ, Lee R, Zulauf K, et al. Open development and clinical validation of multiple 3D-printed sample-collection swabs: rapid resolution of a critical COVID-19 testing bottleneck. Preprint. medRxiv. 2020;2020.04.14.20065094. Published 2020 Apr 17. doi:10.1101/2020.04.14.20065094
7. Decker SJ, Goldstein TA, Ford JM, et al. 3-dimensional printed alternative to the standard synthetic flocked nasopharyngeal swabs used for coronavirus disease 2019 testing. Clin Infect Dis. 2021;73(9):e3027-e3032. doi:10.1093/cid/ciaa1366
8. US Food and Drug Administration. Memorandum of understanding: rapid response to Covid-19 using 3d printing between National Institutes of Health within U.S. Department of Health and Human Services and Food and Drug Administration, U.S. Department of Health and Human Services and Veterans Health Administration within the U.S. Department of Veterans Affairs. March 26, 2020. Accessed August 31, 2023. https://www.fda.gov/about-fda/domestic-mous/mou-225-20-008
9. National Institutes of Health, NIH 3D Print Exchange. Covid 3D trust: trusted repository for users and suppliers through testing. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=search
10. National Institutes of Health, NIH 3D Print Exchange. 3D printed nasal swabs - assessment criteria. August 17, 2020. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=swabassessment
11. National Institutes of Health, NIH 3D Print Exchange. 3D printed nasal swabs - general information. August 17, 2020. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=swabinfo
12. US Department of Defense. Coronavirus: DOD response. December 20, 2022. Accessed August 31, 2023. https://www.defense.gov/Spotlights/Coronavirus-DoD-Response
13. US Department of Health and Human Services, Biomedical Advanced Research and Development Authority. BARDA COVID-19 response. Updated May 25, 2023. Accessed August 31, 2023. https://www.medicalcountermeasures.gov/barda/barda-covid-19-response
14. Green S. Pandemic prevention platform (P3). Accessed August 31, 2023. https://www.darpa.mil/program/pandemic-prevention-platform
15. America Makes. America makes completes successful scenario testing for crisis response program [press release]. May 25, 2021. Accessed August 31, 2023. https://www.americamakes.us/america-makes-completes-successful-scenario-testing-for-crisis-response-program
1. Sacks CA, Kesselheim AS, Fralick M. The shortage of normal saline in the wake of Hurricane Maria. JAMA Intern Med. 2018;178(7):885–886. doi:10.1001/jamainternmed.2018.1936
2. Bauchner H, Fontanarosa PB, Livingston EH. Conserving supply of personal protective equipment–a call for ideas. JAMA. 2020;323(19):1911. doi:10.1001/jama.2020.4770
3. Sinha MS, Bourgeois FT, Sorger PK. Personal protective equipment for COVID-19: distributed fabrication and additive manufacturing. Am J Public Health. 2020;110(8):1162-1164. doi:10.2105/AJPH.2020.305753
4. McCarthy MC, Di Prima M, Cruz P, et al. Trust in the time of Covid-19: 3D printing and additive manufacturing (3DP/AM) as a solution to supply chain gaps. NEJM Catalyst. 2021;2(6). doi:10.1056/CAT.21.0321
5. Ford J, Goldstein T, Trahan S, Neuwirth A, Tatoris K, Decker S. A 3D-printed nasopharyngeal swab for COVID-19 diagnostic testing. 3D Print Med. 2020;6(1):21. Published 2020 Aug 15. doi:10.1186/s41205-020-00076-3
6. Callahan CJ, Lee R, Zulauf K, et al. Open development and clinical validation of multiple 3D-printed sample-collection swabs: rapid resolution of a critical COVID-19 testing bottleneck. Preprint. medRxiv. 2020;2020.04.14.20065094. Published 2020 Apr 17. doi:10.1101/2020.04.14.20065094
7. Decker SJ, Goldstein TA, Ford JM, et al. 3-dimensional printed alternative to the standard synthetic flocked nasopharyngeal swabs used for coronavirus disease 2019 testing. Clin Infect Dis. 2021;73(9):e3027-e3032. doi:10.1093/cid/ciaa1366
8. US Food and Drug Administration. Memorandum of understanding: rapid response to Covid-19 using 3d printing between National Institutes of Health within U.S. Department of Health and Human Services and Food and Drug Administration, U.S. Department of Health and Human Services and Veterans Health Administration within the U.S. Department of Veterans Affairs. March 26, 2020. Accessed August 31, 2023. https://www.fda.gov/about-fda/domestic-mous/mou-225-20-008
9. National Institutes of Health, NIH 3D Print Exchange. Covid 3D trust: trusted repository for users and suppliers through testing. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=search
10. National Institutes of Health, NIH 3D Print Exchange. 3D printed nasal swabs - assessment criteria. August 17, 2020. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=swabassessment
11. National Institutes of Health, NIH 3D Print Exchange. 3D printed nasal swabs - general information. August 17, 2020. Accessed August 31, 2023. https://3d.nih.gov/collections/covid-19-response?tab=swabinfo
12. US Department of Defense. Coronavirus: DOD response. December 20, 2022. Accessed August 31, 2023. https://www.defense.gov/Spotlights/Coronavirus-DoD-Response
13. US Department of Health and Human Services, Biomedical Advanced Research and Development Authority. BARDA COVID-19 response. Updated May 25, 2023. Accessed August 31, 2023. https://www.medicalcountermeasures.gov/barda/barda-covid-19-response
14. Green S. Pandemic prevention platform (P3). Accessed August 31, 2023. https://www.darpa.mil/program/pandemic-prevention-platform
15. America Makes. America makes completes successful scenario testing for crisis response program [press release]. May 25, 2021. Accessed August 31, 2023. https://www.americamakes.us/america-makes-completes-successful-scenario-testing-for-crisis-response-program
VA SHIELD: A Biorepository for Veterans and the Nation
The Veterans Health Administration (VHA) clinicians, clinician-investigators, and investigators perform basic and translational research for the benefit of our nation and are widely recognized for treating patients and discovering cures.1,2 In May 2020, the US Department of Veterans Affairs (VA) launched the VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD). The goal of this novel enterprise was to assemble a comprehensive specimen and data repository for emerging life-threatening diseases and to address future challenges. VA SHIELD was specifically charged with creating a biorepository to advance research, improve diagnostic and therapeutic capabilities, and develop strategies for immediate deployment to VA clinical environments. One main objective of VA SHIELD is to harness the clinical and scientific strengths of the VA in order to create a more cohesive collaboration between preexisting clinical research efforts within the VA.
ANATOMY OF VA SHIELD
The charge and scope of VA SHIELD is unique.3 As an entity, this program leverages the strengths of the diverse VHA network, has a broad potential impact on national health care, is positioned to respond rapidly to national and international health-related events, and substantially contributes to clinical research and development. In addition, VA SHIELD upholds VA’s Fourth Mission, which is to contribute to national emergencies and support emergency management, public health, safety, and homeland security efforts.
VA SHIELD is part of the VA Office of Research and Development (ORD). The coordinating center (CC), headquartered in Cleveland, Ohio, is the central operational partner, leading VA SHIELD and interacting with other important VA programs, including laboratory, clinical science, rehabilitation, and health services. The VA SHIELD CC oversees all aspects of operations, including biospecimen collection, creating and enforcing of standard operating procedures, ensuring the quality of the samples, processing research applications, distribution of samples, financing, and progress reports. The CC also initiates and maintains interagency collaborations, convenes stakeholders, and develops strategic plans to address emerging diseases.
The VA SHIELD Executive Steering Committee (ESC) is composed of infectious disease, biorepository, and public health specialists. The ESC provides scientific and programmatic direction to the CC, including operational activities and guidance regarding biorepository priorities and scientific agenda, and measuring and reporting on VA SHIELD accomplishments.
The primary function of the Programmatic and Scientific Review Board (PSRB) is to evaluate incoming research proposals for specimen and data use for feasibility and make recommendations to the VA SHIELD CC. The PSRB evaluates and ensures that data and specimen use align with VA SHIELD ethical, clinical, and scientific objectives.
VA SHIELD IN PRACTICE
VA SHIELD consisted of 11 specimen collection sites (Atlanta, GA; Boise, ID; Bronx, NY; Cincinnati, OH; Cleveland, OH; Durham, NC; Houston, TX; Los Angeles, CA; Mountain Home, TN; Palo Alto, CA; and Tucson, AZ), a data processing center in Boston, MA, and 2 central biorepositories in Palo Alto, CA, and Tucson, AZ. Information flow is a coordinated process among specimen collection sites, data processing centers, and the biorepositories. Initially, each local collection site identifies residual specimens that would have been discarded after clinical laboratory testing. These samples currently account for the majority of biological material within VA SHIELD via a novel collection protocol known as “Sweep,” which allows residual clinical discarded samples as well as samples from patients with new emerging infectious and noninfectious diseases of concern to be collected at the time of first emergence and submitted to VA SHIELD during the course of routine veteran health care.3 These clinical discarded samples are de-identified and transferred from the clinical laboratory to VA SHIELD. The VA Central Institutional Review Board (cIRB) has approved the use of these samples as nonhuman subject research. Biological samples are collected, processed, aliquoted, shipped to, and stored at the central biorepository sites.
The Umbrella amendment to Sweep that has been approved also by the VA cIRB, will allow VA SHIELD sites to prospectively consent veterans and collect biospecimens and additional clinical and self-reported information. The implementation of Umbrella could significantly enhance collection and research. Although Sweep is a onetime collection of samples, the Umbrella protocol will allow the longitudinal collection of samples from the same patient. Additionally, the Umbrella amendment will allow VA SHIELD to accept samples from other preexisting biorepositories or specimen collections.
Central Biorepositories
VA SHIELD has a federated organization with 2 central specimen biorepositories (Palo Alto, CA and Tucson, AZ), and an enterprise data processing center (Boston, MA). The specimen biorepositories receive de-identified specimens that are stored until distribution to approved research projects. The samples and data are linked using an electronic honest broker system to protect privacy, which integrates de-identified specimens with requested clinical and demographic data as needed for approved projects. The honest broker system is operated by independent personnel and does not have vested interest in any studies being performed under VA SHIELD. The integration of sample and associated data is done only as needed when characterization of the donor/participant is necessary byresearch aims or project outcomes. The process is facilitated by a nationally supported laboratory information management system (LIMS), managed by the VA SHIELD data center, that assists with all data requests. The clinical and demographic data are collected from VA electronic health record (EHR), available through VA Corporate Data Warehouse (CDW) and VA Informatics and Computing Infrastructure (VINCI) as needed and integrated with the biorepository samples information for approved VA SHIELD studies. The CDW is the largest longitudinal EHR data collection in the US and has the ability to provide access to national clinical and demographic data.
VA SHIELD interacts with multiple VA programs and other entities (Figure). For example, Surveillance Platform for Enteric and Respiratory Infectious Organisms at United States Veterans Affairs Medical Centers (SUPERNOVA) is a network of 5 VA medical centers supported by the Centers for Disease Control and Prevention.4 Its initial goal was to perform surveillance for acute gastroenteritis. In 2020, SUPERNOVA shifted to conduct surveillance for COVID-19 variants among veterans.5 VA SHIELD also interacts with VHA genomic surveillance and sequencing programs: the VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE) and VA Sequencing for Research Clinical and Epidemiology (SeqFORCE), described by Krishnan and colleagues.6
Working Groups
To encourage research projects that use biospecimens, VA SHIELD developed content-oriented research working groups. The goal is to inspire collaborations between VA scientists and prevent redundant or overlapping projects. Currently working groups are focused on long COVID, and COVID-19 neurology, pathogen host response, epidemiology and sequencing, cancer and cancer biomarkers, antimicrobial resistance, and vector-borne diseases. Working groups meet regularly to discuss projects and report progress. Working groups also may consider samples that might benefit VA health research and identify potential veteran populations for future research. Working groups connect VA SHIELD and investigators and guide the collection and use of resources.
Ethical Considerations
We recognize the significant ethical concerns for biobanking of specimens. However, there is no general consensus or guideline that addresses all of the complex ethical issues regarding biobanking.7 To address these ethical concerns, we applied the VA Ethical Framework Principles for Access to and Use of Veteran Data principles to VA SHIELD, including all parties who oversee the access to, sharing of, or the use of data, or who access or use its data.8
Conclusions
The VA has assembled a scientific enterprise dedicated to combating emerging infectious diseases and other threats to our patients. This enterprise has been modeled in its structure and oversight to support VA SHIELD. The establishment of a real-time biorepository and data procurement system linked to clinical samples is a bold step forward to address current and future challenges. Similarly, the integration and cooperation of multiple arms within the VA that transcend disciplines and boundaries promise to shepherd a new era of system-wide investigation. In the future, VA SHIELD will integrate with other existing government agencies to advance mutual scientific agendas. VA SHIELD has established the data and biorepository infrastructure to develop innovative and novel technologies to address future challenges. The alignment of basic science, clinical, and translational research goals under one governance is a significant advancement compared with previous models of research coordination.
VA SHIELD was developed to meet an immediate need; it was also framed to be a research enterprise that harnesses the robust clinical and research environment in VHA. The VA SHIELD infrastructure was conceptualized to harmonize specimen and data collection across the VA, allowing researchers to leverage broader collection efforts. Building a biorepository and data collection system within the largest integrated health care system has the potential to provide a lasting impact on VHA and on our nation’s health.
Acknowledgments
The authors wish to acknowledge Ms. Daphne Swancutt for her contribution as copywriter for this manuscript. The authors wish to acknowledge the VA SHIELD investigators: Mary Cloud Ammons, David Beenhouwer, Sheldon T. Brown, Victoria Davey, Abhinav Diwan, John B. Harley, Mark Holodniy, Vincent C. Marconi, Jonathan Moorman, Emerson B. Padiernos, Ian F. Robey, Maria Rodriguez-Barradas, Jason Wertheim, Christopher W. Woods.
1. Lipshy KA, Itani K, Chu D, et al. Sentinel contributions of US Department of Veterans Affairs surgeons in shaping the face of health care. JAMA Surg. 2021;156(4):380-386. doi:10.1001/jamasurg.2020.6372
2. Zucker S, Crabbe JC, Cooper G 4th, et al. Veterans Administration support for medical research: opinions of the endangered species of physician-scientists. FASEB J. 2004;18(13):1481-1486. doi:10.1096/fj.04-1573lfe
3. Harley JB, Pyarajan S, Partan ES, et al. The US Department of Veterans Affairs Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD): a biorepository addressing national health threats. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac641
4. Meites E, Bajema KL, Kambhampati A, et al; SUPERNOVA COVID-19 Surveillance Group. Adapting the Surveillance Platform for Enteric and Respiratory Infectious Organisms at United States Veterans Affairs Medical Centers (SUPERNOVA) for COVID-19 among hospitalized adults: surveillance protocol. Front Public Health. 2021;9:739076. doi:10.3389/fpubh.2021.739076
5. Bajema KL, Dahl RM, Evener SL, et al; SUPERNOVA COVID-19 Surveillance Group; Surveillance Platform for Enteric and Respiratory Infectious Organisms at the VA (SUPERNOVA) COVID-19 Surveillance Group. Comparative effectiveness and antibody responses to Moderna and Pfizer-BioNTech COVID-19 vaccines among hospitalized veterans–five Veterans Affairs Medical Centers, United States, February 1-September 30, 2021. MMWR Morb Mortal Wkly Rep. 2021;70(49):1700-1705. doi:10.15585/mmwr.mm7049a2external icon
6. Krishnan J, Woods C, Holodniy M, et al. Nationwide genomic surveillance and response to coronavirus disease 2019 (COVID-19): SeqCURE and SeqFORCE consortiums. Fed Pract. 2023;40(suppl 5):S44-S47. doi:10.12788/fp.0417
7. Ashcroft JW, Macpherson CC. The complex ethical landscape of biobanking. Lancet Public Health. 2019;(6):e274-e275. doi:10.1016/S2468-2667(19)30081-7
8. Principle-Based Ethics Framework for Access to and Use of Veteran Data. Fed Regist. 2022;87(129):40451-40452.
The Veterans Health Administration (VHA) clinicians, clinician-investigators, and investigators perform basic and translational research for the benefit of our nation and are widely recognized for treating patients and discovering cures.1,2 In May 2020, the US Department of Veterans Affairs (VA) launched the VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD). The goal of this novel enterprise was to assemble a comprehensive specimen and data repository for emerging life-threatening diseases and to address future challenges. VA SHIELD was specifically charged with creating a biorepository to advance research, improve diagnostic and therapeutic capabilities, and develop strategies for immediate deployment to VA clinical environments. One main objective of VA SHIELD is to harness the clinical and scientific strengths of the VA in order to create a more cohesive collaboration between preexisting clinical research efforts within the VA.
ANATOMY OF VA SHIELD
The charge and scope of VA SHIELD is unique.3 As an entity, this program leverages the strengths of the diverse VHA network, has a broad potential impact on national health care, is positioned to respond rapidly to national and international health-related events, and substantially contributes to clinical research and development. In addition, VA SHIELD upholds VA’s Fourth Mission, which is to contribute to national emergencies and support emergency management, public health, safety, and homeland security efforts.
VA SHIELD is part of the VA Office of Research and Development (ORD). The coordinating center (CC), headquartered in Cleveland, Ohio, is the central operational partner, leading VA SHIELD and interacting with other important VA programs, including laboratory, clinical science, rehabilitation, and health services. The VA SHIELD CC oversees all aspects of operations, including biospecimen collection, creating and enforcing of standard operating procedures, ensuring the quality of the samples, processing research applications, distribution of samples, financing, and progress reports. The CC also initiates and maintains interagency collaborations, convenes stakeholders, and develops strategic plans to address emerging diseases.
The VA SHIELD Executive Steering Committee (ESC) is composed of infectious disease, biorepository, and public health specialists. The ESC provides scientific and programmatic direction to the CC, including operational activities and guidance regarding biorepository priorities and scientific agenda, and measuring and reporting on VA SHIELD accomplishments.
The primary function of the Programmatic and Scientific Review Board (PSRB) is to evaluate incoming research proposals for specimen and data use for feasibility and make recommendations to the VA SHIELD CC. The PSRB evaluates and ensures that data and specimen use align with VA SHIELD ethical, clinical, and scientific objectives.
VA SHIELD IN PRACTICE
VA SHIELD consisted of 11 specimen collection sites (Atlanta, GA; Boise, ID; Bronx, NY; Cincinnati, OH; Cleveland, OH; Durham, NC; Houston, TX; Los Angeles, CA; Mountain Home, TN; Palo Alto, CA; and Tucson, AZ), a data processing center in Boston, MA, and 2 central biorepositories in Palo Alto, CA, and Tucson, AZ. Information flow is a coordinated process among specimen collection sites, data processing centers, and the biorepositories. Initially, each local collection site identifies residual specimens that would have been discarded after clinical laboratory testing. These samples currently account for the majority of biological material within VA SHIELD via a novel collection protocol known as “Sweep,” which allows residual clinical discarded samples as well as samples from patients with new emerging infectious and noninfectious diseases of concern to be collected at the time of first emergence and submitted to VA SHIELD during the course of routine veteran health care.3 These clinical discarded samples are de-identified and transferred from the clinical laboratory to VA SHIELD. The VA Central Institutional Review Board (cIRB) has approved the use of these samples as nonhuman subject research. Biological samples are collected, processed, aliquoted, shipped to, and stored at the central biorepository sites.
The Umbrella amendment to Sweep that has been approved also by the VA cIRB, will allow VA SHIELD sites to prospectively consent veterans and collect biospecimens and additional clinical and self-reported information. The implementation of Umbrella could significantly enhance collection and research. Although Sweep is a onetime collection of samples, the Umbrella protocol will allow the longitudinal collection of samples from the same patient. Additionally, the Umbrella amendment will allow VA SHIELD to accept samples from other preexisting biorepositories or specimen collections.
Central Biorepositories
VA SHIELD has a federated organization with 2 central specimen biorepositories (Palo Alto, CA and Tucson, AZ), and an enterprise data processing center (Boston, MA). The specimen biorepositories receive de-identified specimens that are stored until distribution to approved research projects. The samples and data are linked using an electronic honest broker system to protect privacy, which integrates de-identified specimens with requested clinical and demographic data as needed for approved projects. The honest broker system is operated by independent personnel and does not have vested interest in any studies being performed under VA SHIELD. The integration of sample and associated data is done only as needed when characterization of the donor/participant is necessary byresearch aims or project outcomes. The process is facilitated by a nationally supported laboratory information management system (LIMS), managed by the VA SHIELD data center, that assists with all data requests. The clinical and demographic data are collected from VA electronic health record (EHR), available through VA Corporate Data Warehouse (CDW) and VA Informatics and Computing Infrastructure (VINCI) as needed and integrated with the biorepository samples information for approved VA SHIELD studies. The CDW is the largest longitudinal EHR data collection in the US and has the ability to provide access to national clinical and demographic data.
VA SHIELD interacts with multiple VA programs and other entities (Figure). For example, Surveillance Platform for Enteric and Respiratory Infectious Organisms at United States Veterans Affairs Medical Centers (SUPERNOVA) is a network of 5 VA medical centers supported by the Centers for Disease Control and Prevention.4 Its initial goal was to perform surveillance for acute gastroenteritis. In 2020, SUPERNOVA shifted to conduct surveillance for COVID-19 variants among veterans.5 VA SHIELD also interacts with VHA genomic surveillance and sequencing programs: the VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE) and VA Sequencing for Research Clinical and Epidemiology (SeqFORCE), described by Krishnan and colleagues.6
Working Groups
To encourage research projects that use biospecimens, VA SHIELD developed content-oriented research working groups. The goal is to inspire collaborations between VA scientists and prevent redundant or overlapping projects. Currently working groups are focused on long COVID, and COVID-19 neurology, pathogen host response, epidemiology and sequencing, cancer and cancer biomarkers, antimicrobial resistance, and vector-borne diseases. Working groups meet regularly to discuss projects and report progress. Working groups also may consider samples that might benefit VA health research and identify potential veteran populations for future research. Working groups connect VA SHIELD and investigators and guide the collection and use of resources.
Ethical Considerations
We recognize the significant ethical concerns for biobanking of specimens. However, there is no general consensus or guideline that addresses all of the complex ethical issues regarding biobanking.7 To address these ethical concerns, we applied the VA Ethical Framework Principles for Access to and Use of Veteran Data principles to VA SHIELD, including all parties who oversee the access to, sharing of, or the use of data, or who access or use its data.8
Conclusions
The VA has assembled a scientific enterprise dedicated to combating emerging infectious diseases and other threats to our patients. This enterprise has been modeled in its structure and oversight to support VA SHIELD. The establishment of a real-time biorepository and data procurement system linked to clinical samples is a bold step forward to address current and future challenges. Similarly, the integration and cooperation of multiple arms within the VA that transcend disciplines and boundaries promise to shepherd a new era of system-wide investigation. In the future, VA SHIELD will integrate with other existing government agencies to advance mutual scientific agendas. VA SHIELD has established the data and biorepository infrastructure to develop innovative and novel technologies to address future challenges. The alignment of basic science, clinical, and translational research goals under one governance is a significant advancement compared with previous models of research coordination.
VA SHIELD was developed to meet an immediate need; it was also framed to be a research enterprise that harnesses the robust clinical and research environment in VHA. The VA SHIELD infrastructure was conceptualized to harmonize specimen and data collection across the VA, allowing researchers to leverage broader collection efforts. Building a biorepository and data collection system within the largest integrated health care system has the potential to provide a lasting impact on VHA and on our nation’s health.
Acknowledgments
The authors wish to acknowledge Ms. Daphne Swancutt for her contribution as copywriter for this manuscript. The authors wish to acknowledge the VA SHIELD investigators: Mary Cloud Ammons, David Beenhouwer, Sheldon T. Brown, Victoria Davey, Abhinav Diwan, John B. Harley, Mark Holodniy, Vincent C. Marconi, Jonathan Moorman, Emerson B. Padiernos, Ian F. Robey, Maria Rodriguez-Barradas, Jason Wertheim, Christopher W. Woods.
The Veterans Health Administration (VHA) clinicians, clinician-investigators, and investigators perform basic and translational research for the benefit of our nation and are widely recognized for treating patients and discovering cures.1,2 In May 2020, the US Department of Veterans Affairs (VA) launched the VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD). The goal of this novel enterprise was to assemble a comprehensive specimen and data repository for emerging life-threatening diseases and to address future challenges. VA SHIELD was specifically charged with creating a biorepository to advance research, improve diagnostic and therapeutic capabilities, and develop strategies for immediate deployment to VA clinical environments. One main objective of VA SHIELD is to harness the clinical and scientific strengths of the VA in order to create a more cohesive collaboration between preexisting clinical research efforts within the VA.
ANATOMY OF VA SHIELD
The charge and scope of VA SHIELD is unique.3 As an entity, this program leverages the strengths of the diverse VHA network, has a broad potential impact on national health care, is positioned to respond rapidly to national and international health-related events, and substantially contributes to clinical research and development. In addition, VA SHIELD upholds VA’s Fourth Mission, which is to contribute to national emergencies and support emergency management, public health, safety, and homeland security efforts.
VA SHIELD is part of the VA Office of Research and Development (ORD). The coordinating center (CC), headquartered in Cleveland, Ohio, is the central operational partner, leading VA SHIELD and interacting with other important VA programs, including laboratory, clinical science, rehabilitation, and health services. The VA SHIELD CC oversees all aspects of operations, including biospecimen collection, creating and enforcing of standard operating procedures, ensuring the quality of the samples, processing research applications, distribution of samples, financing, and progress reports. The CC also initiates and maintains interagency collaborations, convenes stakeholders, and develops strategic plans to address emerging diseases.
The VA SHIELD Executive Steering Committee (ESC) is composed of infectious disease, biorepository, and public health specialists. The ESC provides scientific and programmatic direction to the CC, including operational activities and guidance regarding biorepository priorities and scientific agenda, and measuring and reporting on VA SHIELD accomplishments.
The primary function of the Programmatic and Scientific Review Board (PSRB) is to evaluate incoming research proposals for specimen and data use for feasibility and make recommendations to the VA SHIELD CC. The PSRB evaluates and ensures that data and specimen use align with VA SHIELD ethical, clinical, and scientific objectives.
VA SHIELD IN PRACTICE
VA SHIELD consisted of 11 specimen collection sites (Atlanta, GA; Boise, ID; Bronx, NY; Cincinnati, OH; Cleveland, OH; Durham, NC; Houston, TX; Los Angeles, CA; Mountain Home, TN; Palo Alto, CA; and Tucson, AZ), a data processing center in Boston, MA, and 2 central biorepositories in Palo Alto, CA, and Tucson, AZ. Information flow is a coordinated process among specimen collection sites, data processing centers, and the biorepositories. Initially, each local collection site identifies residual specimens that would have been discarded after clinical laboratory testing. These samples currently account for the majority of biological material within VA SHIELD via a novel collection protocol known as “Sweep,” which allows residual clinical discarded samples as well as samples from patients with new emerging infectious and noninfectious diseases of concern to be collected at the time of first emergence and submitted to VA SHIELD during the course of routine veteran health care.3 These clinical discarded samples are de-identified and transferred from the clinical laboratory to VA SHIELD. The VA Central Institutional Review Board (cIRB) has approved the use of these samples as nonhuman subject research. Biological samples are collected, processed, aliquoted, shipped to, and stored at the central biorepository sites.
The Umbrella amendment to Sweep that has been approved also by the VA cIRB, will allow VA SHIELD sites to prospectively consent veterans and collect biospecimens and additional clinical and self-reported information. The implementation of Umbrella could significantly enhance collection and research. Although Sweep is a onetime collection of samples, the Umbrella protocol will allow the longitudinal collection of samples from the same patient. Additionally, the Umbrella amendment will allow VA SHIELD to accept samples from other preexisting biorepositories or specimen collections.
Central Biorepositories
VA SHIELD has a federated organization with 2 central specimen biorepositories (Palo Alto, CA and Tucson, AZ), and an enterprise data processing center (Boston, MA). The specimen biorepositories receive de-identified specimens that are stored until distribution to approved research projects. The samples and data are linked using an electronic honest broker system to protect privacy, which integrates de-identified specimens with requested clinical and demographic data as needed for approved projects. The honest broker system is operated by independent personnel and does not have vested interest in any studies being performed under VA SHIELD. The integration of sample and associated data is done only as needed when characterization of the donor/participant is necessary byresearch aims or project outcomes. The process is facilitated by a nationally supported laboratory information management system (LIMS), managed by the VA SHIELD data center, that assists with all data requests. The clinical and demographic data are collected from VA electronic health record (EHR), available through VA Corporate Data Warehouse (CDW) and VA Informatics and Computing Infrastructure (VINCI) as needed and integrated with the biorepository samples information for approved VA SHIELD studies. The CDW is the largest longitudinal EHR data collection in the US and has the ability to provide access to national clinical and demographic data.
VA SHIELD interacts with multiple VA programs and other entities (Figure). For example, Surveillance Platform for Enteric and Respiratory Infectious Organisms at United States Veterans Affairs Medical Centers (SUPERNOVA) is a network of 5 VA medical centers supported by the Centers for Disease Control and Prevention.4 Its initial goal was to perform surveillance for acute gastroenteritis. In 2020, SUPERNOVA shifted to conduct surveillance for COVID-19 variants among veterans.5 VA SHIELD also interacts with VHA genomic surveillance and sequencing programs: the VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE) and VA Sequencing for Research Clinical and Epidemiology (SeqFORCE), described by Krishnan and colleagues.6
Working Groups
To encourage research projects that use biospecimens, VA SHIELD developed content-oriented research working groups. The goal is to inspire collaborations between VA scientists and prevent redundant or overlapping projects. Currently working groups are focused on long COVID, and COVID-19 neurology, pathogen host response, epidemiology and sequencing, cancer and cancer biomarkers, antimicrobial resistance, and vector-borne diseases. Working groups meet regularly to discuss projects and report progress. Working groups also may consider samples that might benefit VA health research and identify potential veteran populations for future research. Working groups connect VA SHIELD and investigators and guide the collection and use of resources.
Ethical Considerations
We recognize the significant ethical concerns for biobanking of specimens. However, there is no general consensus or guideline that addresses all of the complex ethical issues regarding biobanking.7 To address these ethical concerns, we applied the VA Ethical Framework Principles for Access to and Use of Veteran Data principles to VA SHIELD, including all parties who oversee the access to, sharing of, or the use of data, or who access or use its data.8
Conclusions
The VA has assembled a scientific enterprise dedicated to combating emerging infectious diseases and other threats to our patients. This enterprise has been modeled in its structure and oversight to support VA SHIELD. The establishment of a real-time biorepository and data procurement system linked to clinical samples is a bold step forward to address current and future challenges. Similarly, the integration and cooperation of multiple arms within the VA that transcend disciplines and boundaries promise to shepherd a new era of system-wide investigation. In the future, VA SHIELD will integrate with other existing government agencies to advance mutual scientific agendas. VA SHIELD has established the data and biorepository infrastructure to develop innovative and novel technologies to address future challenges. The alignment of basic science, clinical, and translational research goals under one governance is a significant advancement compared with previous models of research coordination.
VA SHIELD was developed to meet an immediate need; it was also framed to be a research enterprise that harnesses the robust clinical and research environment in VHA. The VA SHIELD infrastructure was conceptualized to harmonize specimen and data collection across the VA, allowing researchers to leverage broader collection efforts. Building a biorepository and data collection system within the largest integrated health care system has the potential to provide a lasting impact on VHA and on our nation’s health.
Acknowledgments
The authors wish to acknowledge Ms. Daphne Swancutt for her contribution as copywriter for this manuscript. The authors wish to acknowledge the VA SHIELD investigators: Mary Cloud Ammons, David Beenhouwer, Sheldon T. Brown, Victoria Davey, Abhinav Diwan, John B. Harley, Mark Holodniy, Vincent C. Marconi, Jonathan Moorman, Emerson B. Padiernos, Ian F. Robey, Maria Rodriguez-Barradas, Jason Wertheim, Christopher W. Woods.
1. Lipshy KA, Itani K, Chu D, et al. Sentinel contributions of US Department of Veterans Affairs surgeons in shaping the face of health care. JAMA Surg. 2021;156(4):380-386. doi:10.1001/jamasurg.2020.6372
2. Zucker S, Crabbe JC, Cooper G 4th, et al. Veterans Administration support for medical research: opinions of the endangered species of physician-scientists. FASEB J. 2004;18(13):1481-1486. doi:10.1096/fj.04-1573lfe
3. Harley JB, Pyarajan S, Partan ES, et al. The US Department of Veterans Affairs Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD): a biorepository addressing national health threats. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac641
4. Meites E, Bajema KL, Kambhampati A, et al; SUPERNOVA COVID-19 Surveillance Group. Adapting the Surveillance Platform for Enteric and Respiratory Infectious Organisms at United States Veterans Affairs Medical Centers (SUPERNOVA) for COVID-19 among hospitalized adults: surveillance protocol. Front Public Health. 2021;9:739076. doi:10.3389/fpubh.2021.739076
5. Bajema KL, Dahl RM, Evener SL, et al; SUPERNOVA COVID-19 Surveillance Group; Surveillance Platform for Enteric and Respiratory Infectious Organisms at the VA (SUPERNOVA) COVID-19 Surveillance Group. Comparative effectiveness and antibody responses to Moderna and Pfizer-BioNTech COVID-19 vaccines among hospitalized veterans–five Veterans Affairs Medical Centers, United States, February 1-September 30, 2021. MMWR Morb Mortal Wkly Rep. 2021;70(49):1700-1705. doi:10.15585/mmwr.mm7049a2external icon
6. Krishnan J, Woods C, Holodniy M, et al. Nationwide genomic surveillance and response to coronavirus disease 2019 (COVID-19): SeqCURE and SeqFORCE consortiums. Fed Pract. 2023;40(suppl 5):S44-S47. doi:10.12788/fp.0417
7. Ashcroft JW, Macpherson CC. The complex ethical landscape of biobanking. Lancet Public Health. 2019;(6):e274-e275. doi:10.1016/S2468-2667(19)30081-7
8. Principle-Based Ethics Framework for Access to and Use of Veteran Data. Fed Regist. 2022;87(129):40451-40452.
1. Lipshy KA, Itani K, Chu D, et al. Sentinel contributions of US Department of Veterans Affairs surgeons in shaping the face of health care. JAMA Surg. 2021;156(4):380-386. doi:10.1001/jamasurg.2020.6372
2. Zucker S, Crabbe JC, Cooper G 4th, et al. Veterans Administration support for medical research: opinions of the endangered species of physician-scientists. FASEB J. 2004;18(13):1481-1486. doi:10.1096/fj.04-1573lfe
3. Harley JB, Pyarajan S, Partan ES, et al. The US Department of Veterans Affairs Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD): a biorepository addressing national health threats. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac641
4. Meites E, Bajema KL, Kambhampati A, et al; SUPERNOVA COVID-19 Surveillance Group. Adapting the Surveillance Platform for Enteric and Respiratory Infectious Organisms at United States Veterans Affairs Medical Centers (SUPERNOVA) for COVID-19 among hospitalized adults: surveillance protocol. Front Public Health. 2021;9:739076. doi:10.3389/fpubh.2021.739076
5. Bajema KL, Dahl RM, Evener SL, et al; SUPERNOVA COVID-19 Surveillance Group; Surveillance Platform for Enteric and Respiratory Infectious Organisms at the VA (SUPERNOVA) COVID-19 Surveillance Group. Comparative effectiveness and antibody responses to Moderna and Pfizer-BioNTech COVID-19 vaccines among hospitalized veterans–five Veterans Affairs Medical Centers, United States, February 1-September 30, 2021. MMWR Morb Mortal Wkly Rep. 2021;70(49):1700-1705. doi:10.15585/mmwr.mm7049a2external icon
6. Krishnan J, Woods C, Holodniy M, et al. Nationwide genomic surveillance and response to coronavirus disease 2019 (COVID-19): SeqCURE and SeqFORCE consortiums. Fed Pract. 2023;40(suppl 5):S44-S47. doi:10.12788/fp.0417
7. Ashcroft JW, Macpherson CC. The complex ethical landscape of biobanking. Lancet Public Health. 2019;(6):e274-e275. doi:10.1016/S2468-2667(19)30081-7
8. Principle-Based Ethics Framework for Access to and Use of Veteran Data. Fed Regist. 2022;87(129):40451-40452.
Nationwide Genomic Surveillance and Response to COVID-19: The VA SeqFORCE and SeqCURE Consortiums
The COVID-19 virus and its associated pandemic have highlighted the urgent need for a national infrastructure to rapidly identify and respond to emerging pathogens. The importance of understanding viral population dynamics through genetic sequencing has become apparent over time, particularly as the vaccine responses, clinical implications, and therapeutic effectiveness of treatments have varied substantially with COVID-19 variants.1,2
As the largest integrated health care system in the US, the US Department of Veterans Affairs (VA) is uniquely situated to help with pandemic detection and response. This article highlights 2 VA programs dedicated to COVID-19 sequencing at the forefront of pandemic response and research: VA Sequencing for Research Clinical and Epidemiology (SeqFORCE) and VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE) (Table).
VA Seq FORCE
VA SeqFORCE was established March 2021 to facilitate clinical surveillance of COVID-19 variants in the US veteran population and in VA employees. VA SeqFORCE consists of 9 Clinical Laboratory Improvement Amendment (CLIA)–certified laboratories in VA medical centers, including the VA Public Health Reference Laboratory in Palo Alto, California, and 8 Veterans Health Administration (VHA) clinical laboratories (Los Angeles, California; Boise, Idaho; Iowa City, Iowa; Bronx, New York; West Haven, Connecticut; Indianapolis, Indiana; Denver, Colorado; and Orlando, Florida).3 Specimen standards (eg, real-time polymerase chain reaction [RT-PCR] cycle threshold [Ct] ≤ 30, minimum volume, etc) and clinical criteria (eg, COVID-19–related deaths, COVID-19 vaccine escape, etc) for submitting samples to VA SeqFORCE laboratories were established, and logistics for sample sequencing was centralized, including providing centralized instructions for sample preparation and to which VA SeqFORCE laboratory samples should be sent.
These laboratories sequenced samples from patients and employees with COVID-19 to understand patterns of variant evolution, vaccine, antiviral and monoclonal antibody response, health care–associated outbreaks, and COVID-19 transmission. As clinically relevant findings, such as monoclonal antibody treatment failure, emerged with novel viral variants, VA SeqFORCE was well positioned to rapidly detect the emergent variants and inform better clinical care of patients with COVID-19. Other clinical indications identified for sequencing within VA SeqFORCE included outbreak investigation, re-infection with COVID-19 > 90 days but < 6 months after a prior infection, extended hospitalization of > 21 days, death due to COVID-19, infection with a history of recent nondomestic travel, rebound of symptoms after improvement on oral antiviral therapy, and epidemiologic surveillance.
VA SeqFORCE laboratories use a variety of sequencing platforms, although a federated system was developed that electronically linked all laboratories using a software system (PraediGene, Bitscopic) for sample management, COVID-19 variant analytics, and automated result reporting of clade and lineage into the Veterans Health Information Systems and Technology Architecture (VistA) Computerized Patient Record System. In addition, generated nucleic acid sequence alignment through FASTA consensus sequence files have been archived for secondary research analyses. By archiving the consensus sequences, retrospective studies within the VA have the added benefit of being able to clinically annotate investigations into COVID-19 variant patterns. As of August 2023, 43,003 samples containing COVID-19 have been sequenced, and FASTA file and metadata upload are ongoing to the Global Initiative on Sharing Avian Influenza Data, which houses > 15 million COVID-19 files from global submissions.
VA SeqFORCE’s clinical sequencing efforts have created opportunities for multicenter collaboration in variant surveillance. In work from December 2021, investigators from the James J. Peters VA Medical Center in Bronx, New York, collaborated with the VHA Pathology and Laboratory Medicine Services and Public Health national program offices in Washington, DC, to develop an RT-PCR assay to rapidly differentiate Omicron from Delta variants.4 Samples from VA hospitals across the nation were used in this study.
Lessons from VA SeqFORCE have also been cited as inspiration to address COVID-19 clinical problems, including outbreak investigations in hospital settings and beyond. Researchers at the Iowa City VA Health Care System, for example, proposed a novel probabilistic quantitative method for determining genetic-relatedness among COVID-19 viral strains in an outbreak setting.5 They extended the scope of work to develop COVID-19 outbreak screening tools combining publicly available algorithms with targeted sequencing data to identify outbreaks as they arise.6 We expect VA SeqFORCE, in conjunction with its complement VA SeqCURE, will continue to further pandemic surveillance and response.
VA Seq CURE
As the research-focused complement to VA SeqFORCE, VA SeqCURE is dedicated to a broader study of the COVID-19 genome through sequencing. Established January 2021, the VA SeqCURE network consists of 6 research laboratories in Boise, Idaho; Bronx, New York; Cleveland, Ohio; Durham, North Carolina; Iowa City, Iowa; and Temple, Texas.
Samples are collected as a subset of the broader VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD) biorepository sweep protocol for discarded blood and nasal swab specimens of VHA patients hospitalized with COVID-19, as described by Epstein and colleagues.7-9 While VA SeqFORCE sequences samples positive for COVID-19 by RT-PCR with a Ct value of ≤ 30 for diagnostic purposes, VA SeqCURE laboratories sequence more broadly for nondiagnostic purposes, including samples with a Ct value > 30. The 6 VA SeqCURE laboratories generate sequencing data using various platforms, amplification kits, and formats. To ensure maximum quality and metadata on the sequences generated across the different laboratories, a sequence intake pipeline has been developed, adapting the ViralRecon bioinformatics platform.10 This harmonized analysis pipeline accommodates different file formats and performs quality control, alignment, variant calling, lineage assignment, clade assignment, and annotation. As of August 2023, VA SeqCURE has identified viral sequences from 24,107 unique specimens. Annotated COVID-19 sequences with the appropriate metadata will be available to VA researchers through VA SHIELD.
Research projects include descriptive epidemiology of COVID-19 variants in individuals who receive VHA care, COVID-19 vaccine and therapy effectiveness, and the unique distribution of variants and vaccine effectiveness in rural settings.3 True to its core mission, members of the VA SeqCURE consortium have contributed to the COVID-19 viral sequencing literature over the past 2 years. Researchers also are accessing VA SeqCURE to study COVID-19 persistence and rebound among individuals with mild disease taking nirmatrelvir/ritonavir compared with other COVID-19 therapeutics and untreated controls. Finally, COVID-19 samples and their sequences are stored in the VA SHIELD biorepository, which leverages these samples and data to advance scientific understanding of COVID-19 and future emerging infectious diseases.7-9
Important work from investigators at the Central Texas Veterans Health Care System confronted the issue of whole genome sequencing data from COVID-19 samples with low viral loads, a common issue with COVID-19 sequencing. They found that yields of 2 sequencing protocols, which generated high-sequence coverage, were enhanced further by combining the results of both methods.11 This project, which has potentially broad applications for sequencing in research and clinical settings, is an example of VA SeqCURE’s efforts to address the COVID-19 pandemic. The VA SeqCURE program has substantial potential as a large viral sequencing repository with broad geographic and demographic representation, such that future large-scale sequencing analyses may be generated from preexisting nested cohorts within the repository.
NEXT STEPS
Promising new directions of clinical and laboratory-based research are planned for VA SeqFORCE and VA SeqCURE. While the impact of COVID-19 and other viruses with epidemic potential is perhaps most feared in urban settings, evidence suggests that the distribution of COVID-19 in rural settings is unique and associated with worse outcomes.12,13 Given the wide catchment areas of VA hospitals that encompass both rural and urban settings, the VA’s ongoing COVID-19 sequencing programs and repositories are uniquely positioned to understand viral dynamics in areas of differing population density.
While rates of infection, hospitalization, and death resulting from COVID-19 have substantially dropped, the long-term impact of the pandemic is just beginning to be recognized in conditions such as long COVID or postacute COVID-19 syndrome. Long COVID has already proven to be biologically multifaceted, difficult to diagnose, and unpredictable in identifying the most at-risk patients.14-16 Much remains to be determined in our understanding of long COVID, including a unified definition that can effectively be used in clinical settings to diagnose and treat patients. However, research indicates that comorbidities common in veterans, such as diabetes and cardiovascular disease, are associated with worse long-term outcomes.17,18 Collaborations between VA scientists, clinicians, and national cooperative programs (such as a network of VHA long COVID clinics) create an unmatched opportunity for VA SeqFORCE and VA SeqCURE programs to provide insight into a disease likely to become a chronic disease outcome of the pandemic.
With VA SeqFORCE and VA SeqCURE programs, the VA now has infrastructure ready to respond to new infectious diseases. During the mpox outbreak of 2022, the VA Public Health Reference Laboratory received > 80% of all VA mpox samples for orthopox screening and mpox confirmatory testing. A subset of these samples underwent whole genome sequencing with the identification of 10 unique lineages across VA, and > 200 positive and 400 negative samples have been aliquoted and submitted to VA SHIELD for research. Furthermore, the VA SeqFORCE and VA SeqCURE sequencing processes might be adapted to identify outbreaks of multidrug-resistant organisms among VA patients trialed at other institutions.19 We are hopeful that VA SeqFORCE and VA SeqCURE will become invaluable components of health care delivery and infection prevention at the hospital level and beyond.
Finally, the robust data infrastructure and associated repositories of VA SeqFORCE and VA SeqCURE may be leveraged to study noninfectious diseases. Research groups are starting to apply these programs to cancer sequencing. We anticipate that these efforts may have a substantial impact on our understanding of cancer epidemiology and region-specific risk factors for malignancy, given the size and breadth of VA SeqFORCE and VA SeqCURE. Common oncogenic mutations identified through these programs could be targets for precision oncology therapeutics. Similarly, we envision applications of the VA SeqFORCE and VA SeqCURE data infrastructures and repositories toward other precision medicine fields, including pharmacogenomics and nutrition, to tailor interventions to meet the specific individual needs of veterans.
CONCLUSIONS
The productivity of VA SeqFORCE and VA SeqCURE programs over the past 2 years continues to increase in response to the COVID-19 pandemic. We anticipate that they will be vital components in our nation’s responses to infectious threats and beyond.
1. Iuliano AD, Brunkard JM, Boehmer TK, et al. Trends in disease severity and health care utilization during the early Omicron variant period compared with previous SARS-CoV-2 high transmission periods - United States, December 2020-January 2022. MMWR Morb Mortal Wkly Rep. 2022;71(4):146-152. Published 2022 Jan 28. doi:10.15585/mmwr.mm7104e4
2. Nyberg T, Ferguson NM, Nash SG, et al. Comparative analysis of the risks of hospitalisation and death associated with SARS-CoV-2 omicron (B.1.1.529) and delta (B.1.617.2) variants in England: a cohort study. Lancet. 2022;399(10332):1303-1312. doi:10.1016/S0140-6736(22)00462-7
3. Veterans Health Administration. Coronavirus Disease 2019 (COVID-19) response report - annex C. December 5, 2022. Accessed August 28, 2023. https://www.va.gov/HEALTH/docs/VHA-COVID-19-Response-2022-Annex-C.pdf 4. Barasch NJ, Iqbal J, Coombs M, et al. Utilization of a SARS-CoV-2 variant assay for the rapid differentiation of Omicron and Delta. medRxiv. Preprint posted online December 27, 2021. doi:10.1101/2021.12.22.21268195
5. Bilal MY. Similarity Index-probabilistic confidence estimation of SARS-CoV-2 strain relatedness in localized outbreaks. Epidemiologia (Basel). 2022;3(2):238-249. doi:10.3390/epidemiologia3020019
6. Bilal MY, Klutts JS. Molecular Epidemiological investigations of localized SARS-CoV-2 outbreaks-utility of public algorithms. Epidemiologia (Basel). 2022;3(3):402-411. doi:10.3390/epidemiologia3030031
7. Veterans Health Administration, Office of Research & Development. VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD). Updated November 23, 2022. Accessed August 28, 2023. https://www.research.va.gov/programs/shield/about.cfm
8. Harley JB, Pyarajan S, Partan ES, et al. The US Department of Veterans Affairs Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD): a biorepository addressing national health threats. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac641
9. Epstein L, Shive C, Garcia AP, et al. VA SHIELD: a biorepository for our veterans and the nation. Fed Pract. 2023;40(suppl 5):S48-S51. doi:10.12788/fp.0424
10. Patel H, Varona S, Monzón S, et al. Version 2.5. nf-core/viralrecon: nf-core/viralrecon v2.5 - Manganese Monkey (2.5). Zenodo. July 13, 2022. doi:10.5281/zenodo.6827984
11. Choi H, Hwang M, Navarathna DH, Xu J, Lukey J, Jinadatha C. Performance of COVIDSeq and swift normalase amplicon SARS-CoV-2 panels for SARS-CoV-2 genome sequencing: practical guide and combining FASTQ strategy. J Clin Microbiol. 2022;60(4):e0002522. doi:10.1128/jcm.00025-22
12. Cuadros DF, Branscum AJ, Mukandavire Z, Miller FD, MacKinnon N. Dynamics of the COVID-19 epidemic in urban and rural areas in the United States. Ann Epidemiol. 2021;59:16-20. doi:10.1016/j.annepidem.2021.04.007
13. Anzalone AJ, Horswell R, Hendricks BM, et al. Higher hospitalization and mortality rates among SARS-CoV-2-infected persons in rural America. J Rural Health. 2023;39(1):39-54. doi:10.1111/jrh.12689
14. Su Y, Yuan D, Chen DG, et al. Multiple early factors anticipate post-acute COVID-19 sequelae. Cell. 2022;185(5):881-895.e20. doi:10.1016/j.cell.2022.01.014
15. Pfaff ER, Girvin AT, Bennett TD, et al. Identifying who has long COVID in the USA: a machine learning approach using N3C data. Lancet Digit Health. 2022;4(7):e532-e541. doi:10.1016/S2589-7500(22)00048-6
16. Subramanian A, Nirantharakumar K, Hughes S, et al. Symptoms and risk factors for long COVID in non-hospitalized adults. Nat Med. 2022;28(8):1706-1714. doi:10.1038/s41591-022-01909-w
17. Munblit D, O’Hara ME, Akrami A, Perego E, Olliaro P, Needham DM. Long COVID: aiming for a consensus. Lancet Respir Med. 2022;10(7):632-634. doi:10.1016/S2213-2600(22)00135-7
18. Thaweethai T, Jolley SE, Karlson EW, et al. Development of a definition of postacute sequelae of SARS-CoV-2 infection. JAMA. 2023;329(22):1934-1946. doi:10.1001/jama.2023.8823
19. Sundermann AJ, Chen J, Kumar P, et al. Whole-genome sequencing surveillance and machine learning of the electronic health record for enhanced healthcare outbreak detection. Clin Infect Dis. 2022;75(3):476-482. doi:10.1093/cid/ciab946
The COVID-19 virus and its associated pandemic have highlighted the urgent need for a national infrastructure to rapidly identify and respond to emerging pathogens. The importance of understanding viral population dynamics through genetic sequencing has become apparent over time, particularly as the vaccine responses, clinical implications, and therapeutic effectiveness of treatments have varied substantially with COVID-19 variants.1,2
As the largest integrated health care system in the US, the US Department of Veterans Affairs (VA) is uniquely situated to help with pandemic detection and response. This article highlights 2 VA programs dedicated to COVID-19 sequencing at the forefront of pandemic response and research: VA Sequencing for Research Clinical and Epidemiology (SeqFORCE) and VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE) (Table).
VA Seq FORCE
VA SeqFORCE was established March 2021 to facilitate clinical surveillance of COVID-19 variants in the US veteran population and in VA employees. VA SeqFORCE consists of 9 Clinical Laboratory Improvement Amendment (CLIA)–certified laboratories in VA medical centers, including the VA Public Health Reference Laboratory in Palo Alto, California, and 8 Veterans Health Administration (VHA) clinical laboratories (Los Angeles, California; Boise, Idaho; Iowa City, Iowa; Bronx, New York; West Haven, Connecticut; Indianapolis, Indiana; Denver, Colorado; and Orlando, Florida).3 Specimen standards (eg, real-time polymerase chain reaction [RT-PCR] cycle threshold [Ct] ≤ 30, minimum volume, etc) and clinical criteria (eg, COVID-19–related deaths, COVID-19 vaccine escape, etc) for submitting samples to VA SeqFORCE laboratories were established, and logistics for sample sequencing was centralized, including providing centralized instructions for sample preparation and to which VA SeqFORCE laboratory samples should be sent.
These laboratories sequenced samples from patients and employees with COVID-19 to understand patterns of variant evolution, vaccine, antiviral and monoclonal antibody response, health care–associated outbreaks, and COVID-19 transmission. As clinically relevant findings, such as monoclonal antibody treatment failure, emerged with novel viral variants, VA SeqFORCE was well positioned to rapidly detect the emergent variants and inform better clinical care of patients with COVID-19. Other clinical indications identified for sequencing within VA SeqFORCE included outbreak investigation, re-infection with COVID-19 > 90 days but < 6 months after a prior infection, extended hospitalization of > 21 days, death due to COVID-19, infection with a history of recent nondomestic travel, rebound of symptoms after improvement on oral antiviral therapy, and epidemiologic surveillance.
VA SeqFORCE laboratories use a variety of sequencing platforms, although a federated system was developed that electronically linked all laboratories using a software system (PraediGene, Bitscopic) for sample management, COVID-19 variant analytics, and automated result reporting of clade and lineage into the Veterans Health Information Systems and Technology Architecture (VistA) Computerized Patient Record System. In addition, generated nucleic acid sequence alignment through FASTA consensus sequence files have been archived for secondary research analyses. By archiving the consensus sequences, retrospective studies within the VA have the added benefit of being able to clinically annotate investigations into COVID-19 variant patterns. As of August 2023, 43,003 samples containing COVID-19 have been sequenced, and FASTA file and metadata upload are ongoing to the Global Initiative on Sharing Avian Influenza Data, which houses > 15 million COVID-19 files from global submissions.
VA SeqFORCE’s clinical sequencing efforts have created opportunities for multicenter collaboration in variant surveillance. In work from December 2021, investigators from the James J. Peters VA Medical Center in Bronx, New York, collaborated with the VHA Pathology and Laboratory Medicine Services and Public Health national program offices in Washington, DC, to develop an RT-PCR assay to rapidly differentiate Omicron from Delta variants.4 Samples from VA hospitals across the nation were used in this study.
Lessons from VA SeqFORCE have also been cited as inspiration to address COVID-19 clinical problems, including outbreak investigations in hospital settings and beyond. Researchers at the Iowa City VA Health Care System, for example, proposed a novel probabilistic quantitative method for determining genetic-relatedness among COVID-19 viral strains in an outbreak setting.5 They extended the scope of work to develop COVID-19 outbreak screening tools combining publicly available algorithms with targeted sequencing data to identify outbreaks as they arise.6 We expect VA SeqFORCE, in conjunction with its complement VA SeqCURE, will continue to further pandemic surveillance and response.
VA Seq CURE
As the research-focused complement to VA SeqFORCE, VA SeqCURE is dedicated to a broader study of the COVID-19 genome through sequencing. Established January 2021, the VA SeqCURE network consists of 6 research laboratories in Boise, Idaho; Bronx, New York; Cleveland, Ohio; Durham, North Carolina; Iowa City, Iowa; and Temple, Texas.
Samples are collected as a subset of the broader VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD) biorepository sweep protocol for discarded blood and nasal swab specimens of VHA patients hospitalized with COVID-19, as described by Epstein and colleagues.7-9 While VA SeqFORCE sequences samples positive for COVID-19 by RT-PCR with a Ct value of ≤ 30 for diagnostic purposes, VA SeqCURE laboratories sequence more broadly for nondiagnostic purposes, including samples with a Ct value > 30. The 6 VA SeqCURE laboratories generate sequencing data using various platforms, amplification kits, and formats. To ensure maximum quality and metadata on the sequences generated across the different laboratories, a sequence intake pipeline has been developed, adapting the ViralRecon bioinformatics platform.10 This harmonized analysis pipeline accommodates different file formats and performs quality control, alignment, variant calling, lineage assignment, clade assignment, and annotation. As of August 2023, VA SeqCURE has identified viral sequences from 24,107 unique specimens. Annotated COVID-19 sequences with the appropriate metadata will be available to VA researchers through VA SHIELD.
Research projects include descriptive epidemiology of COVID-19 variants in individuals who receive VHA care, COVID-19 vaccine and therapy effectiveness, and the unique distribution of variants and vaccine effectiveness in rural settings.3 True to its core mission, members of the VA SeqCURE consortium have contributed to the COVID-19 viral sequencing literature over the past 2 years. Researchers also are accessing VA SeqCURE to study COVID-19 persistence and rebound among individuals with mild disease taking nirmatrelvir/ritonavir compared with other COVID-19 therapeutics and untreated controls. Finally, COVID-19 samples and their sequences are stored in the VA SHIELD biorepository, which leverages these samples and data to advance scientific understanding of COVID-19 and future emerging infectious diseases.7-9
Important work from investigators at the Central Texas Veterans Health Care System confronted the issue of whole genome sequencing data from COVID-19 samples with low viral loads, a common issue with COVID-19 sequencing. They found that yields of 2 sequencing protocols, which generated high-sequence coverage, were enhanced further by combining the results of both methods.11 This project, which has potentially broad applications for sequencing in research and clinical settings, is an example of VA SeqCURE’s efforts to address the COVID-19 pandemic. The VA SeqCURE program has substantial potential as a large viral sequencing repository with broad geographic and demographic representation, such that future large-scale sequencing analyses may be generated from preexisting nested cohorts within the repository.
NEXT STEPS
Promising new directions of clinical and laboratory-based research are planned for VA SeqFORCE and VA SeqCURE. While the impact of COVID-19 and other viruses with epidemic potential is perhaps most feared in urban settings, evidence suggests that the distribution of COVID-19 in rural settings is unique and associated with worse outcomes.12,13 Given the wide catchment areas of VA hospitals that encompass both rural and urban settings, the VA’s ongoing COVID-19 sequencing programs and repositories are uniquely positioned to understand viral dynamics in areas of differing population density.
While rates of infection, hospitalization, and death resulting from COVID-19 have substantially dropped, the long-term impact of the pandemic is just beginning to be recognized in conditions such as long COVID or postacute COVID-19 syndrome. Long COVID has already proven to be biologically multifaceted, difficult to diagnose, and unpredictable in identifying the most at-risk patients.14-16 Much remains to be determined in our understanding of long COVID, including a unified definition that can effectively be used in clinical settings to diagnose and treat patients. However, research indicates that comorbidities common in veterans, such as diabetes and cardiovascular disease, are associated with worse long-term outcomes.17,18 Collaborations between VA scientists, clinicians, and national cooperative programs (such as a network of VHA long COVID clinics) create an unmatched opportunity for VA SeqFORCE and VA SeqCURE programs to provide insight into a disease likely to become a chronic disease outcome of the pandemic.
With VA SeqFORCE and VA SeqCURE programs, the VA now has infrastructure ready to respond to new infectious diseases. During the mpox outbreak of 2022, the VA Public Health Reference Laboratory received > 80% of all VA mpox samples for orthopox screening and mpox confirmatory testing. A subset of these samples underwent whole genome sequencing with the identification of 10 unique lineages across VA, and > 200 positive and 400 negative samples have been aliquoted and submitted to VA SHIELD for research. Furthermore, the VA SeqFORCE and VA SeqCURE sequencing processes might be adapted to identify outbreaks of multidrug-resistant organisms among VA patients trialed at other institutions.19 We are hopeful that VA SeqFORCE and VA SeqCURE will become invaluable components of health care delivery and infection prevention at the hospital level and beyond.
Finally, the robust data infrastructure and associated repositories of VA SeqFORCE and VA SeqCURE may be leveraged to study noninfectious diseases. Research groups are starting to apply these programs to cancer sequencing. We anticipate that these efforts may have a substantial impact on our understanding of cancer epidemiology and region-specific risk factors for malignancy, given the size and breadth of VA SeqFORCE and VA SeqCURE. Common oncogenic mutations identified through these programs could be targets for precision oncology therapeutics. Similarly, we envision applications of the VA SeqFORCE and VA SeqCURE data infrastructures and repositories toward other precision medicine fields, including pharmacogenomics and nutrition, to tailor interventions to meet the specific individual needs of veterans.
CONCLUSIONS
The productivity of VA SeqFORCE and VA SeqCURE programs over the past 2 years continues to increase in response to the COVID-19 pandemic. We anticipate that they will be vital components in our nation’s responses to infectious threats and beyond.
The COVID-19 virus and its associated pandemic have highlighted the urgent need for a national infrastructure to rapidly identify and respond to emerging pathogens. The importance of understanding viral population dynamics through genetic sequencing has become apparent over time, particularly as the vaccine responses, clinical implications, and therapeutic effectiveness of treatments have varied substantially with COVID-19 variants.1,2
As the largest integrated health care system in the US, the US Department of Veterans Affairs (VA) is uniquely situated to help with pandemic detection and response. This article highlights 2 VA programs dedicated to COVID-19 sequencing at the forefront of pandemic response and research: VA Sequencing for Research Clinical and Epidemiology (SeqFORCE) and VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE) (Table).
VA Seq FORCE
VA SeqFORCE was established March 2021 to facilitate clinical surveillance of COVID-19 variants in the US veteran population and in VA employees. VA SeqFORCE consists of 9 Clinical Laboratory Improvement Amendment (CLIA)–certified laboratories in VA medical centers, including the VA Public Health Reference Laboratory in Palo Alto, California, and 8 Veterans Health Administration (VHA) clinical laboratories (Los Angeles, California; Boise, Idaho; Iowa City, Iowa; Bronx, New York; West Haven, Connecticut; Indianapolis, Indiana; Denver, Colorado; and Orlando, Florida).3 Specimen standards (eg, real-time polymerase chain reaction [RT-PCR] cycle threshold [Ct] ≤ 30, minimum volume, etc) and clinical criteria (eg, COVID-19–related deaths, COVID-19 vaccine escape, etc) for submitting samples to VA SeqFORCE laboratories were established, and logistics for sample sequencing was centralized, including providing centralized instructions for sample preparation and to which VA SeqFORCE laboratory samples should be sent.
These laboratories sequenced samples from patients and employees with COVID-19 to understand patterns of variant evolution, vaccine, antiviral and monoclonal antibody response, health care–associated outbreaks, and COVID-19 transmission. As clinically relevant findings, such as monoclonal antibody treatment failure, emerged with novel viral variants, VA SeqFORCE was well positioned to rapidly detect the emergent variants and inform better clinical care of patients with COVID-19. Other clinical indications identified for sequencing within VA SeqFORCE included outbreak investigation, re-infection with COVID-19 > 90 days but < 6 months after a prior infection, extended hospitalization of > 21 days, death due to COVID-19, infection with a history of recent nondomestic travel, rebound of symptoms after improvement on oral antiviral therapy, and epidemiologic surveillance.
VA SeqFORCE laboratories use a variety of sequencing platforms, although a federated system was developed that electronically linked all laboratories using a software system (PraediGene, Bitscopic) for sample management, COVID-19 variant analytics, and automated result reporting of clade and lineage into the Veterans Health Information Systems and Technology Architecture (VistA) Computerized Patient Record System. In addition, generated nucleic acid sequence alignment through FASTA consensus sequence files have been archived for secondary research analyses. By archiving the consensus sequences, retrospective studies within the VA have the added benefit of being able to clinically annotate investigations into COVID-19 variant patterns. As of August 2023, 43,003 samples containing COVID-19 have been sequenced, and FASTA file and metadata upload are ongoing to the Global Initiative on Sharing Avian Influenza Data, which houses > 15 million COVID-19 files from global submissions.
VA SeqFORCE’s clinical sequencing efforts have created opportunities for multicenter collaboration in variant surveillance. In work from December 2021, investigators from the James J. Peters VA Medical Center in Bronx, New York, collaborated with the VHA Pathology and Laboratory Medicine Services and Public Health national program offices in Washington, DC, to develop an RT-PCR assay to rapidly differentiate Omicron from Delta variants.4 Samples from VA hospitals across the nation were used in this study.
Lessons from VA SeqFORCE have also been cited as inspiration to address COVID-19 clinical problems, including outbreak investigations in hospital settings and beyond. Researchers at the Iowa City VA Health Care System, for example, proposed a novel probabilistic quantitative method for determining genetic-relatedness among COVID-19 viral strains in an outbreak setting.5 They extended the scope of work to develop COVID-19 outbreak screening tools combining publicly available algorithms with targeted sequencing data to identify outbreaks as they arise.6 We expect VA SeqFORCE, in conjunction with its complement VA SeqCURE, will continue to further pandemic surveillance and response.
VA Seq CURE
As the research-focused complement to VA SeqFORCE, VA SeqCURE is dedicated to a broader study of the COVID-19 genome through sequencing. Established January 2021, the VA SeqCURE network consists of 6 research laboratories in Boise, Idaho; Bronx, New York; Cleveland, Ohio; Durham, North Carolina; Iowa City, Iowa; and Temple, Texas.
Samples are collected as a subset of the broader VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD) biorepository sweep protocol for discarded blood and nasal swab specimens of VHA patients hospitalized with COVID-19, as described by Epstein and colleagues.7-9 While VA SeqFORCE sequences samples positive for COVID-19 by RT-PCR with a Ct value of ≤ 30 for diagnostic purposes, VA SeqCURE laboratories sequence more broadly for nondiagnostic purposes, including samples with a Ct value > 30. The 6 VA SeqCURE laboratories generate sequencing data using various platforms, amplification kits, and formats. To ensure maximum quality and metadata on the sequences generated across the different laboratories, a sequence intake pipeline has been developed, adapting the ViralRecon bioinformatics platform.10 This harmonized analysis pipeline accommodates different file formats and performs quality control, alignment, variant calling, lineage assignment, clade assignment, and annotation. As of August 2023, VA SeqCURE has identified viral sequences from 24,107 unique specimens. Annotated COVID-19 sequences with the appropriate metadata will be available to VA researchers through VA SHIELD.
Research projects include descriptive epidemiology of COVID-19 variants in individuals who receive VHA care, COVID-19 vaccine and therapy effectiveness, and the unique distribution of variants and vaccine effectiveness in rural settings.3 True to its core mission, members of the VA SeqCURE consortium have contributed to the COVID-19 viral sequencing literature over the past 2 years. Researchers also are accessing VA SeqCURE to study COVID-19 persistence and rebound among individuals with mild disease taking nirmatrelvir/ritonavir compared with other COVID-19 therapeutics and untreated controls. Finally, COVID-19 samples and their sequences are stored in the VA SHIELD biorepository, which leverages these samples and data to advance scientific understanding of COVID-19 and future emerging infectious diseases.7-9
Important work from investigators at the Central Texas Veterans Health Care System confronted the issue of whole genome sequencing data from COVID-19 samples with low viral loads, a common issue with COVID-19 sequencing. They found that yields of 2 sequencing protocols, which generated high-sequence coverage, were enhanced further by combining the results of both methods.11 This project, which has potentially broad applications for sequencing in research and clinical settings, is an example of VA SeqCURE’s efforts to address the COVID-19 pandemic. The VA SeqCURE program has substantial potential as a large viral sequencing repository with broad geographic and demographic representation, such that future large-scale sequencing analyses may be generated from preexisting nested cohorts within the repository.
NEXT STEPS
Promising new directions of clinical and laboratory-based research are planned for VA SeqFORCE and VA SeqCURE. While the impact of COVID-19 and other viruses with epidemic potential is perhaps most feared in urban settings, evidence suggests that the distribution of COVID-19 in rural settings is unique and associated with worse outcomes.12,13 Given the wide catchment areas of VA hospitals that encompass both rural and urban settings, the VA’s ongoing COVID-19 sequencing programs and repositories are uniquely positioned to understand viral dynamics in areas of differing population density.
While rates of infection, hospitalization, and death resulting from COVID-19 have substantially dropped, the long-term impact of the pandemic is just beginning to be recognized in conditions such as long COVID or postacute COVID-19 syndrome. Long COVID has already proven to be biologically multifaceted, difficult to diagnose, and unpredictable in identifying the most at-risk patients.14-16 Much remains to be determined in our understanding of long COVID, including a unified definition that can effectively be used in clinical settings to diagnose and treat patients. However, research indicates that comorbidities common in veterans, such as diabetes and cardiovascular disease, are associated with worse long-term outcomes.17,18 Collaborations between VA scientists, clinicians, and national cooperative programs (such as a network of VHA long COVID clinics) create an unmatched opportunity for VA SeqFORCE and VA SeqCURE programs to provide insight into a disease likely to become a chronic disease outcome of the pandemic.
With VA SeqFORCE and VA SeqCURE programs, the VA now has infrastructure ready to respond to new infectious diseases. During the mpox outbreak of 2022, the VA Public Health Reference Laboratory received > 80% of all VA mpox samples for orthopox screening and mpox confirmatory testing. A subset of these samples underwent whole genome sequencing with the identification of 10 unique lineages across VA, and > 200 positive and 400 negative samples have been aliquoted and submitted to VA SHIELD for research. Furthermore, the VA SeqFORCE and VA SeqCURE sequencing processes might be adapted to identify outbreaks of multidrug-resistant organisms among VA patients trialed at other institutions.19 We are hopeful that VA SeqFORCE and VA SeqCURE will become invaluable components of health care delivery and infection prevention at the hospital level and beyond.
Finally, the robust data infrastructure and associated repositories of VA SeqFORCE and VA SeqCURE may be leveraged to study noninfectious diseases. Research groups are starting to apply these programs to cancer sequencing. We anticipate that these efforts may have a substantial impact on our understanding of cancer epidemiology and region-specific risk factors for malignancy, given the size and breadth of VA SeqFORCE and VA SeqCURE. Common oncogenic mutations identified through these programs could be targets for precision oncology therapeutics. Similarly, we envision applications of the VA SeqFORCE and VA SeqCURE data infrastructures and repositories toward other precision medicine fields, including pharmacogenomics and nutrition, to tailor interventions to meet the specific individual needs of veterans.
CONCLUSIONS
The productivity of VA SeqFORCE and VA SeqCURE programs over the past 2 years continues to increase in response to the COVID-19 pandemic. We anticipate that they will be vital components in our nation’s responses to infectious threats and beyond.
1. Iuliano AD, Brunkard JM, Boehmer TK, et al. Trends in disease severity and health care utilization during the early Omicron variant period compared with previous SARS-CoV-2 high transmission periods - United States, December 2020-January 2022. MMWR Morb Mortal Wkly Rep. 2022;71(4):146-152. Published 2022 Jan 28. doi:10.15585/mmwr.mm7104e4
2. Nyberg T, Ferguson NM, Nash SG, et al. Comparative analysis of the risks of hospitalisation and death associated with SARS-CoV-2 omicron (B.1.1.529) and delta (B.1.617.2) variants in England: a cohort study. Lancet. 2022;399(10332):1303-1312. doi:10.1016/S0140-6736(22)00462-7
3. Veterans Health Administration. Coronavirus Disease 2019 (COVID-19) response report - annex C. December 5, 2022. Accessed August 28, 2023. https://www.va.gov/HEALTH/docs/VHA-COVID-19-Response-2022-Annex-C.pdf 4. Barasch NJ, Iqbal J, Coombs M, et al. Utilization of a SARS-CoV-2 variant assay for the rapid differentiation of Omicron and Delta. medRxiv. Preprint posted online December 27, 2021. doi:10.1101/2021.12.22.21268195
5. Bilal MY. Similarity Index-probabilistic confidence estimation of SARS-CoV-2 strain relatedness in localized outbreaks. Epidemiologia (Basel). 2022;3(2):238-249. doi:10.3390/epidemiologia3020019
6. Bilal MY, Klutts JS. Molecular Epidemiological investigations of localized SARS-CoV-2 outbreaks-utility of public algorithms. Epidemiologia (Basel). 2022;3(3):402-411. doi:10.3390/epidemiologia3030031
7. Veterans Health Administration, Office of Research & Development. VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD). Updated November 23, 2022. Accessed August 28, 2023. https://www.research.va.gov/programs/shield/about.cfm
8. Harley JB, Pyarajan S, Partan ES, et al. The US Department of Veterans Affairs Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD): a biorepository addressing national health threats. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac641
9. Epstein L, Shive C, Garcia AP, et al. VA SHIELD: a biorepository for our veterans and the nation. Fed Pract. 2023;40(suppl 5):S48-S51. doi:10.12788/fp.0424
10. Patel H, Varona S, Monzón S, et al. Version 2.5. nf-core/viralrecon: nf-core/viralrecon v2.5 - Manganese Monkey (2.5). Zenodo. July 13, 2022. doi:10.5281/zenodo.6827984
11. Choi H, Hwang M, Navarathna DH, Xu J, Lukey J, Jinadatha C. Performance of COVIDSeq and swift normalase amplicon SARS-CoV-2 panels for SARS-CoV-2 genome sequencing: practical guide and combining FASTQ strategy. J Clin Microbiol. 2022;60(4):e0002522. doi:10.1128/jcm.00025-22
12. Cuadros DF, Branscum AJ, Mukandavire Z, Miller FD, MacKinnon N. Dynamics of the COVID-19 epidemic in urban and rural areas in the United States. Ann Epidemiol. 2021;59:16-20. doi:10.1016/j.annepidem.2021.04.007
13. Anzalone AJ, Horswell R, Hendricks BM, et al. Higher hospitalization and mortality rates among SARS-CoV-2-infected persons in rural America. J Rural Health. 2023;39(1):39-54. doi:10.1111/jrh.12689
14. Su Y, Yuan D, Chen DG, et al. Multiple early factors anticipate post-acute COVID-19 sequelae. Cell. 2022;185(5):881-895.e20. doi:10.1016/j.cell.2022.01.014
15. Pfaff ER, Girvin AT, Bennett TD, et al. Identifying who has long COVID in the USA: a machine learning approach using N3C data. Lancet Digit Health. 2022;4(7):e532-e541. doi:10.1016/S2589-7500(22)00048-6
16. Subramanian A, Nirantharakumar K, Hughes S, et al. Symptoms and risk factors for long COVID in non-hospitalized adults. Nat Med. 2022;28(8):1706-1714. doi:10.1038/s41591-022-01909-w
17. Munblit D, O’Hara ME, Akrami A, Perego E, Olliaro P, Needham DM. Long COVID: aiming for a consensus. Lancet Respir Med. 2022;10(7):632-634. doi:10.1016/S2213-2600(22)00135-7
18. Thaweethai T, Jolley SE, Karlson EW, et al. Development of a definition of postacute sequelae of SARS-CoV-2 infection. JAMA. 2023;329(22):1934-1946. doi:10.1001/jama.2023.8823
19. Sundermann AJ, Chen J, Kumar P, et al. Whole-genome sequencing surveillance and machine learning of the electronic health record for enhanced healthcare outbreak detection. Clin Infect Dis. 2022;75(3):476-482. doi:10.1093/cid/ciab946
1. Iuliano AD, Brunkard JM, Boehmer TK, et al. Trends in disease severity and health care utilization during the early Omicron variant period compared with previous SARS-CoV-2 high transmission periods - United States, December 2020-January 2022. MMWR Morb Mortal Wkly Rep. 2022;71(4):146-152. Published 2022 Jan 28. doi:10.15585/mmwr.mm7104e4
2. Nyberg T, Ferguson NM, Nash SG, et al. Comparative analysis of the risks of hospitalisation and death associated with SARS-CoV-2 omicron (B.1.1.529) and delta (B.1.617.2) variants in England: a cohort study. Lancet. 2022;399(10332):1303-1312. doi:10.1016/S0140-6736(22)00462-7
3. Veterans Health Administration. Coronavirus Disease 2019 (COVID-19) response report - annex C. December 5, 2022. Accessed August 28, 2023. https://www.va.gov/HEALTH/docs/VHA-COVID-19-Response-2022-Annex-C.pdf 4. Barasch NJ, Iqbal J, Coombs M, et al. Utilization of a SARS-CoV-2 variant assay for the rapid differentiation of Omicron and Delta. medRxiv. Preprint posted online December 27, 2021. doi:10.1101/2021.12.22.21268195
5. Bilal MY. Similarity Index-probabilistic confidence estimation of SARS-CoV-2 strain relatedness in localized outbreaks. Epidemiologia (Basel). 2022;3(2):238-249. doi:10.3390/epidemiologia3020019
6. Bilal MY, Klutts JS. Molecular Epidemiological investigations of localized SARS-CoV-2 outbreaks-utility of public algorithms. Epidemiologia (Basel). 2022;3(3):402-411. doi:10.3390/epidemiologia3030031
7. Veterans Health Administration, Office of Research & Development. VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD). Updated November 23, 2022. Accessed August 28, 2023. https://www.research.va.gov/programs/shield/about.cfm
8. Harley JB, Pyarajan S, Partan ES, et al. The US Department of Veterans Affairs Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD): a biorepository addressing national health threats. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac641
9. Epstein L, Shive C, Garcia AP, et al. VA SHIELD: a biorepository for our veterans and the nation. Fed Pract. 2023;40(suppl 5):S48-S51. doi:10.12788/fp.0424
10. Patel H, Varona S, Monzón S, et al. Version 2.5. nf-core/viralrecon: nf-core/viralrecon v2.5 - Manganese Monkey (2.5). Zenodo. July 13, 2022. doi:10.5281/zenodo.6827984
11. Choi H, Hwang M, Navarathna DH, Xu J, Lukey J, Jinadatha C. Performance of COVIDSeq and swift normalase amplicon SARS-CoV-2 panels for SARS-CoV-2 genome sequencing: practical guide and combining FASTQ strategy. J Clin Microbiol. 2022;60(4):e0002522. doi:10.1128/jcm.00025-22
12. Cuadros DF, Branscum AJ, Mukandavire Z, Miller FD, MacKinnon N. Dynamics of the COVID-19 epidemic in urban and rural areas in the United States. Ann Epidemiol. 2021;59:16-20. doi:10.1016/j.annepidem.2021.04.007
13. Anzalone AJ, Horswell R, Hendricks BM, et al. Higher hospitalization and mortality rates among SARS-CoV-2-infected persons in rural America. J Rural Health. 2023;39(1):39-54. doi:10.1111/jrh.12689
14. Su Y, Yuan D, Chen DG, et al. Multiple early factors anticipate post-acute COVID-19 sequelae. Cell. 2022;185(5):881-895.e20. doi:10.1016/j.cell.2022.01.014
15. Pfaff ER, Girvin AT, Bennett TD, et al. Identifying who has long COVID in the USA: a machine learning approach using N3C data. Lancet Digit Health. 2022;4(7):e532-e541. doi:10.1016/S2589-7500(22)00048-6
16. Subramanian A, Nirantharakumar K, Hughes S, et al. Symptoms and risk factors for long COVID in non-hospitalized adults. Nat Med. 2022;28(8):1706-1714. doi:10.1038/s41591-022-01909-w
17. Munblit D, O’Hara ME, Akrami A, Perego E, Olliaro P, Needham DM. Long COVID: aiming for a consensus. Lancet Respir Med. 2022;10(7):632-634. doi:10.1016/S2213-2600(22)00135-7
18. Thaweethai T, Jolley SE, Karlson EW, et al. Development of a definition of postacute sequelae of SARS-CoV-2 infection. JAMA. 2023;329(22):1934-1946. doi:10.1001/jama.2023.8823
19. Sundermann AJ, Chen J, Kumar P, et al. Whole-genome sequencing surveillance and machine learning of the electronic health record for enhanced healthcare outbreak detection. Clin Infect Dis. 2022;75(3):476-482. doi:10.1093/cid/ciab946
VA Big Data Science: A Model for Improved National Pandemic Response Present and Future
The COVID-19 pandemic emphasized the need for rapid response research in health care. The robust enterprise approach used by the US Department of Veterans Affairs (VA), termed VA Research, is meeting these needs by using existing outstanding data resources and interdisciplinary collaborations.1 In the first 7 months of 2021 alone, while many US health care systems struggled with limited data, VA Research published more than 300 unique and instrumental research papers addressing urgent questions about transmission, vaccination, therapeutics, and health impacts of COVID-19 on its high-risk population.1 The ability to leverage the VA electronic health record (EHR) and Corporate Data Warehouse (CDW)—a fully established data system bringing together test results, prescriptions, and complete patient health records, readily accessible and updated daily—was substantial.
With more than 9 million veterans enrolled in care at 171 medical centers and 1113 outpatient facilities across the US and its territories, the CDW provides an unprecedented opportunity to examine outcomes in real time. This allowed research groups such as the VA St Louis Health Care System Research and Education Service to build a cohort of 181,280 veterans with diabetes and positive COVID-19 test results within a 6-month period in 2021 to study the incidence of new diagnoses of diabetes after COVID-19 infection.2 Similarly, the Clinical Epidemiology Program (CEP) at VA White River Junction Health Care System built a cohort of 1,363,180 veterans who received at least 1 COVID-19 vaccine by March 7, 2021, to analyze coverage and effectiveness of those vaccines
The innovation and speed of COVID-19 vaccine development and distribution in the US were unprecedented. The rapid discovery and implementation of multiple preventives and therapeutics for COVID-19 could not have been possible without shared information within a competitive industry. VA studies added significantly to understanding the clinical performance of the messenger RNA (mRNA) COVID-19 vaccines, antivirals, and monoclonal treatments in a real-world setting. For example, a vaccine coverage study by VA Research illustrated how successful vaccination for COVID-19 at the VA has been in protecting a diverse community of patients from hospitalization and death, particularly the highly comorbid, racial and ethnic minorities, and other high-risk populations.3 The study demonstrated the power of the VA system to generate robust and compelling clinical endpoint effectiveness data across a broad range of high-risk groups.
This success is promising. However, the COVID-19 pandemic is not over, and the next could prove even more challenging. For example, through a recent partnership with the US Department of Defense (DoD), the VA was able to rapidly analyze the effectiveness of previous smallpox vaccination efforts in the military for preventing mpox infections.5 We should take this opportunity to think creatively about ways to improve our existing infrastructure based on what we have learned.
A Role for VA Research in Efficacy
The US Food and Drug Administration (FDA) Reauthorization Act of 2017 requires that manufacturers submit evidence establishing a product’s benefits (effectiveness) outweigh its risks (safety) before it can be promoted and distributed.6 As such, the FDA has been obligated by external stakeholders and Congress to be more explicit and transparent about benefit-risk profile supporting its decisions on licensure. This process led to requiring more phase 4 postmarketing observational studies for safety and effectiveness.7 Although the FDA postlicensure system remains vigilant toward safety, effectiveness information is limited due to insufficient reporting (with exceptions of manufacturer studies for new indications or to exhibit superior comparative effectiveness). The agency typically relies on a static set of efficacy data generated prelicensure with a dynamic and evolving set of safety data accrued postlicensure to support its assessment that benefits outweigh risks.
For example, operating in near real time, postauthorization safety monitoring systems, led by the Centers for Disease Control and Prevention and other federal systems, identified a safety signal for thrombosis following the Janssen COVID-19 vaccination. Distribution was quickly paused, the safety signal was investigated, the magnitude of the risk was characterized, new language describing the risk and providing guidance regarding clinical management was included in labeling, and distribution was resumed, all within a few weeks. This remarkable success demonstrated how timely the safety system can operate to evaluate risk.
In contrast, the duration and extent of protection against COVID-19 variants are largely limited to the assessment of immune biomarker surrogates. Such clinical effectiveness data are urgently needed for the FDA’s Center for Biologics Evaluation and Research and Center for Drug Evaluation and Research to make accurate benefit-risk assessments and continue to conclude the balance is favorable. As we prepare for the next pandemic, we must consider plans for monitoring postauthorization/postlicensure effectiveness as well as safety in real time. VA Research is ideally situated for this task.
Published studies on effectiveness at the VA serve as a prototype and could lead the way to initiating those preparations.4,8-11 One of the striking features of the VA system that became apparent in the preparation of the mRNA vaccine study was the speed at which an enormous volume of COVID-19 testing data were produced. This enabled implementation of methodologically sound test-negative and case-control analysis. Analyses sufficiently powered to conclude mRNA vaccines were highly effective when used in real-world conditions among a diverse population from nearly every state and territory during a period in which multiple COVID-19 variants were already circulating.3 This is unique to the VA and would not be possible for any other US health care system. With planning, the VA system could produce product-specific, real-world evidence of effectiveness comparable to the timeliness and quality of the safety data currently produced to support regulatory benefit-risk assessments. For example, the VA conducted an effectiveness study of tixagevimab/cilgavimab for preventing COVID-19 during the initial Omicron surge, which is continually updated while Omicron circulates and repeatable for different subvariants.12
The FDA continues to collaborate with the VA on demonstration projects to evaluate the impact of available vaccines and treatment against COVID-19 variants. The VA has also initiated several large-scale sequencing programs for COVID-19 specimens that will support these efforts, including VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD), VA Sequencing for Research Clinical and Epidemiology (SeqFORCE), and VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE).13,14 Successful proof-of-concept studies using these data could provide a template for VA and other medical systems/databases to report effectiveness in near real time.
Interagency Collaboration
The potential advantages of federal agencies working with the VA to build an infrastructure capable of generating real-world evidence effectiveness analyses in near real time is not limited to needs that will arise in the next pandemic. For example, generating randomized, placebo-controlled, clinical trial endpoint data on the effectiveness of new variant vaccines will be difficult from a feasibility and ethical standpoint. Combining the VA’s robust virus sequencing program with preexisting mechanisms, such as expanded access studies (allowed under FDA Investigational New Drug regulations), researchers could enable a large-scale effective evaluation program of vaccination with variant or universal COVID-19 vaccines, using rapidly accruing effectiveness data.
The pandemic created opportunities to advance innovative approaches to medical product development. Some have advocated these innovative approaches should proceed together toward a seamless convergence between the domains of medical research and clinical care. A shift toward expecting, as a matter of routine, effectiveness data to be generated in near real time and made available for benefit-risk assessment would be a useful step in that direction.
Expanding and sharing analytical platforms, including methodology and programming codes, will allow increased access to rapidly refreshed real-world data. A common adaptive platform of complete and continuously updated data will also enable a wider community of researchers to create multiple investigatory groups simultaneously accessing fully de-identified data for concurrent observational studies. In turn, researchers need to have programming, study design, and methodology ready in an open-source platform. An efficient platform would also require the adoption of artificial intelligence, natural language processing, imaging processing, and quantum computing for validation and improved data quality.
COVID-19 has demonstrated the need for open science data synchronization with universal access for faster action and improved outcomes able to gain public confidence. OpenSafely (UK), a software platform for analysis of EHR data that is shared automatically and openly for scientific review and efficient reuse, created a cohort of about 23.4 million records for observational review of monoclonal COVID-19 treatments. To keep pace with the UK, Israel, and other nationalized systems, the US would benefit from duplicating this example of coordination between federal agencies and their data repositories. For example, combining data between the DoD, which captures active military health care data through TRICARE, and VA, which follows postmilitary discharge, would create datasets encompassing complete life spans. Additionally, expanding the National COVID Cohort Collaborative (N3C) program—one of the largest collections of clinical data related to COVID-19 symptoms and patient outcomes in the US—to include EHR data from DoD, VA, Medicare, and Test to Treat initiative partners would further expand research capabilities. This could be accomplished through a framework of anonymized, readily available, harmonized data. EHRs with synchronized datasets from every health care practitioner—independent pharmacies, primary care physicians, and hospitals—could all work to create a de-identified, comprehensive, continuously updated, near real-time dataset accessible to all federal researchers.
Conclusions
The VA has been lauded for its rapid, effective response to the current pandemic. The successful management and prescription of vaccines and treatment to the largely high-risk veteran population was possible because of the existing data framework within the VA. VA Research continues to build and refine infrastructure to improve speed, quality, and value of data analytics. We can do more. Expanding partnerships to use existing VA data strategies in designing a cooperative national data alliance would deliver necessary progress to research and public health.
Acknowledgments
The authors thank Jeff Roberts, MD, for his insight on the US Food and Drug Administration, its responsibilities, and the potential benefit of real world data to its missions.
1. US Department of Veterans Affairs, Veterans Health Administration. Third report details VA’s continued efforts addressing COVID-19 pandemic. Accessed August 15, 2023. https://www.va.gov/opa/pressrel/pressrelease.cfm?id=5748
2. Xie Y, Ziyad A. Risks and burdens of incident diabetes in long COVID: a cohort study. Lancet Diabetes Endocrinol. 2022;10(5):311-321. doi:10.1016/S2213-8587(22)00044-4
3. Young-Xu Y, Korves C, Roberts J, et al. Coverage and estimated effectiveness of mRNA COVID-19 vaccines among US veterans. JAMA Netw Open. 2021;4(10):e2128391. doi:10.1001/jamanetworkopen.2021.28391
4. Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in U.S. veterans. N Engl J Med. 2022;386(2):105-115. doi:10.1056/NEJMoa2115463
5. Titanji BK, Eick-Cost A, Partan ES, et al. Effectiveness of smallpox vaccination to prevent mpox in military personnel. N Engl J Med. 2023;389(12):1147-1148. doi:10.1056/NEJMc2300805
6. Sarata AK, Dabrowska A, Johnson JA, Thaul S. FDA Reauthorization Act of 2017. Accessed August 15, 2023. https://sgp.fas.org/crs/misc/R44961.pdf
7. US Food and Drug Administration. FDA’s sentinel initiative–background. February 2, 2022. Updated February 4, 2022. Accessed August 15, 2023. https://www.fda.gov/safety/fdas-sentinel-initiative/fdas-sentinel-initiative-background
8. Bajema KL, Dahl RM, Prill MM, et al; SUPERNOVA COVID-19; Surveillance Group. Effectiveness of COVID-19 mRNA vaccines against COVID-19–associated hospitalization—five Veterans Affairs medical centers, United States, February 1–August 6, 2021. MMWR Morb Mortal Wkly. 2021;70(37):1294-1299. doi:10.15585/mmwr.mm7037e3
9. Sharma A, Oda G, Holodniy M. COVID-19 vaccine breakthrough infections in Veterans Health Administration. medRxiv. Posted September 26, 2021. doi:10.1101/2021.09.23.21263864
10. Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of third doses of mRNA-based COVID-19 vaccines in US veterans. Nat Microbiol. 2023;8(1):55-63. doi:10.1038/s41564-022-01272-z
11. Tang F, Hammel IS, Andrew MK, Ruiz JG. Frailty reduces vaccine effectiveness against SARS-CoV-2 infection: a test-negative case control study using national VA data. J Nutr Health Aging. 2023;27(2):81-88. doi:10.1007/s12603-023-1885-1
12. Young-Xu Y, Epstein L, Marconi VC, et al. Tixagevimab/cilgavimab for preventing COVID-19 during the Omicron surge: retrospective analysis of National Veterans Health Administration electronic data. mBio. 2023;14(4):e0102423. doi:10.1128/mbio.01024-23
13. US Department of Veterans Affairs. VA science and health initiative to combat infectious and emerging life-threatening diseases. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac64
14. Bilal MY. Similarity index–probabilistic confidence estimation of SARS-CoV-2 strain relatedness in localized outbreaks. Epidemiologia. 2022;3(2):238-249. doi:10.3390/epidemiologia3020019
The COVID-19 pandemic emphasized the need for rapid response research in health care. The robust enterprise approach used by the US Department of Veterans Affairs (VA), termed VA Research, is meeting these needs by using existing outstanding data resources and interdisciplinary collaborations.1 In the first 7 months of 2021 alone, while many US health care systems struggled with limited data, VA Research published more than 300 unique and instrumental research papers addressing urgent questions about transmission, vaccination, therapeutics, and health impacts of COVID-19 on its high-risk population.1 The ability to leverage the VA electronic health record (EHR) and Corporate Data Warehouse (CDW)—a fully established data system bringing together test results, prescriptions, and complete patient health records, readily accessible and updated daily—was substantial.
With more than 9 million veterans enrolled in care at 171 medical centers and 1113 outpatient facilities across the US and its territories, the CDW provides an unprecedented opportunity to examine outcomes in real time. This allowed research groups such as the VA St Louis Health Care System Research and Education Service to build a cohort of 181,280 veterans with diabetes and positive COVID-19 test results within a 6-month period in 2021 to study the incidence of new diagnoses of diabetes after COVID-19 infection.2 Similarly, the Clinical Epidemiology Program (CEP) at VA White River Junction Health Care System built a cohort of 1,363,180 veterans who received at least 1 COVID-19 vaccine by March 7, 2021, to analyze coverage and effectiveness of those vaccines
The innovation and speed of COVID-19 vaccine development and distribution in the US were unprecedented. The rapid discovery and implementation of multiple preventives and therapeutics for COVID-19 could not have been possible without shared information within a competitive industry. VA studies added significantly to understanding the clinical performance of the messenger RNA (mRNA) COVID-19 vaccines, antivirals, and monoclonal treatments in a real-world setting. For example, a vaccine coverage study by VA Research illustrated how successful vaccination for COVID-19 at the VA has been in protecting a diverse community of patients from hospitalization and death, particularly the highly comorbid, racial and ethnic minorities, and other high-risk populations.3 The study demonstrated the power of the VA system to generate robust and compelling clinical endpoint effectiveness data across a broad range of high-risk groups.
This success is promising. However, the COVID-19 pandemic is not over, and the next could prove even more challenging. For example, through a recent partnership with the US Department of Defense (DoD), the VA was able to rapidly analyze the effectiveness of previous smallpox vaccination efforts in the military for preventing mpox infections.5 We should take this opportunity to think creatively about ways to improve our existing infrastructure based on what we have learned.
A Role for VA Research in Efficacy
The US Food and Drug Administration (FDA) Reauthorization Act of 2017 requires that manufacturers submit evidence establishing a product’s benefits (effectiveness) outweigh its risks (safety) before it can be promoted and distributed.6 As such, the FDA has been obligated by external stakeholders and Congress to be more explicit and transparent about benefit-risk profile supporting its decisions on licensure. This process led to requiring more phase 4 postmarketing observational studies for safety and effectiveness.7 Although the FDA postlicensure system remains vigilant toward safety, effectiveness information is limited due to insufficient reporting (with exceptions of manufacturer studies for new indications or to exhibit superior comparative effectiveness). The agency typically relies on a static set of efficacy data generated prelicensure with a dynamic and evolving set of safety data accrued postlicensure to support its assessment that benefits outweigh risks.
For example, operating in near real time, postauthorization safety monitoring systems, led by the Centers for Disease Control and Prevention and other federal systems, identified a safety signal for thrombosis following the Janssen COVID-19 vaccination. Distribution was quickly paused, the safety signal was investigated, the magnitude of the risk was characterized, new language describing the risk and providing guidance regarding clinical management was included in labeling, and distribution was resumed, all within a few weeks. This remarkable success demonstrated how timely the safety system can operate to evaluate risk.
In contrast, the duration and extent of protection against COVID-19 variants are largely limited to the assessment of immune biomarker surrogates. Such clinical effectiveness data are urgently needed for the FDA’s Center for Biologics Evaluation and Research and Center for Drug Evaluation and Research to make accurate benefit-risk assessments and continue to conclude the balance is favorable. As we prepare for the next pandemic, we must consider plans for monitoring postauthorization/postlicensure effectiveness as well as safety in real time. VA Research is ideally situated for this task.
Published studies on effectiveness at the VA serve as a prototype and could lead the way to initiating those preparations.4,8-11 One of the striking features of the VA system that became apparent in the preparation of the mRNA vaccine study was the speed at which an enormous volume of COVID-19 testing data were produced. This enabled implementation of methodologically sound test-negative and case-control analysis. Analyses sufficiently powered to conclude mRNA vaccines were highly effective when used in real-world conditions among a diverse population from nearly every state and territory during a period in which multiple COVID-19 variants were already circulating.3 This is unique to the VA and would not be possible for any other US health care system. With planning, the VA system could produce product-specific, real-world evidence of effectiveness comparable to the timeliness and quality of the safety data currently produced to support regulatory benefit-risk assessments. For example, the VA conducted an effectiveness study of tixagevimab/cilgavimab for preventing COVID-19 during the initial Omicron surge, which is continually updated while Omicron circulates and repeatable for different subvariants.12
The FDA continues to collaborate with the VA on demonstration projects to evaluate the impact of available vaccines and treatment against COVID-19 variants. The VA has also initiated several large-scale sequencing programs for COVID-19 specimens that will support these efforts, including VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD), VA Sequencing for Research Clinical and Epidemiology (SeqFORCE), and VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE).13,14 Successful proof-of-concept studies using these data could provide a template for VA and other medical systems/databases to report effectiveness in near real time.
Interagency Collaboration
The potential advantages of federal agencies working with the VA to build an infrastructure capable of generating real-world evidence effectiveness analyses in near real time is not limited to needs that will arise in the next pandemic. For example, generating randomized, placebo-controlled, clinical trial endpoint data on the effectiveness of new variant vaccines will be difficult from a feasibility and ethical standpoint. Combining the VA’s robust virus sequencing program with preexisting mechanisms, such as expanded access studies (allowed under FDA Investigational New Drug regulations), researchers could enable a large-scale effective evaluation program of vaccination with variant or universal COVID-19 vaccines, using rapidly accruing effectiveness data.
The pandemic created opportunities to advance innovative approaches to medical product development. Some have advocated these innovative approaches should proceed together toward a seamless convergence between the domains of medical research and clinical care. A shift toward expecting, as a matter of routine, effectiveness data to be generated in near real time and made available for benefit-risk assessment would be a useful step in that direction.
Expanding and sharing analytical platforms, including methodology and programming codes, will allow increased access to rapidly refreshed real-world data. A common adaptive platform of complete and continuously updated data will also enable a wider community of researchers to create multiple investigatory groups simultaneously accessing fully de-identified data for concurrent observational studies. In turn, researchers need to have programming, study design, and methodology ready in an open-source platform. An efficient platform would also require the adoption of artificial intelligence, natural language processing, imaging processing, and quantum computing for validation and improved data quality.
COVID-19 has demonstrated the need for open science data synchronization with universal access for faster action and improved outcomes able to gain public confidence. OpenSafely (UK), a software platform for analysis of EHR data that is shared automatically and openly for scientific review and efficient reuse, created a cohort of about 23.4 million records for observational review of monoclonal COVID-19 treatments. To keep pace with the UK, Israel, and other nationalized systems, the US would benefit from duplicating this example of coordination between federal agencies and their data repositories. For example, combining data between the DoD, which captures active military health care data through TRICARE, and VA, which follows postmilitary discharge, would create datasets encompassing complete life spans. Additionally, expanding the National COVID Cohort Collaborative (N3C) program—one of the largest collections of clinical data related to COVID-19 symptoms and patient outcomes in the US—to include EHR data from DoD, VA, Medicare, and Test to Treat initiative partners would further expand research capabilities. This could be accomplished through a framework of anonymized, readily available, harmonized data. EHRs with synchronized datasets from every health care practitioner—independent pharmacies, primary care physicians, and hospitals—could all work to create a de-identified, comprehensive, continuously updated, near real-time dataset accessible to all federal researchers.
Conclusions
The VA has been lauded for its rapid, effective response to the current pandemic. The successful management and prescription of vaccines and treatment to the largely high-risk veteran population was possible because of the existing data framework within the VA. VA Research continues to build and refine infrastructure to improve speed, quality, and value of data analytics. We can do more. Expanding partnerships to use existing VA data strategies in designing a cooperative national data alliance would deliver necessary progress to research and public health.
Acknowledgments
The authors thank Jeff Roberts, MD, for his insight on the US Food and Drug Administration, its responsibilities, and the potential benefit of real world data to its missions.
The COVID-19 pandemic emphasized the need for rapid response research in health care. The robust enterprise approach used by the US Department of Veterans Affairs (VA), termed VA Research, is meeting these needs by using existing outstanding data resources and interdisciplinary collaborations.1 In the first 7 months of 2021 alone, while many US health care systems struggled with limited data, VA Research published more than 300 unique and instrumental research papers addressing urgent questions about transmission, vaccination, therapeutics, and health impacts of COVID-19 on its high-risk population.1 The ability to leverage the VA electronic health record (EHR) and Corporate Data Warehouse (CDW)—a fully established data system bringing together test results, prescriptions, and complete patient health records, readily accessible and updated daily—was substantial.
With more than 9 million veterans enrolled in care at 171 medical centers and 1113 outpatient facilities across the US and its territories, the CDW provides an unprecedented opportunity to examine outcomes in real time. This allowed research groups such as the VA St Louis Health Care System Research and Education Service to build a cohort of 181,280 veterans with diabetes and positive COVID-19 test results within a 6-month period in 2021 to study the incidence of new diagnoses of diabetes after COVID-19 infection.2 Similarly, the Clinical Epidemiology Program (CEP) at VA White River Junction Health Care System built a cohort of 1,363,180 veterans who received at least 1 COVID-19 vaccine by March 7, 2021, to analyze coverage and effectiveness of those vaccines
The innovation and speed of COVID-19 vaccine development and distribution in the US were unprecedented. The rapid discovery and implementation of multiple preventives and therapeutics for COVID-19 could not have been possible without shared information within a competitive industry. VA studies added significantly to understanding the clinical performance of the messenger RNA (mRNA) COVID-19 vaccines, antivirals, and monoclonal treatments in a real-world setting. For example, a vaccine coverage study by VA Research illustrated how successful vaccination for COVID-19 at the VA has been in protecting a diverse community of patients from hospitalization and death, particularly the highly comorbid, racial and ethnic minorities, and other high-risk populations.3 The study demonstrated the power of the VA system to generate robust and compelling clinical endpoint effectiveness data across a broad range of high-risk groups.
This success is promising. However, the COVID-19 pandemic is not over, and the next could prove even more challenging. For example, through a recent partnership with the US Department of Defense (DoD), the VA was able to rapidly analyze the effectiveness of previous smallpox vaccination efforts in the military for preventing mpox infections.5 We should take this opportunity to think creatively about ways to improve our existing infrastructure based on what we have learned.
A Role for VA Research in Efficacy
The US Food and Drug Administration (FDA) Reauthorization Act of 2017 requires that manufacturers submit evidence establishing a product’s benefits (effectiveness) outweigh its risks (safety) before it can be promoted and distributed.6 As such, the FDA has been obligated by external stakeholders and Congress to be more explicit and transparent about benefit-risk profile supporting its decisions on licensure. This process led to requiring more phase 4 postmarketing observational studies for safety and effectiveness.7 Although the FDA postlicensure system remains vigilant toward safety, effectiveness information is limited due to insufficient reporting (with exceptions of manufacturer studies for new indications or to exhibit superior comparative effectiveness). The agency typically relies on a static set of efficacy data generated prelicensure with a dynamic and evolving set of safety data accrued postlicensure to support its assessment that benefits outweigh risks.
For example, operating in near real time, postauthorization safety monitoring systems, led by the Centers for Disease Control and Prevention and other federal systems, identified a safety signal for thrombosis following the Janssen COVID-19 vaccination. Distribution was quickly paused, the safety signal was investigated, the magnitude of the risk was characterized, new language describing the risk and providing guidance regarding clinical management was included in labeling, and distribution was resumed, all within a few weeks. This remarkable success demonstrated how timely the safety system can operate to evaluate risk.
In contrast, the duration and extent of protection against COVID-19 variants are largely limited to the assessment of immune biomarker surrogates. Such clinical effectiveness data are urgently needed for the FDA’s Center for Biologics Evaluation and Research and Center for Drug Evaluation and Research to make accurate benefit-risk assessments and continue to conclude the balance is favorable. As we prepare for the next pandemic, we must consider plans for monitoring postauthorization/postlicensure effectiveness as well as safety in real time. VA Research is ideally situated for this task.
Published studies on effectiveness at the VA serve as a prototype and could lead the way to initiating those preparations.4,8-11 One of the striking features of the VA system that became apparent in the preparation of the mRNA vaccine study was the speed at which an enormous volume of COVID-19 testing data were produced. This enabled implementation of methodologically sound test-negative and case-control analysis. Analyses sufficiently powered to conclude mRNA vaccines were highly effective when used in real-world conditions among a diverse population from nearly every state and territory during a period in which multiple COVID-19 variants were already circulating.3 This is unique to the VA and would not be possible for any other US health care system. With planning, the VA system could produce product-specific, real-world evidence of effectiveness comparable to the timeliness and quality of the safety data currently produced to support regulatory benefit-risk assessments. For example, the VA conducted an effectiveness study of tixagevimab/cilgavimab for preventing COVID-19 during the initial Omicron surge, which is continually updated while Omicron circulates and repeatable for different subvariants.12
The FDA continues to collaborate with the VA on demonstration projects to evaluate the impact of available vaccines and treatment against COVID-19 variants. The VA has also initiated several large-scale sequencing programs for COVID-19 specimens that will support these efforts, including VA Science and Health Initiative to Combat Infectious and Emerging Life-Threatening Diseases (VA SHIELD), VA Sequencing for Research Clinical and Epidemiology (SeqFORCE), and VA Sequencing Collaborations United for Research and Epidemiology (SeqCURE).13,14 Successful proof-of-concept studies using these data could provide a template for VA and other medical systems/databases to report effectiveness in near real time.
Interagency Collaboration
The potential advantages of federal agencies working with the VA to build an infrastructure capable of generating real-world evidence effectiveness analyses in near real time is not limited to needs that will arise in the next pandemic. For example, generating randomized, placebo-controlled, clinical trial endpoint data on the effectiveness of new variant vaccines will be difficult from a feasibility and ethical standpoint. Combining the VA’s robust virus sequencing program with preexisting mechanisms, such as expanded access studies (allowed under FDA Investigational New Drug regulations), researchers could enable a large-scale effective evaluation program of vaccination with variant or universal COVID-19 vaccines, using rapidly accruing effectiveness data.
The pandemic created opportunities to advance innovative approaches to medical product development. Some have advocated these innovative approaches should proceed together toward a seamless convergence between the domains of medical research and clinical care. A shift toward expecting, as a matter of routine, effectiveness data to be generated in near real time and made available for benefit-risk assessment would be a useful step in that direction.
Expanding and sharing analytical platforms, including methodology and programming codes, will allow increased access to rapidly refreshed real-world data. A common adaptive platform of complete and continuously updated data will also enable a wider community of researchers to create multiple investigatory groups simultaneously accessing fully de-identified data for concurrent observational studies. In turn, researchers need to have programming, study design, and methodology ready in an open-source platform. An efficient platform would also require the adoption of artificial intelligence, natural language processing, imaging processing, and quantum computing for validation and improved data quality.
COVID-19 has demonstrated the need for open science data synchronization with universal access for faster action and improved outcomes able to gain public confidence. OpenSafely (UK), a software platform for analysis of EHR data that is shared automatically and openly for scientific review and efficient reuse, created a cohort of about 23.4 million records for observational review of monoclonal COVID-19 treatments. To keep pace with the UK, Israel, and other nationalized systems, the US would benefit from duplicating this example of coordination between federal agencies and their data repositories. For example, combining data between the DoD, which captures active military health care data through TRICARE, and VA, which follows postmilitary discharge, would create datasets encompassing complete life spans. Additionally, expanding the National COVID Cohort Collaborative (N3C) program—one of the largest collections of clinical data related to COVID-19 symptoms and patient outcomes in the US—to include EHR data from DoD, VA, Medicare, and Test to Treat initiative partners would further expand research capabilities. This could be accomplished through a framework of anonymized, readily available, harmonized data. EHRs with synchronized datasets from every health care practitioner—independent pharmacies, primary care physicians, and hospitals—could all work to create a de-identified, comprehensive, continuously updated, near real-time dataset accessible to all federal researchers.
Conclusions
The VA has been lauded for its rapid, effective response to the current pandemic. The successful management and prescription of vaccines and treatment to the largely high-risk veteran population was possible because of the existing data framework within the VA. VA Research continues to build and refine infrastructure to improve speed, quality, and value of data analytics. We can do more. Expanding partnerships to use existing VA data strategies in designing a cooperative national data alliance would deliver necessary progress to research and public health.
Acknowledgments
The authors thank Jeff Roberts, MD, for his insight on the US Food and Drug Administration, its responsibilities, and the potential benefit of real world data to its missions.
1. US Department of Veterans Affairs, Veterans Health Administration. Third report details VA’s continued efforts addressing COVID-19 pandemic. Accessed August 15, 2023. https://www.va.gov/opa/pressrel/pressrelease.cfm?id=5748
2. Xie Y, Ziyad A. Risks and burdens of incident diabetes in long COVID: a cohort study. Lancet Diabetes Endocrinol. 2022;10(5):311-321. doi:10.1016/S2213-8587(22)00044-4
3. Young-Xu Y, Korves C, Roberts J, et al. Coverage and estimated effectiveness of mRNA COVID-19 vaccines among US veterans. JAMA Netw Open. 2021;4(10):e2128391. doi:10.1001/jamanetworkopen.2021.28391
4. Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in U.S. veterans. N Engl J Med. 2022;386(2):105-115. doi:10.1056/NEJMoa2115463
5. Titanji BK, Eick-Cost A, Partan ES, et al. Effectiveness of smallpox vaccination to prevent mpox in military personnel. N Engl J Med. 2023;389(12):1147-1148. doi:10.1056/NEJMc2300805
6. Sarata AK, Dabrowska A, Johnson JA, Thaul S. FDA Reauthorization Act of 2017. Accessed August 15, 2023. https://sgp.fas.org/crs/misc/R44961.pdf
7. US Food and Drug Administration. FDA’s sentinel initiative–background. February 2, 2022. Updated February 4, 2022. Accessed August 15, 2023. https://www.fda.gov/safety/fdas-sentinel-initiative/fdas-sentinel-initiative-background
8. Bajema KL, Dahl RM, Prill MM, et al; SUPERNOVA COVID-19; Surveillance Group. Effectiveness of COVID-19 mRNA vaccines against COVID-19–associated hospitalization—five Veterans Affairs medical centers, United States, February 1–August 6, 2021. MMWR Morb Mortal Wkly. 2021;70(37):1294-1299. doi:10.15585/mmwr.mm7037e3
9. Sharma A, Oda G, Holodniy M. COVID-19 vaccine breakthrough infections in Veterans Health Administration. medRxiv. Posted September 26, 2021. doi:10.1101/2021.09.23.21263864
10. Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of third doses of mRNA-based COVID-19 vaccines in US veterans. Nat Microbiol. 2023;8(1):55-63. doi:10.1038/s41564-022-01272-z
11. Tang F, Hammel IS, Andrew MK, Ruiz JG. Frailty reduces vaccine effectiveness against SARS-CoV-2 infection: a test-negative case control study using national VA data. J Nutr Health Aging. 2023;27(2):81-88. doi:10.1007/s12603-023-1885-1
12. Young-Xu Y, Epstein L, Marconi VC, et al. Tixagevimab/cilgavimab for preventing COVID-19 during the Omicron surge: retrospective analysis of National Veterans Health Administration electronic data. mBio. 2023;14(4):e0102423. doi:10.1128/mbio.01024-23
13. US Department of Veterans Affairs. VA science and health initiative to combat infectious and emerging life-threatening diseases. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac64
14. Bilal MY. Similarity index–probabilistic confidence estimation of SARS-CoV-2 strain relatedness in localized outbreaks. Epidemiologia. 2022;3(2):238-249. doi:10.3390/epidemiologia3020019
1. US Department of Veterans Affairs, Veterans Health Administration. Third report details VA’s continued efforts addressing COVID-19 pandemic. Accessed August 15, 2023. https://www.va.gov/opa/pressrel/pressrelease.cfm?id=5748
2. Xie Y, Ziyad A. Risks and burdens of incident diabetes in long COVID: a cohort study. Lancet Diabetes Endocrinol. 2022;10(5):311-321. doi:10.1016/S2213-8587(22)00044-4
3. Young-Xu Y, Korves C, Roberts J, et al. Coverage and estimated effectiveness of mRNA COVID-19 vaccines among US veterans. JAMA Netw Open. 2021;4(10):e2128391. doi:10.1001/jamanetworkopen.2021.28391
4. Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in U.S. veterans. N Engl J Med. 2022;386(2):105-115. doi:10.1056/NEJMoa2115463
5. Titanji BK, Eick-Cost A, Partan ES, et al. Effectiveness of smallpox vaccination to prevent mpox in military personnel. N Engl J Med. 2023;389(12):1147-1148. doi:10.1056/NEJMc2300805
6. Sarata AK, Dabrowska A, Johnson JA, Thaul S. FDA Reauthorization Act of 2017. Accessed August 15, 2023. https://sgp.fas.org/crs/misc/R44961.pdf
7. US Food and Drug Administration. FDA’s sentinel initiative–background. February 2, 2022. Updated February 4, 2022. Accessed August 15, 2023. https://www.fda.gov/safety/fdas-sentinel-initiative/fdas-sentinel-initiative-background
8. Bajema KL, Dahl RM, Prill MM, et al; SUPERNOVA COVID-19; Surveillance Group. Effectiveness of COVID-19 mRNA vaccines against COVID-19–associated hospitalization—five Veterans Affairs medical centers, United States, February 1–August 6, 2021. MMWR Morb Mortal Wkly. 2021;70(37):1294-1299. doi:10.15585/mmwr.mm7037e3
9. Sharma A, Oda G, Holodniy M. COVID-19 vaccine breakthrough infections in Veterans Health Administration. medRxiv. Posted September 26, 2021. doi:10.1101/2021.09.23.21263864
10. Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of third doses of mRNA-based COVID-19 vaccines in US veterans. Nat Microbiol. 2023;8(1):55-63. doi:10.1038/s41564-022-01272-z
11. Tang F, Hammel IS, Andrew MK, Ruiz JG. Frailty reduces vaccine effectiveness against SARS-CoV-2 infection: a test-negative case control study using national VA data. J Nutr Health Aging. 2023;27(2):81-88. doi:10.1007/s12603-023-1885-1
12. Young-Xu Y, Epstein L, Marconi VC, et al. Tixagevimab/cilgavimab for preventing COVID-19 during the Omicron surge: retrospective analysis of National Veterans Health Administration electronic data. mBio. 2023;14(4):e0102423. doi:10.1128/mbio.01024-23
13. US Department of Veterans Affairs. VA science and health initiative to combat infectious and emerging life-threatening diseases. Open Forum Infect Dis. 2022;9(12):ofac641. doi:10.1093/ofid/ofac64
14. Bilal MY. Similarity index–probabilistic confidence estimation of SARS-CoV-2 strain relatedness in localized outbreaks. Epidemiologia. 2022;3(2):238-249. doi:10.3390/epidemiologia3020019
Leveraging the Million Veteran Program Infrastructure and Data for a Rapid Research Response to COVID-19
The Million Veteran Program (MVP) was launched in 2011 by the US Department of Veterans Affairs (VA) to enroll at least 1 million veterans in a longitudinal cohort to better understand how genes, lifestyle, military experience, and environmental exposures interact to influence health and illness and ultimately enable precision health care. The MVP has established a national, centralized infrastructure for recruitment and enrollment, biospecimen and data collection and storage, data generation and curation, and secure data access. When the COVID-19 pandemic hit in 2020, the MVP was leveraged to support research utilizing the following key infrastructure components: (1) MVP recruitment and enrollment platform to provide support for COVID-19 vaccine and treatment trials and to collect COVID-19 data from MVP participants; (2) using MVP Phenomics for COVID-19 research data cleaning and curation, assisting with the development of a VA Severity Index for COVID-19, and forming 6 scientific working groups to coordinate COVID-19 research questions; and (3) the VA/MVP and US Department of Energy (DOE) partnership to assist in responding to COVID-19 research questions identified by the US Food and Drug Administration (FDA). This article describes these infrastructure components in more detail and highlights key findings from the MVP COVID-19 research efforts.
MVP Infrastructure
The Veterans Health Administration (VHA) Office of Research and Development (ORD) oversaw efforts to develop the VA Coronavirus Research Volunteer List (the COVID-19 registry). To support the registry, the MVP leveraged its infrastructure to facilitate a rapid response. The MVP is designed as a full-service and centralized recruitment and enrollment platform. This includes MVP office oversight; MVP coordinating centers that manage the centralized platform; an information center that handles inbound and outbound calls; an informatics system built for recruitment and enrollment monitoring and tracking; and a network of more than 70 participating MVP sites with dedicated staff to conduct recruitment and enrollment activities. The MVP used its informatics infrastructure to support secure data storage for the registry volunteer information. MVP coordinating center staff worked with the COVID-19 registry to invite > 125,000 MVP participants from approximately 20 MVP sites. Additionally, MVP information center staff made > 4000 calls to prospective registry volunteers. This work resulted in 1300 volunteers agreeing to be
New Data Collection
The MVP protocol was approved by the VA Central Institutional Review Board (IRB) in 2011. As part of initial enrollment in MVP, participants consented to recontact for additional self-report information along with access to their electronic health record (EHR). This allows for the linkage of EHR and survey response data, thus providing a comprehensive understanding of health history before and after a self-reported COVID-19 diagnosis. Between May 2020 and September 2021, the MVP COVID-19 survey was distributed to existing MVP participants via mail, telephone, and email with the ability to complete the survey by paper and pencil or through the MVP online system. Dissemination of the survey was approved by the VA Central IRB in 2020, with nearly 730,000 eligible MVP participants contacted. As of June 2022, 255,737 MVP participants (35% of the eligible cohort) had completed the survey; 86% completed a paper survey while 14% completed it online. Respondents were primarily older (≥ 65 years); 90% were male; close to 7% reported Hispanic ethnicity, and 11% reported Black race.
Findings from this survey provide insight into pandemic behaviors not consistently captured in EHRs, such as psychosocial aspects, including social and emotional support, loss of tangible and intangible resources, as well as COVID-19–related behaviors, such as social distancing and self-protective practices.1 MVP COVID-19 survey data combined with veteran EHRs, responses to other MVP surveys, and genetic data enable MVP researchers to better understand epidemiological, clinical, and psychosocial aspects of the disease. Future COVID-19 studies may use self-reported survey responses to enrich understanding about the effects of the disease on a veteran’s daily life, and possibly validate existing EHR COVID-19 diagnoses and hospitalization findings. This comprehensive data resource provides a unique opportunity to identify new targets for disease prevention, treatment, and management with an emphasis on individual variability in genes, environment, and lifestyle.
COVID-19 Research
In early 2020, the burden of COVID-19 on the US was unprecedented, and little was known about risk factors for severe COVID-19 and deaths. The MVP Phenomics team quickly responded with a large-scale phenome-wide association study (PheWAS) of >
To broaden disease progression data curation and fit the specific needs of the VA, we operationalized and validated the World Health Organization clinical severity scale and used VA EHR data to create the VA Severity Index for COVID-19 (VASIC).3 The VASIC category is now part of the MVP core data repository, where volumes of data from multiple activities are integrated through an automated process to create monthly research-ready data cubes. These activities include extensive data curation, mapping, phenotyping, and adjudication that are performed to curate oxygen supplementation status and other procedures related to treatment that are processed and understood in real time. The data cubes were provisioned to MVP COVID-19 researchers. In addition, the VASIC scale variable is now integrated within the larger VA system for all researchers to use as part of its wider COVID-19 initiative. The VA Centralized Interactive Phenomics Resource (CIPHER) phenomics library now hosts the details of VASIC, codes, metadata, and related COVID-19 data products for all VA communities. In partnership with CIPHER and other internal and external COVID-19 initiatives, the MVP continues to play an integral part for the VA and beyond in the development of a phenomics algorithm for long COVID, or post-acute COVID-19 syndrome (PACS).
Host Genetics in COVID-19
As the SARS-CoV-2 virus continued to spread globally, it became clear that the symptoms and severity of infection experienced by patients varied across a broad spectrum, from being asymptomatic carriers to experiencing severe symptoms in 1 or more organ systems in the body, resulting in death. This variability suggested that host genetics and other host factors may play a role in determining the severity of COVID-19 infection. The MVP dataset, with genetic and health information on > 600,000 MVP participants, provided an ideal dataset to explore host contributions to COVID-19.
In late spring 2020, the MVP executive committee issued a call to the MVP research community to propose study aims around the COVID-19 pandemic that could leverage the phenotypic and genetic data and resources. The MVP quickly formed 6 rapid-response scientific working groups. Their mission was to cultivate collaboration and inclusivity and to coordinate COVID-19 research questions. A steering committee composed of the MVP executive committee, staff from computational environments, working group cochairs, and an administrator, who was responsible for daily oversight of the working groups. In addition, the ORD COVID-19 steering committee reviewed and approved research activities to ensure scientific rigor, as well as alignment with overall ongoing research activities.
The MVP COVID-19 working groups included dozens of researchers who used MVP data to identify disease mechanisms; understand the impact of host genetics on susceptibility, morbidity, and mortality; and identify potential targets for treatments and therapies. The working groups were further supported by MVP analysts to work cross-functionally on genomics, phenomics, statistical genetics, and PheWAS. Each working group chair was responsible for prioritizing concepts and moving them forward in coordination with the MVP and ORD COVID-19 steering committees. An overview of the MVP COVID-19 working groups follows (Table).4-9
Druggable genome. This working group researched drug-repurposing opportunities to prevent severe COVID-19, defined as hospitalization with oxygen therapy (high flow), intubation, mechanical ventilation, vasopressors, dialysis, or death from COVID-19; and prevent complications in patients hospitalized by COVID-19.
Pharmacogenomics. This working group focused on 2 main aims: the impact of apolipoprotein L1 risk variants on acute kidney injury (AKI) and death in Black veterans with COVID-19; and pharmacogenetic analysis of remdesivir-induced liver chemistry abnormalities.
Disease mechanisms. Understanding the underlying pathways and mechanisms behind COVID-19 has been a difficult but important challenge overall in the scientific community. This working group investigated specific genetic markers and effects on COVID-19, including polygenic predisposition to venous thromboembolism associated with increased COVID-19 susceptibility; renal comorbidities and new AKI and unfavorable outcomes among COVID-19–positive sickle cell trait carriers; and mucin 5B, oligomeric mucus/gel-forming gene polymorphism, and protective effects in COVID-19 infection.
Genomics for risk prediction, polygenic risk scores, and mendelian randomization. Risk prediction for COVID-19 has been widely studied mostly aiming at comorbidities and preexisting conditions. The MVP cohort provided a unique opportunity to understand how genetic information can enhance our understanding of COVID-19 risk. This working group focused on: (1) ABO blood group typing and the protective effects of the O blood group on COVID-19 infection; (2) polygenic risk scores and COVID-19 outcomes; (3) human leukocyte antigen typing and COVID-19 outcomes; and (4) a transcriptome-wide association study of COVID-19–positive MVP participants.
Genome-Wide Association Study (GWAS) and Downstream Analysis. This working group performed GWAS of the main COVID-19 outcomes. Results from GWAS unveiled new genetic loci to suggest further investigation on these candidate genes. The results were used by other MVP COVID-19 working groups for their activities. The results also contributed to external collaborations, such as the COVID-19 Host Genetics Initiative.
COVID-19–Related PheWAS. This working group focused on understanding the potential clinical significance of genetic variants associated with susceptibility to, or outcomes of, COVID-19 infection. They worked to identify traits that share genetic variants associated with severe COVID-19 from the Host Genetics Initiative. The group also studied the phenotypic consequences of acquired mosaic chromosomal alterations with early data linking to COVID-19 susceptibility.
COVID-19 Research Partnerships
In 2016, the VA and DOE formed an interagency partnership known as Computational Health Analytics for Medical Precision to Improve Outcomes Now (CHAMPION) to demonstrate the power of combining the VA EHR system, MVP genetic data, and clinical research expertise with DOE high-performance computing infrastructure and artificial intelligence expertise. The VA EHR captures longitudinal care information on veterans with records that go back decades. Furthermore, the VA covers the costs of medications and
The DOE Oak Ridge National Laboratory (ORNL) in Tennessee securely maintains this rich database for the VA. The ORNL Summit supercomputer can complete trillions of calculations per second to provide critical and timely analyses, applying the most advanced and powerful artificial intelligence methods, which would not be possible in more conventional research settings. CHAMPION taught the VA and DOE how to bring their disparate research cultures together for innovative collaborative investigation. Moreover, this collaboration produced a cadre of VA and DOE scientists familiar with VA patient data and experienced in conducting joint research successfully and integrating omics data with clinical data for a better mechanistic understanding. Because of this preexisting collaboration between the VA and DOE, interagency teams were prepared at the start of the COVID-19 pandemic.10-15
Other recently completed studies have developed and validated short-term mortality indices in individuals with COVID-19 based on their preexisting conditions, assessed the generalizability of VA COVID-19 experiences to the US population, and evaluated the effectiveness of hydroxychloroquine with and without azithromycin in VA patients with COVID-19.12,15 A recent study demonstrated the benefit of prophylactic anticoagulation at initial hospitalization.14
The VA also provided the FDA with daily reports on aggregate VA COVID-19 cases and their distribution across the VA system, demographics of VA patients with COVID-19, and analyses of predictive models for positive test results and death. The VA regularly sent the FDA aggregated data showing patterns of medication use and retrospective analyses of the effectiveness of certain medications (including remdesivir and some antithrombotic agents). The FDA used these data along with other data to understand the scope of the pandemic and to predict drug shortages or needs for additional medical equipment, including ventilators.
Limitations
For the most part, MVP infrastructure and partnerships were efficiently leveraged to significantly advance our understanding of the biological basis of COVID-19 and to develop treatments and vaccines. However, there were a few limitations that may have slowed timely and optimal outcomes. An issue not limited to the MVP or VA was the continual evolution of the pandemic and its response. This included evolving definitions of disease, symptomatology, testing, vaccines, and public health recommendations. Keeping pace with the emerging knowledge from these domains was a struggle for the entire scientific community. A more discrete limitation was the number of participants in the MVP with positive COVID-19 test results and positive symptoms; however, this was mitigated by partnering with other groups like the COVID-19 Host Genetics Initiative to increase study participant numbers. Finally, there were logistical and regulatory challenges associated with coordination of national clinical trial recruitment across a VA system with > 100 discrete hospitals.
Conclusions
Having a centralized infrastructure for recruitment and enrollment, including a national research volunteer registry, information center, research staff, and coordinating centers, can allow for expedited enrollment in vaccine and treatment trials in the face of future public health emergencies. VA assets, including its rich EHR and MVP, the world’s largest genomic cohort, have contributed to improving our understanding and management of COVID-19.
1. Whitbourne SB, Nguyen XT, Song RJ, et al. Million Veteran Program’s response to COVID-19: survey development and preliminary findings. PLoS One. 2022;17(4):e0266381. doi:10.1371/journal.pone.0266381
2. Song RJ, Ho YL, Schubert P, et al. Phenome-wide association of 1809 phenotypes and COVID-19 disease progression in the Veterans Health Administration Million Veteran Program. PLoS One. 2021;16(5):e0251651. doi:10.1371/journal.pone.0251651
3. Galloway A, Park Y, Tanukonda V, et al. Impact of COVID-19 severity on long-term events in US veterans using the Veterans Affairs Severity Index for COVID-19 (VASIC). J Infect Dis. 2022;226(12):2113-2117. doi:10.1093/infdis/jiac182
4. Gaziano L, Giambartolomei C, Pereira AC, et al. Actionable druggable genome-wide Mendelian randomization identifies repurposing opportunities for COVID-19. Nat Med. 2021;27(4):668-676. doi:10.0138/s41591-021-01310-z
5. Hung AM, Sha SC, Bick AG, et al. APOL1 risk variants, acute kidney injury, and death in participants with African ancestry hospitalized with COVID-19 from the Million Veteran Program. JAMA Intern Med. 2022;182(4):386-395. doi:10.1001/jamainternmed.2021.8538
6. Verma A, Huffman JE, Gao L, et al. Association of kidney comorbidities and acute kidney failure with unfavorable outcomes after COVID-19 in individuals with the sickle cell trait. JAMA Intern Med. 2022;182(8):796-804. doi:10.1001/jamainternmed.2022.2141
7. Verma A, Tsao NL, Thomann LO, et al. A phenome-wide association study of genes associated with COVID-19 severity reveals shared genetics with complex diseases in the Million Veteran Program. PLoS Genet. 2022;18(4):e1010113. doi:10.1371/journal.pgen.1010113
8. Peloso GM, Tcheandjieu C, McGeary JE, et al. Genetic loci associated with COVID-19 positivity and hospitalization in White, Black, and Hispanic Veterans of the VA Million Veteran Program. Front Genetic. 2022;12:777076. doi:10.3389/fgene.2021.777076
9. Verma A, Minnier J, Wan ES, et al. A MUC5B gene polymorphism, rs35705950-T confers protective effects against COVID-19 hospitalization but not severe disease or mortality. Am J Respir Crit Care Med. 2022;182(8):796-804. doi:10.1164/rccm.202109-2166OC
10. Garvin MR, Alvarez C, Miller JI, et al. A mechanistic model and therapeutic interventions for COVID-19 involving a RAS-mediated bradykinin storm. Elife. 2020;e59177. doi:10.7554/eLife.59177
11. Rentsch CT, Kidwai-Khan F, Tate JP, et al. Patterns of COVID-19 testing and mortality by race and ethnicity among United States veterans: A nationwide cohort study. PLoS Med. 2020;17(9):e1003379. doi:10.1371/journal.pmed.1003379
12. King JT, Yoon JS, Rentsch CT, et al. Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: the Veterans Health Administration COVID-19 (VACO) Index. PLoS One. 2020;15(11):e0241825. doi:10.1371/journal.pone.0241825
13. Joubert W, Weighill D, Kainer D, et al. Attacking the opioid epidemic: determining the epistatic and pleiotropic genetic architectures for chronic pain and opioid addiction. SC18: International Conference for High Performance Computing, Networking, Storage and Analysis. Dallas, TX, USA, 2018:717-730. doi:10.1109/SC.2018.00060
14. Rentsch CT, Beckman JA, Tomlinson L, et al. Early initiation of prophylactic anticoagulation for prevention of COVID-19 mortality: a nationwide cohort study of hospitalized patients in the United States. BMJ. 2021;372:n311. doi:10.1136/bmj.n311
15. Gerlovin H, Posner DC, Ho YL, et al. Pharmacoepidemiology, machine learning, and COVID-19: an intent-to-treat analysis of hydroxychloroquine, with or without Azithromycin, and COVID-19 outcomes among hospitalized US Veterans. Am J Epidemiol. 2021;190(11): 2405-2419. doi:10.1093/aje/kwab183
The Million Veteran Program (MVP) was launched in 2011 by the US Department of Veterans Affairs (VA) to enroll at least 1 million veterans in a longitudinal cohort to better understand how genes, lifestyle, military experience, and environmental exposures interact to influence health and illness and ultimately enable precision health care. The MVP has established a national, centralized infrastructure for recruitment and enrollment, biospecimen and data collection and storage, data generation and curation, and secure data access. When the COVID-19 pandemic hit in 2020, the MVP was leveraged to support research utilizing the following key infrastructure components: (1) MVP recruitment and enrollment platform to provide support for COVID-19 vaccine and treatment trials and to collect COVID-19 data from MVP participants; (2) using MVP Phenomics for COVID-19 research data cleaning and curation, assisting with the development of a VA Severity Index for COVID-19, and forming 6 scientific working groups to coordinate COVID-19 research questions; and (3) the VA/MVP and US Department of Energy (DOE) partnership to assist in responding to COVID-19 research questions identified by the US Food and Drug Administration (FDA). This article describes these infrastructure components in more detail and highlights key findings from the MVP COVID-19 research efforts.
MVP Infrastructure
The Veterans Health Administration (VHA) Office of Research and Development (ORD) oversaw efforts to develop the VA Coronavirus Research Volunteer List (the COVID-19 registry). To support the registry, the MVP leveraged its infrastructure to facilitate a rapid response. The MVP is designed as a full-service and centralized recruitment and enrollment platform. This includes MVP office oversight; MVP coordinating centers that manage the centralized platform; an information center that handles inbound and outbound calls; an informatics system built for recruitment and enrollment monitoring and tracking; and a network of more than 70 participating MVP sites with dedicated staff to conduct recruitment and enrollment activities. The MVP used its informatics infrastructure to support secure data storage for the registry volunteer information. MVP coordinating center staff worked with the COVID-19 registry to invite > 125,000 MVP participants from approximately 20 MVP sites. Additionally, MVP information center staff made > 4000 calls to prospective registry volunteers. This work resulted in 1300 volunteers agreeing to be
New Data Collection
The MVP protocol was approved by the VA Central Institutional Review Board (IRB) in 2011. As part of initial enrollment in MVP, participants consented to recontact for additional self-report information along with access to their electronic health record (EHR). This allows for the linkage of EHR and survey response data, thus providing a comprehensive understanding of health history before and after a self-reported COVID-19 diagnosis. Between May 2020 and September 2021, the MVP COVID-19 survey was distributed to existing MVP participants via mail, telephone, and email with the ability to complete the survey by paper and pencil or through the MVP online system. Dissemination of the survey was approved by the VA Central IRB in 2020, with nearly 730,000 eligible MVP participants contacted. As of June 2022, 255,737 MVP participants (35% of the eligible cohort) had completed the survey; 86% completed a paper survey while 14% completed it online. Respondents were primarily older (≥ 65 years); 90% were male; close to 7% reported Hispanic ethnicity, and 11% reported Black race.
Findings from this survey provide insight into pandemic behaviors not consistently captured in EHRs, such as psychosocial aspects, including social and emotional support, loss of tangible and intangible resources, as well as COVID-19–related behaviors, such as social distancing and self-protective practices.1 MVP COVID-19 survey data combined with veteran EHRs, responses to other MVP surveys, and genetic data enable MVP researchers to better understand epidemiological, clinical, and psychosocial aspects of the disease. Future COVID-19 studies may use self-reported survey responses to enrich understanding about the effects of the disease on a veteran’s daily life, and possibly validate existing EHR COVID-19 diagnoses and hospitalization findings. This comprehensive data resource provides a unique opportunity to identify new targets for disease prevention, treatment, and management with an emphasis on individual variability in genes, environment, and lifestyle.
COVID-19 Research
In early 2020, the burden of COVID-19 on the US was unprecedented, and little was known about risk factors for severe COVID-19 and deaths. The MVP Phenomics team quickly responded with a large-scale phenome-wide association study (PheWAS) of >
To broaden disease progression data curation and fit the specific needs of the VA, we operationalized and validated the World Health Organization clinical severity scale and used VA EHR data to create the VA Severity Index for COVID-19 (VASIC).3 The VASIC category is now part of the MVP core data repository, where volumes of data from multiple activities are integrated through an automated process to create monthly research-ready data cubes. These activities include extensive data curation, mapping, phenotyping, and adjudication that are performed to curate oxygen supplementation status and other procedures related to treatment that are processed and understood in real time. The data cubes were provisioned to MVP COVID-19 researchers. In addition, the VASIC scale variable is now integrated within the larger VA system for all researchers to use as part of its wider COVID-19 initiative. The VA Centralized Interactive Phenomics Resource (CIPHER) phenomics library now hosts the details of VASIC, codes, metadata, and related COVID-19 data products for all VA communities. In partnership with CIPHER and other internal and external COVID-19 initiatives, the MVP continues to play an integral part for the VA and beyond in the development of a phenomics algorithm for long COVID, or post-acute COVID-19 syndrome (PACS).
Host Genetics in COVID-19
As the SARS-CoV-2 virus continued to spread globally, it became clear that the symptoms and severity of infection experienced by patients varied across a broad spectrum, from being asymptomatic carriers to experiencing severe symptoms in 1 or more organ systems in the body, resulting in death. This variability suggested that host genetics and other host factors may play a role in determining the severity of COVID-19 infection. The MVP dataset, with genetic and health information on > 600,000 MVP participants, provided an ideal dataset to explore host contributions to COVID-19.
In late spring 2020, the MVP executive committee issued a call to the MVP research community to propose study aims around the COVID-19 pandemic that could leverage the phenotypic and genetic data and resources. The MVP quickly formed 6 rapid-response scientific working groups. Their mission was to cultivate collaboration and inclusivity and to coordinate COVID-19 research questions. A steering committee composed of the MVP executive committee, staff from computational environments, working group cochairs, and an administrator, who was responsible for daily oversight of the working groups. In addition, the ORD COVID-19 steering committee reviewed and approved research activities to ensure scientific rigor, as well as alignment with overall ongoing research activities.
The MVP COVID-19 working groups included dozens of researchers who used MVP data to identify disease mechanisms; understand the impact of host genetics on susceptibility, morbidity, and mortality; and identify potential targets for treatments and therapies. The working groups were further supported by MVP analysts to work cross-functionally on genomics, phenomics, statistical genetics, and PheWAS. Each working group chair was responsible for prioritizing concepts and moving them forward in coordination with the MVP and ORD COVID-19 steering committees. An overview of the MVP COVID-19 working groups follows (Table).4-9
Druggable genome. This working group researched drug-repurposing opportunities to prevent severe COVID-19, defined as hospitalization with oxygen therapy (high flow), intubation, mechanical ventilation, vasopressors, dialysis, or death from COVID-19; and prevent complications in patients hospitalized by COVID-19.
Pharmacogenomics. This working group focused on 2 main aims: the impact of apolipoprotein L1 risk variants on acute kidney injury (AKI) and death in Black veterans with COVID-19; and pharmacogenetic analysis of remdesivir-induced liver chemistry abnormalities.
Disease mechanisms. Understanding the underlying pathways and mechanisms behind COVID-19 has been a difficult but important challenge overall in the scientific community. This working group investigated specific genetic markers and effects on COVID-19, including polygenic predisposition to venous thromboembolism associated with increased COVID-19 susceptibility; renal comorbidities and new AKI and unfavorable outcomes among COVID-19–positive sickle cell trait carriers; and mucin 5B, oligomeric mucus/gel-forming gene polymorphism, and protective effects in COVID-19 infection.
Genomics for risk prediction, polygenic risk scores, and mendelian randomization. Risk prediction for COVID-19 has been widely studied mostly aiming at comorbidities and preexisting conditions. The MVP cohort provided a unique opportunity to understand how genetic information can enhance our understanding of COVID-19 risk. This working group focused on: (1) ABO blood group typing and the protective effects of the O blood group on COVID-19 infection; (2) polygenic risk scores and COVID-19 outcomes; (3) human leukocyte antigen typing and COVID-19 outcomes; and (4) a transcriptome-wide association study of COVID-19–positive MVP participants.
Genome-Wide Association Study (GWAS) and Downstream Analysis. This working group performed GWAS of the main COVID-19 outcomes. Results from GWAS unveiled new genetic loci to suggest further investigation on these candidate genes. The results were used by other MVP COVID-19 working groups for their activities. The results also contributed to external collaborations, such as the COVID-19 Host Genetics Initiative.
COVID-19–Related PheWAS. This working group focused on understanding the potential clinical significance of genetic variants associated with susceptibility to, or outcomes of, COVID-19 infection. They worked to identify traits that share genetic variants associated with severe COVID-19 from the Host Genetics Initiative. The group also studied the phenotypic consequences of acquired mosaic chromosomal alterations with early data linking to COVID-19 susceptibility.
COVID-19 Research Partnerships
In 2016, the VA and DOE formed an interagency partnership known as Computational Health Analytics for Medical Precision to Improve Outcomes Now (CHAMPION) to demonstrate the power of combining the VA EHR system, MVP genetic data, and clinical research expertise with DOE high-performance computing infrastructure and artificial intelligence expertise. The VA EHR captures longitudinal care information on veterans with records that go back decades. Furthermore, the VA covers the costs of medications and
The DOE Oak Ridge National Laboratory (ORNL) in Tennessee securely maintains this rich database for the VA. The ORNL Summit supercomputer can complete trillions of calculations per second to provide critical and timely analyses, applying the most advanced and powerful artificial intelligence methods, which would not be possible in more conventional research settings. CHAMPION taught the VA and DOE how to bring their disparate research cultures together for innovative collaborative investigation. Moreover, this collaboration produced a cadre of VA and DOE scientists familiar with VA patient data and experienced in conducting joint research successfully and integrating omics data with clinical data for a better mechanistic understanding. Because of this preexisting collaboration between the VA and DOE, interagency teams were prepared at the start of the COVID-19 pandemic.10-15
Other recently completed studies have developed and validated short-term mortality indices in individuals with COVID-19 based on their preexisting conditions, assessed the generalizability of VA COVID-19 experiences to the US population, and evaluated the effectiveness of hydroxychloroquine with and without azithromycin in VA patients with COVID-19.12,15 A recent study demonstrated the benefit of prophylactic anticoagulation at initial hospitalization.14
The VA also provided the FDA with daily reports on aggregate VA COVID-19 cases and their distribution across the VA system, demographics of VA patients with COVID-19, and analyses of predictive models for positive test results and death. The VA regularly sent the FDA aggregated data showing patterns of medication use and retrospective analyses of the effectiveness of certain medications (including remdesivir and some antithrombotic agents). The FDA used these data along with other data to understand the scope of the pandemic and to predict drug shortages or needs for additional medical equipment, including ventilators.
Limitations
For the most part, MVP infrastructure and partnerships were efficiently leveraged to significantly advance our understanding of the biological basis of COVID-19 and to develop treatments and vaccines. However, there were a few limitations that may have slowed timely and optimal outcomes. An issue not limited to the MVP or VA was the continual evolution of the pandemic and its response. This included evolving definitions of disease, symptomatology, testing, vaccines, and public health recommendations. Keeping pace with the emerging knowledge from these domains was a struggle for the entire scientific community. A more discrete limitation was the number of participants in the MVP with positive COVID-19 test results and positive symptoms; however, this was mitigated by partnering with other groups like the COVID-19 Host Genetics Initiative to increase study participant numbers. Finally, there were logistical and regulatory challenges associated with coordination of national clinical trial recruitment across a VA system with > 100 discrete hospitals.
Conclusions
Having a centralized infrastructure for recruitment and enrollment, including a national research volunteer registry, information center, research staff, and coordinating centers, can allow for expedited enrollment in vaccine and treatment trials in the face of future public health emergencies. VA assets, including its rich EHR and MVP, the world’s largest genomic cohort, have contributed to improving our understanding and management of COVID-19.
The Million Veteran Program (MVP) was launched in 2011 by the US Department of Veterans Affairs (VA) to enroll at least 1 million veterans in a longitudinal cohort to better understand how genes, lifestyle, military experience, and environmental exposures interact to influence health and illness and ultimately enable precision health care. The MVP has established a national, centralized infrastructure for recruitment and enrollment, biospecimen and data collection and storage, data generation and curation, and secure data access. When the COVID-19 pandemic hit in 2020, the MVP was leveraged to support research utilizing the following key infrastructure components: (1) MVP recruitment and enrollment platform to provide support for COVID-19 vaccine and treatment trials and to collect COVID-19 data from MVP participants; (2) using MVP Phenomics for COVID-19 research data cleaning and curation, assisting with the development of a VA Severity Index for COVID-19, and forming 6 scientific working groups to coordinate COVID-19 research questions; and (3) the VA/MVP and US Department of Energy (DOE) partnership to assist in responding to COVID-19 research questions identified by the US Food and Drug Administration (FDA). This article describes these infrastructure components in more detail and highlights key findings from the MVP COVID-19 research efforts.
MVP Infrastructure
The Veterans Health Administration (VHA) Office of Research and Development (ORD) oversaw efforts to develop the VA Coronavirus Research Volunteer List (the COVID-19 registry). To support the registry, the MVP leveraged its infrastructure to facilitate a rapid response. The MVP is designed as a full-service and centralized recruitment and enrollment platform. This includes MVP office oversight; MVP coordinating centers that manage the centralized platform; an information center that handles inbound and outbound calls; an informatics system built for recruitment and enrollment monitoring and tracking; and a network of more than 70 participating MVP sites with dedicated staff to conduct recruitment and enrollment activities. The MVP used its informatics infrastructure to support secure data storage for the registry volunteer information. MVP coordinating center staff worked with the COVID-19 registry to invite > 125,000 MVP participants from approximately 20 MVP sites. Additionally, MVP information center staff made > 4000 calls to prospective registry volunteers. This work resulted in 1300 volunteers agreeing to be
New Data Collection
The MVP protocol was approved by the VA Central Institutional Review Board (IRB) in 2011. As part of initial enrollment in MVP, participants consented to recontact for additional self-report information along with access to their electronic health record (EHR). This allows for the linkage of EHR and survey response data, thus providing a comprehensive understanding of health history before and after a self-reported COVID-19 diagnosis. Between May 2020 and September 2021, the MVP COVID-19 survey was distributed to existing MVP participants via mail, telephone, and email with the ability to complete the survey by paper and pencil or through the MVP online system. Dissemination of the survey was approved by the VA Central IRB in 2020, with nearly 730,000 eligible MVP participants contacted. As of June 2022, 255,737 MVP participants (35% of the eligible cohort) had completed the survey; 86% completed a paper survey while 14% completed it online. Respondents were primarily older (≥ 65 years); 90% were male; close to 7% reported Hispanic ethnicity, and 11% reported Black race.
Findings from this survey provide insight into pandemic behaviors not consistently captured in EHRs, such as psychosocial aspects, including social and emotional support, loss of tangible and intangible resources, as well as COVID-19–related behaviors, such as social distancing and self-protective practices.1 MVP COVID-19 survey data combined with veteran EHRs, responses to other MVP surveys, and genetic data enable MVP researchers to better understand epidemiological, clinical, and psychosocial aspects of the disease. Future COVID-19 studies may use self-reported survey responses to enrich understanding about the effects of the disease on a veteran’s daily life, and possibly validate existing EHR COVID-19 diagnoses and hospitalization findings. This comprehensive data resource provides a unique opportunity to identify new targets for disease prevention, treatment, and management with an emphasis on individual variability in genes, environment, and lifestyle.
COVID-19 Research
In early 2020, the burden of COVID-19 on the US was unprecedented, and little was known about risk factors for severe COVID-19 and deaths. The MVP Phenomics team quickly responded with a large-scale phenome-wide association study (PheWAS) of >
To broaden disease progression data curation and fit the specific needs of the VA, we operationalized and validated the World Health Organization clinical severity scale and used VA EHR data to create the VA Severity Index for COVID-19 (VASIC).3 The VASIC category is now part of the MVP core data repository, where volumes of data from multiple activities are integrated through an automated process to create monthly research-ready data cubes. These activities include extensive data curation, mapping, phenotyping, and adjudication that are performed to curate oxygen supplementation status and other procedures related to treatment that are processed and understood in real time. The data cubes were provisioned to MVP COVID-19 researchers. In addition, the VASIC scale variable is now integrated within the larger VA system for all researchers to use as part of its wider COVID-19 initiative. The VA Centralized Interactive Phenomics Resource (CIPHER) phenomics library now hosts the details of VASIC, codes, metadata, and related COVID-19 data products for all VA communities. In partnership with CIPHER and other internal and external COVID-19 initiatives, the MVP continues to play an integral part for the VA and beyond in the development of a phenomics algorithm for long COVID, or post-acute COVID-19 syndrome (PACS).
Host Genetics in COVID-19
As the SARS-CoV-2 virus continued to spread globally, it became clear that the symptoms and severity of infection experienced by patients varied across a broad spectrum, from being asymptomatic carriers to experiencing severe symptoms in 1 or more organ systems in the body, resulting in death. This variability suggested that host genetics and other host factors may play a role in determining the severity of COVID-19 infection. The MVP dataset, with genetic and health information on > 600,000 MVP participants, provided an ideal dataset to explore host contributions to COVID-19.
In late spring 2020, the MVP executive committee issued a call to the MVP research community to propose study aims around the COVID-19 pandemic that could leverage the phenotypic and genetic data and resources. The MVP quickly formed 6 rapid-response scientific working groups. Their mission was to cultivate collaboration and inclusivity and to coordinate COVID-19 research questions. A steering committee composed of the MVP executive committee, staff from computational environments, working group cochairs, and an administrator, who was responsible for daily oversight of the working groups. In addition, the ORD COVID-19 steering committee reviewed and approved research activities to ensure scientific rigor, as well as alignment with overall ongoing research activities.
The MVP COVID-19 working groups included dozens of researchers who used MVP data to identify disease mechanisms; understand the impact of host genetics on susceptibility, morbidity, and mortality; and identify potential targets for treatments and therapies. The working groups were further supported by MVP analysts to work cross-functionally on genomics, phenomics, statistical genetics, and PheWAS. Each working group chair was responsible for prioritizing concepts and moving them forward in coordination with the MVP and ORD COVID-19 steering committees. An overview of the MVP COVID-19 working groups follows (Table).4-9
Druggable genome. This working group researched drug-repurposing opportunities to prevent severe COVID-19, defined as hospitalization with oxygen therapy (high flow), intubation, mechanical ventilation, vasopressors, dialysis, or death from COVID-19; and prevent complications in patients hospitalized by COVID-19.
Pharmacogenomics. This working group focused on 2 main aims: the impact of apolipoprotein L1 risk variants on acute kidney injury (AKI) and death in Black veterans with COVID-19; and pharmacogenetic analysis of remdesivir-induced liver chemistry abnormalities.
Disease mechanisms. Understanding the underlying pathways and mechanisms behind COVID-19 has been a difficult but important challenge overall in the scientific community. This working group investigated specific genetic markers and effects on COVID-19, including polygenic predisposition to venous thromboembolism associated with increased COVID-19 susceptibility; renal comorbidities and new AKI and unfavorable outcomes among COVID-19–positive sickle cell trait carriers; and mucin 5B, oligomeric mucus/gel-forming gene polymorphism, and protective effects in COVID-19 infection.
Genomics for risk prediction, polygenic risk scores, and mendelian randomization. Risk prediction for COVID-19 has been widely studied mostly aiming at comorbidities and preexisting conditions. The MVP cohort provided a unique opportunity to understand how genetic information can enhance our understanding of COVID-19 risk. This working group focused on: (1) ABO blood group typing and the protective effects of the O blood group on COVID-19 infection; (2) polygenic risk scores and COVID-19 outcomes; (3) human leukocyte antigen typing and COVID-19 outcomes; and (4) a transcriptome-wide association study of COVID-19–positive MVP participants.
Genome-Wide Association Study (GWAS) and Downstream Analysis. This working group performed GWAS of the main COVID-19 outcomes. Results from GWAS unveiled new genetic loci to suggest further investigation on these candidate genes. The results were used by other MVP COVID-19 working groups for their activities. The results also contributed to external collaborations, such as the COVID-19 Host Genetics Initiative.
COVID-19–Related PheWAS. This working group focused on understanding the potential clinical significance of genetic variants associated with susceptibility to, or outcomes of, COVID-19 infection. They worked to identify traits that share genetic variants associated with severe COVID-19 from the Host Genetics Initiative. The group also studied the phenotypic consequences of acquired mosaic chromosomal alterations with early data linking to COVID-19 susceptibility.
COVID-19 Research Partnerships
In 2016, the VA and DOE formed an interagency partnership known as Computational Health Analytics for Medical Precision to Improve Outcomes Now (CHAMPION) to demonstrate the power of combining the VA EHR system, MVP genetic data, and clinical research expertise with DOE high-performance computing infrastructure and artificial intelligence expertise. The VA EHR captures longitudinal care information on veterans with records that go back decades. Furthermore, the VA covers the costs of medications and
The DOE Oak Ridge National Laboratory (ORNL) in Tennessee securely maintains this rich database for the VA. The ORNL Summit supercomputer can complete trillions of calculations per second to provide critical and timely analyses, applying the most advanced and powerful artificial intelligence methods, which would not be possible in more conventional research settings. CHAMPION taught the VA and DOE how to bring their disparate research cultures together for innovative collaborative investigation. Moreover, this collaboration produced a cadre of VA and DOE scientists familiar with VA patient data and experienced in conducting joint research successfully and integrating omics data with clinical data for a better mechanistic understanding. Because of this preexisting collaboration between the VA and DOE, interagency teams were prepared at the start of the COVID-19 pandemic.10-15
Other recently completed studies have developed and validated short-term mortality indices in individuals with COVID-19 based on their preexisting conditions, assessed the generalizability of VA COVID-19 experiences to the US population, and evaluated the effectiveness of hydroxychloroquine with and without azithromycin in VA patients with COVID-19.12,15 A recent study demonstrated the benefit of prophylactic anticoagulation at initial hospitalization.14
The VA also provided the FDA with daily reports on aggregate VA COVID-19 cases and their distribution across the VA system, demographics of VA patients with COVID-19, and analyses of predictive models for positive test results and death. The VA regularly sent the FDA aggregated data showing patterns of medication use and retrospective analyses of the effectiveness of certain medications (including remdesivir and some antithrombotic agents). The FDA used these data along with other data to understand the scope of the pandemic and to predict drug shortages or needs for additional medical equipment, including ventilators.
Limitations
For the most part, MVP infrastructure and partnerships were efficiently leveraged to significantly advance our understanding of the biological basis of COVID-19 and to develop treatments and vaccines. However, there were a few limitations that may have slowed timely and optimal outcomes. An issue not limited to the MVP or VA was the continual evolution of the pandemic and its response. This included evolving definitions of disease, symptomatology, testing, vaccines, and public health recommendations. Keeping pace with the emerging knowledge from these domains was a struggle for the entire scientific community. A more discrete limitation was the number of participants in the MVP with positive COVID-19 test results and positive symptoms; however, this was mitigated by partnering with other groups like the COVID-19 Host Genetics Initiative to increase study participant numbers. Finally, there were logistical and regulatory challenges associated with coordination of national clinical trial recruitment across a VA system with > 100 discrete hospitals.
Conclusions
Having a centralized infrastructure for recruitment and enrollment, including a national research volunteer registry, information center, research staff, and coordinating centers, can allow for expedited enrollment in vaccine and treatment trials in the face of future public health emergencies. VA assets, including its rich EHR and MVP, the world’s largest genomic cohort, have contributed to improving our understanding and management of COVID-19.
1. Whitbourne SB, Nguyen XT, Song RJ, et al. Million Veteran Program’s response to COVID-19: survey development and preliminary findings. PLoS One. 2022;17(4):e0266381. doi:10.1371/journal.pone.0266381
2. Song RJ, Ho YL, Schubert P, et al. Phenome-wide association of 1809 phenotypes and COVID-19 disease progression in the Veterans Health Administration Million Veteran Program. PLoS One. 2021;16(5):e0251651. doi:10.1371/journal.pone.0251651
3. Galloway A, Park Y, Tanukonda V, et al. Impact of COVID-19 severity on long-term events in US veterans using the Veterans Affairs Severity Index for COVID-19 (VASIC). J Infect Dis. 2022;226(12):2113-2117. doi:10.1093/infdis/jiac182
4. Gaziano L, Giambartolomei C, Pereira AC, et al. Actionable druggable genome-wide Mendelian randomization identifies repurposing opportunities for COVID-19. Nat Med. 2021;27(4):668-676. doi:10.0138/s41591-021-01310-z
5. Hung AM, Sha SC, Bick AG, et al. APOL1 risk variants, acute kidney injury, and death in participants with African ancestry hospitalized with COVID-19 from the Million Veteran Program. JAMA Intern Med. 2022;182(4):386-395. doi:10.1001/jamainternmed.2021.8538
6. Verma A, Huffman JE, Gao L, et al. Association of kidney comorbidities and acute kidney failure with unfavorable outcomes after COVID-19 in individuals with the sickle cell trait. JAMA Intern Med. 2022;182(8):796-804. doi:10.1001/jamainternmed.2022.2141
7. Verma A, Tsao NL, Thomann LO, et al. A phenome-wide association study of genes associated with COVID-19 severity reveals shared genetics with complex diseases in the Million Veteran Program. PLoS Genet. 2022;18(4):e1010113. doi:10.1371/journal.pgen.1010113
8. Peloso GM, Tcheandjieu C, McGeary JE, et al. Genetic loci associated with COVID-19 positivity and hospitalization in White, Black, and Hispanic Veterans of the VA Million Veteran Program. Front Genetic. 2022;12:777076. doi:10.3389/fgene.2021.777076
9. Verma A, Minnier J, Wan ES, et al. A MUC5B gene polymorphism, rs35705950-T confers protective effects against COVID-19 hospitalization but not severe disease or mortality. Am J Respir Crit Care Med. 2022;182(8):796-804. doi:10.1164/rccm.202109-2166OC
10. Garvin MR, Alvarez C, Miller JI, et al. A mechanistic model and therapeutic interventions for COVID-19 involving a RAS-mediated bradykinin storm. Elife. 2020;e59177. doi:10.7554/eLife.59177
11. Rentsch CT, Kidwai-Khan F, Tate JP, et al. Patterns of COVID-19 testing and mortality by race and ethnicity among United States veterans: A nationwide cohort study. PLoS Med. 2020;17(9):e1003379. doi:10.1371/journal.pmed.1003379
12. King JT, Yoon JS, Rentsch CT, et al. Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: the Veterans Health Administration COVID-19 (VACO) Index. PLoS One. 2020;15(11):e0241825. doi:10.1371/journal.pone.0241825
13. Joubert W, Weighill D, Kainer D, et al. Attacking the opioid epidemic: determining the epistatic and pleiotropic genetic architectures for chronic pain and opioid addiction. SC18: International Conference for High Performance Computing, Networking, Storage and Analysis. Dallas, TX, USA, 2018:717-730. doi:10.1109/SC.2018.00060
14. Rentsch CT, Beckman JA, Tomlinson L, et al. Early initiation of prophylactic anticoagulation for prevention of COVID-19 mortality: a nationwide cohort study of hospitalized patients in the United States. BMJ. 2021;372:n311. doi:10.1136/bmj.n311
15. Gerlovin H, Posner DC, Ho YL, et al. Pharmacoepidemiology, machine learning, and COVID-19: an intent-to-treat analysis of hydroxychloroquine, with or without Azithromycin, and COVID-19 outcomes among hospitalized US Veterans. Am J Epidemiol. 2021;190(11): 2405-2419. doi:10.1093/aje/kwab183
1. Whitbourne SB, Nguyen XT, Song RJ, et al. Million Veteran Program’s response to COVID-19: survey development and preliminary findings. PLoS One. 2022;17(4):e0266381. doi:10.1371/journal.pone.0266381
2. Song RJ, Ho YL, Schubert P, et al. Phenome-wide association of 1809 phenotypes and COVID-19 disease progression in the Veterans Health Administration Million Veteran Program. PLoS One. 2021;16(5):e0251651. doi:10.1371/journal.pone.0251651
3. Galloway A, Park Y, Tanukonda V, et al. Impact of COVID-19 severity on long-term events in US veterans using the Veterans Affairs Severity Index for COVID-19 (VASIC). J Infect Dis. 2022;226(12):2113-2117. doi:10.1093/infdis/jiac182
4. Gaziano L, Giambartolomei C, Pereira AC, et al. Actionable druggable genome-wide Mendelian randomization identifies repurposing opportunities for COVID-19. Nat Med. 2021;27(4):668-676. doi:10.0138/s41591-021-01310-z
5. Hung AM, Sha SC, Bick AG, et al. APOL1 risk variants, acute kidney injury, and death in participants with African ancestry hospitalized with COVID-19 from the Million Veteran Program. JAMA Intern Med. 2022;182(4):386-395. doi:10.1001/jamainternmed.2021.8538
6. Verma A, Huffman JE, Gao L, et al. Association of kidney comorbidities and acute kidney failure with unfavorable outcomes after COVID-19 in individuals with the sickle cell trait. JAMA Intern Med. 2022;182(8):796-804. doi:10.1001/jamainternmed.2022.2141
7. Verma A, Tsao NL, Thomann LO, et al. A phenome-wide association study of genes associated with COVID-19 severity reveals shared genetics with complex diseases in the Million Veteran Program. PLoS Genet. 2022;18(4):e1010113. doi:10.1371/journal.pgen.1010113
8. Peloso GM, Tcheandjieu C, McGeary JE, et al. Genetic loci associated with COVID-19 positivity and hospitalization in White, Black, and Hispanic Veterans of the VA Million Veteran Program. Front Genetic. 2022;12:777076. doi:10.3389/fgene.2021.777076
9. Verma A, Minnier J, Wan ES, et al. A MUC5B gene polymorphism, rs35705950-T confers protective effects against COVID-19 hospitalization but not severe disease or mortality. Am J Respir Crit Care Med. 2022;182(8):796-804. doi:10.1164/rccm.202109-2166OC
10. Garvin MR, Alvarez C, Miller JI, et al. A mechanistic model and therapeutic interventions for COVID-19 involving a RAS-mediated bradykinin storm. Elife. 2020;e59177. doi:10.7554/eLife.59177
11. Rentsch CT, Kidwai-Khan F, Tate JP, et al. Patterns of COVID-19 testing and mortality by race and ethnicity among United States veterans: A nationwide cohort study. PLoS Med. 2020;17(9):e1003379. doi:10.1371/journal.pmed.1003379
12. King JT, Yoon JS, Rentsch CT, et al. Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: the Veterans Health Administration COVID-19 (VACO) Index. PLoS One. 2020;15(11):e0241825. doi:10.1371/journal.pone.0241825
13. Joubert W, Weighill D, Kainer D, et al. Attacking the opioid epidemic: determining the epistatic and pleiotropic genetic architectures for chronic pain and opioid addiction. SC18: International Conference for High Performance Computing, Networking, Storage and Analysis. Dallas, TX, USA, 2018:717-730. doi:10.1109/SC.2018.00060
14. Rentsch CT, Beckman JA, Tomlinson L, et al. Early initiation of prophylactic anticoagulation for prevention of COVID-19 mortality: a nationwide cohort study of hospitalized patients in the United States. BMJ. 2021;372:n311. doi:10.1136/bmj.n311
15. Gerlovin H, Posner DC, Ho YL, et al. Pharmacoepidemiology, machine learning, and COVID-19: an intent-to-treat analysis of hydroxychloroquine, with or without Azithromycin, and COVID-19 outcomes among hospitalized US Veterans. Am J Epidemiol. 2021;190(11): 2405-2419. doi:10.1093/aje/kwab183
VA Lessons From Partnering in COVID-19 Clinical Trials
The US Department of Veterans Affairs (VA), through its Office of Research and Development (ORD), supports an extensive and experienced clinical research enterprise, including the first multisite trials in the US.1 These resources contribute to the ORD support for the largest US integrated health care system, with a primary focus on the care and well-being of veterans. While the history of VA research has facilitated the creation of an experienced and organized research enterprise, the COVID-19 pandemic challenged VA to contribute even more significantly. These challenges became pronounced given the urgency associated with standing up VA sites for both therapeutic and vaccine trials.
VA Clinical Research Enterprise
The VA recognized an early need for an organized research response not only to address operational challenges resulting from COVID-19 but also ensure that the agency would be ready to support new scientific efforts focused specifically on the virus and related outcomes.2 As a result, the ORD took decisive action first by establishing itself as a central headquarters for VA COVID-19 research activities, and second, by leveraging existing resources, initiatives, and infrastructure to develop new mechanisms that would ensure that the VA was well positioned to develop or participate in research endeavors being driven by the VA as well federal, industry, and non-VA partners.
Prior to the pandemic, the ORD, through its Cooperative Studies Program (CSP), had strategies to address challenges associated with clinical trial startup and improved efficient conduct.3 For example, the VA Network of Dedicated Enrollment Sites (NODES) is a consortium of 23 VA medical centers (VAMCs) dedicated to rapid startup and recruitment into VA-sponsored clinical trials. NODES provides site-level expertise on clinical trial management, including troubleshooting challenges that may occur during clinical research execution.4 Another initiative, Access to Clinical Trials (ACT) for Veterans, engaged industry, academic, patient advocacy, and other partners to identify potential regulatory and operational hurdles to efficient startup activities specific to externally sponsored multisite clinical trials. Under ACT for Veterans, stakeholders emphasized the importance of developing a single VA point of contact for external partners to work with to more efficiently understand and navigate the VA system. In turn, such a resource could be designed to facilitate substantive research and long-term relationships with compatible external partners. Targeted to launch in April 2020, the Partnered Research Program (PRP) was expedited to respond to the pandemic.
During the pandemic, new VA efforts included the creation of the VA CoronavirUs Research and Efficacy Studies (VA CURES) network, initially established as a clinical trial master protocol framework to support and maximize VA-funded COVID-19 trial efficiency.5 VA CURES joined the consortium of trials networks funded by the National Heart, Lung, and Blood Institute. It began treatment trials under Accelerating COVID-19 Therapeutic Interventions and Vaccination (ACTIV), specifically ACTIV-4. The VA also partnered with the National Institutes of Allergy and Infectious Diseases (NIAID) by organizing the VA International Coordinating Center (VA ICC) for other ACTIV trials (ACTIV-2 and -3). When approached to startup studies that included veterans and the VA health care system, these capabilities comprised the VA research response.
A Need for a New Approach
As the impact of the pandemic expanded and the need for effective treatments and vaccines grew, national calls were made to assess the capabilities and readiness of available clinical trials networks. Additionally, the US Department of Health and Human Services Biomedical Advanced Research and Development Authority, ACTIV, NIAID Division of Clinical Research and Division of AIDS, and many pharmaceutical companies were starting to roll out trials of new therapeutics and vaccines. These groups approached the VA to help evaluate the safety and efficacy of several therapeutics and vaccines because they recognized several advantages of the VA enterprise, including its position as the nation’s largest integrated health care system, its diverse patient population, and its expertise in conducting clinical trials.
Although the VA was well positioned as an important player in a collaborative investigational approach to COVID-19 research, these trials required startup approaches that were significantly different from those it had employed in traditional, prepandemic, clinical research. Despite the VA being a single federal agency, each VAMC conducting research establishes its own practices to address both operational and regulatory requirements. This structure results in individual units that operate under different standard operating procedures. Efforts must be taken centrally to organize them into a singular network for the entire health care system. During a national crisis, when there was a need for rapid trial startup to answer safety and efficacy questions and participate under a common approach to protocol execution, this variability was neither manageable nor acceptable. Additionally, the intense resource demands associated with such research, coupled with frequent reporting requirements by VA leaders, Congress, and the White House, required that VAMCs function more like a single unit. Therefore, the ORD needed to develop VAMCs’ abilities to work collectively toward a common goal, share knowledge and experience, and capitalize on potential efficiencies concerning legal, regulatory, and operational processes.
Beginning August 2020, 39 VAMCs joined 7 large-scale collaborative COVID-19 therapeutic and vaccine trials. Through its COVID-19 Research Response Team, the ORD identified, engaged, and directed appropriate resources to support the VAMC under a centralized framework for study management (Table). Centralized management not only afforded VAMCs the opportunity to work more collectively and efficiently but also provided an important advantage by enabling the VA to collect and organize its experiences (and on occasion data) to provide a base for continual learning and improvement efforts. While others have described efforts undertaken across networks to advance learning health systems, the VA’s national scope and integration of research and clinical care allow greater opportunities to learn in a practical setting.6
Challenges and Best Practices
Using surveys, webinars, interviews, and observation from site and VA Central Office personnel, the ORD identified specific variables that prevented the VAMCs from quickly starting up as a clinical trial site. We also documented strategies, solutions, and recommendations for improving startup time lines. These were organized into 8 categories: (1) site infrastructure needs and capabilities; (2) study management roles and responsibilities; (3) educational resources and training; (4) local review requirements and procedures; (5) study design demands; (6) contracting and budgeting; (7) central-level systems and processes; and (8) communication between external partners and within the VA.
Site Infrastructure Needs and Capabilities
A primary impediment to rapid study startup was a lack of basic infrastructure, including staff, space, and the agility necessary for the changing demands of high-priority, high-enrolling trials. This observation is not unique to the VA.7 Initially, certain facilities located in hot spots where COVID-19 was more prevalent became high-interest targets for study placement, despite varying degrees of available research infrastructure. Furthermore, pandemic shutdowns and quarantines permitted fewer employees onsite. This resulted in inadequate staffing in personnel needed to support required startup activities and those needed to handle the high volume of study participants who were being recruited, screened, enrolled, and followed. Additionally, as clinical care needs and infection control practices were prioritized, clinical research space was often appropriated for these needs, making it difficult to find the space to conduct trials. Lastly, supply chain issues also posed unique challenges, sometimes making it difficult for participating VAMCs to obtain needed materials, such as IV solution bags of specific sizes and contents, safety injection needles, and IV line filters.
The VA was able to use central purchasing/contracting at coordinating centers or the VA Central Office to support investigators and assist with finding supplies and clinical research space. VAMCs with research operating budgets to cover startup costs were better positioned to handle funding delays. During the pandemic, the ORD further contracted to supply administrative support to research offices to address regulatory and other requirements needed for startup activities. The ability to expand such central contracts to procure clinical research staff and outpatient clinical research space may also prove useful in meeting key needs at a site.
Management Roles and Responsibilities
Ambiguous and variable roles and responsibilities among the various partners and stakeholders represented a challenge given the large-scale, national, or international operations involved in the trials. VA attempts to operate uniformly were further limited given that each sponsor or group had preferred methods for operating and/or organizing work under urgent time lines. For example, one trial involved a coordinating center, a contract research organization, and federal partners that each worked with individual sites. Consequently, VA study teams would receive messages that were conflicting or unclear.
The VA learned that studies need a single “source of truth” and/or central command structure in times of urgency. To mitigate conflicting messages, vaccine trials relied on a clearinghouse through the PRP to interpret requirements or work on behalf of all sites before key actions were taken. For studies with the NIAID, the VA relied on experienced staff at the CSP coordinating center at the Perry Point, Maryland, VAMC before beginning. This approach especially helped with the challenges of understaffing and sites’ lack of familiarity with complex platform trial designs and already-established network practices within the ACTIV-2 and ACTIV-3 studies.
Educational Resources and Training
Since VA participation in externally sponsored, multisite clinical trials traditionally relies on an individual VAMC study team and its local resources, transitioning to centralized approaches for COVID-19 multisite studies created barriers. Many VAMCs were unfamiliar with newer capabilities for more rapid regulatory reviews and approvals involving commercial institutional review boards (IRBs) and central VA information security and privacy reviews. While tools and resources were available to facilitate these processes, real-time use had not been fully tested. As a result, everyone had to learn as they went along.
The simultaneous establishment of workflows required the ORD to centralize operations and provide training and guidance to field personnel. Although many principal investigators and clinical research coordinators had trial experience, training required unlearning previous understandings of requirements to meet urgent time lines. ORD enterprise road maps, central tools, and training materials also were made available on a study-by-study basis. Open communication was vital to train on central study materials while opportunities to discuss, question, and share experiences and ideas were promoted. The ORD also sent regular emails to prepare for upcoming work and/or raise awareness of identified challenges.
Local Review Requirements/Procedures
The clinical trials were impacted by varying VAMC review requirements and approval processes. Although VA policy defines standard requirements, the timing and procedures are left to the individual facility to determine any local factors to accommodate and/or resource availability. While such an approach is well understood within the VA, external sponsors were not as familiar and assumed a more uniform approach across all sites. In response, some VAMCs established ad hoc research and development committee review procedures, allowing study teams to obtain the necessary reviews in a timely fashion. However, not all VAMCs had the infrastructure (especially when clinical personnel had been redeployed to other priorities) to respond with such agility. One critical role of the VA Central Office coordinating entities was to communicate and manage external sponsor and group expectations surrounding individual site review time lines. However, establishing policies and procedures that focus on streamlining local review processes helped to broadly mitigate the COVID-19 trial challenges.
Study Design Demands
The design of COVID-19 studies combined with the uncertainty of the pandemic required rapid protocol changes and adaptations that were often difficult to deliver. The multinetwork trials that the VA collaborated on were platform or master protocol designs. These designs emphasized overall goals (eg, treating patients requiring intensive care unit care). However, because this trial strategy also introduces complexities that may impact review and execution among those unfamiliar with it, there is a need for increased discussion and understanding of this methodology.8 For example, there can be shared control groups, reliance on specific criteria for halting because of safety or futility concerns, or continuation and expansion applied through an external review board. Delays may arise when changes to study protocols occur rapidly or frequently and necessitate new regulatory reviews, negotiation of new agreements, modifications to contracts, changes to entry criteria, etc.
While the VA has adopted a quality by design framework, VA investigators noted many missed opportunities related to looking at outcomes with new diagnostics, studies of serology, outcomes related to vaccinations, and understanding the natural history of disease in these trials.9 The limited opportunities for investigator input suggested that the advantages offered by platform designs were not maximized during pandemic-focused urgencies. It was unclear whether this barrier was created by a general lack of awareness by sponsors or a lack of opportunities. At the very least, quality by design approaches may help avoid redundancies in documentation or study processes at the central and site levels.
Contracting and Budgeting
Given external sponsorship of COVID-19 trials, efficient contracting and budgeting were critical for a rapid start up. The variability of processes associated with these trials created several challenges that were compounded by issues, such as site sub-agreements and budget documents that did not always go to the correct groups and individuals. Furthermore, the VA’s ability to use contracted resources (eg, tents, trailers, personnel) that external sponsors had built into their contracts was more difficult for VA as a federal agency governed by other statues and policies. This also put VAMCs at a disadvantage from a timing perspective, as the VA often required additional time to find equivalent solutions that met federal regulations.
Although the VA was able to establish contract solutions to some issues, time was still lost while working to secure initial funding. Additionally, for needs such as home phlebotomy—commended for convenience to veterans and research staff—and engaging a specialized research team in the Office of General Counsel, early awareness of protocol needs and sponsor solutions could allow VA to pursue alternatives sooner.
Central-Level Systems and Processes
Not all challenges were at the VAMC level. As the ORD explored solutions, it learned that various tools and study platforms were available but not considered. Applications, such as eConsent, and file-sharing platforms that met existing information security and privacy requirements were needed but had to comply with the Privacy Act of 1974, Federal Information Security Modernization Act, and other requirements. Using sponsor-provided devices, such as drug temperature monitoring equipment, required additional review to ensure that they met system requirements for a national health care system. In addition, the VA uses a clinical trials management review system; however, its implementation was new at the time these trials began. Furthermore, the system engaged with some commercial IRBs but not all. This resulted in additional delays as VAMCs and central resources worked to familiarize themselves with the system and procedures.
The ability to work collaboratively across the VA includes having a framework in which key startup processes are standardized. This allows for efficiency and minimizes variability. Also, all stakeholders should understand the importance of holding discussions to identify appropriate solutions, guidance, and instruction. Finally, the VA must strive to be more nimble when adapting technological, regulatory, and financial processes.
Internal and External Communication
The value of communication—both internal and external—cannot be understated. Minimizing confusion, managing expectations, and ensuring consistent messaging were essential for rapid trial execution. Despite being the second largest federal agency, the VA did not have a seat at the study leadership table for several protocols. When it joined later, several study aspects were set and/or difficult to revise. Challenges affecting time and securing resources have been noted. The ability to plan and then share expectations and responsibilities across and within the respective participating organizations early in the process was perhaps the single factor that was most addressable. The VA enterprise organization and integration with other units could accentuate key communications that would be essential in time-sensitive activities.
VA as a Partner for Future Research
Before the pandemic, the VA had already undertaken a path to enhance its ability to partner as part of the national biomedical research enterprise. The need for COVID-19 therapeutic and vaccine trials accelerated opportunities to plan and develop processes and capabilities to advance this path. As a key strength for VA scientific activities, clinical trials represent a primary medium by which to develop its partnerships. Learning and development have become part of a culture that expedites opportunities for veterans who actively seek ways to contribute to medical knowledge and treatments for their peers and the nation.
CONCLUSIONS
Challenges associated with rapid startup and completion of clinical trials have been discussed for some time. During the pandemic, needs and barriers were magnified because of the heightened urgency for evidence-based therapeutics and vaccines. While the VA faced similar problems as well as those specific to it as a health care system, it had the opportunity to learn and more systematically implement solutions to help in its partnered efforts.10 As an enterprise, the VA hopes to apply lessons learned, strategies, and best practices to further its goals to enhance veteran access to clinical trials and respond to any future need to quickly establish evidence bases in pandemics and other health emergencies that warrant the rapid implementation of research.
Acknowledgments
The activities reported here were supported by the US Department of Veterans Affairs, Office of Research and Development.
1. Hays MT; US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development. A historical look at the establishment of the Department of Veterans Affairs Research & Development Program. Accessed August 28, 2023. https://www.research.va.gov/pubs/docs/ORD-85yrHistory.pdf
2. Garcia AP, Huang GD, Arnheim L, Ramoni R, Clancy C. The VA research enterprise: a platform for national partnerships toward evidence building and scientific innovation. Fed Pract. 2023;40(suppl 5):S12-S17. doi:10.12788/fp.0425
3. Johnston SC, Lewis-Hall F, Bajpai A, et al. It’s time to harmonize clinical trial site standards. NAM Perspectives. October 9, 2017. Accessed August 28, 2023. https://nam.edu/wp-content/uploads/2017/10/Its-Time-to-Harmonize-Clinical-Trial-Site-1.pdf
4. Condon DL, Beck D, Kenworthy-Heinige T, et al. A cross-cutting approach to enhancing clinical trial site success: the Department of Veterans Affairs’ Network of Dedicated Enrollment Sites (NODES) model. Contemp Clin Trials Commun. 2017;6:78-84. Published 2017 Mar 29. doi:10.1016/j.conctc.2017.03.006
5. US Food and Drug Administration. Master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry. March 2022. Accessed August 23, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/master-protocols-efficient-clinical-trial-design-strategies-expedite-development-oncology-drugs-and
6. IOM Roundtable on Value & Science-Driven Care; Institute of Medicine. Continuous learning and improvement in health care. In: Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. National Academies Press (US); 2015:chap 2. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK284654 7. Institute of Medicine (US). Building an infrastructure to support clinical trials. In: Envisioning a Transformed Clinical Trials Enterprise in the United States. National Academies Press (US); 2012:chap 5. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK114656
8. Park JJH, Harari O, Dron L, Lester RT, Thorlund K, Mills EJ. An overview of platform trials with a checklist for clinical readers. J Clin Epidemiol. 2020;125:1-8. doi:10.1016/j.jclinepi.2020.04.025
9. Meeker-O’Connell A, Glessner C, Behm M, et al. Enhancing clinical evidence by proactively building quality into clinical trials. Clin Trials. 2016;13(4):439-444. doi:10.1177/1740774516643491
10. McClure J, Asghar A, Krajec A, et al. Clinical trial facilitators: a novel approach to support the execution of clinical research at the study site level. Contemp Clin Trials Commun. 2023;33:101106. doi:10.1016/j.conctc.2023.101106
The US Department of Veterans Affairs (VA), through its Office of Research and Development (ORD), supports an extensive and experienced clinical research enterprise, including the first multisite trials in the US.1 These resources contribute to the ORD support for the largest US integrated health care system, with a primary focus on the care and well-being of veterans. While the history of VA research has facilitated the creation of an experienced and organized research enterprise, the COVID-19 pandemic challenged VA to contribute even more significantly. These challenges became pronounced given the urgency associated with standing up VA sites for both therapeutic and vaccine trials.
VA Clinical Research Enterprise
The VA recognized an early need for an organized research response not only to address operational challenges resulting from COVID-19 but also ensure that the agency would be ready to support new scientific efforts focused specifically on the virus and related outcomes.2 As a result, the ORD took decisive action first by establishing itself as a central headquarters for VA COVID-19 research activities, and second, by leveraging existing resources, initiatives, and infrastructure to develop new mechanisms that would ensure that the VA was well positioned to develop or participate in research endeavors being driven by the VA as well federal, industry, and non-VA partners.
Prior to the pandemic, the ORD, through its Cooperative Studies Program (CSP), had strategies to address challenges associated with clinical trial startup and improved efficient conduct.3 For example, the VA Network of Dedicated Enrollment Sites (NODES) is a consortium of 23 VA medical centers (VAMCs) dedicated to rapid startup and recruitment into VA-sponsored clinical trials. NODES provides site-level expertise on clinical trial management, including troubleshooting challenges that may occur during clinical research execution.4 Another initiative, Access to Clinical Trials (ACT) for Veterans, engaged industry, academic, patient advocacy, and other partners to identify potential regulatory and operational hurdles to efficient startup activities specific to externally sponsored multisite clinical trials. Under ACT for Veterans, stakeholders emphasized the importance of developing a single VA point of contact for external partners to work with to more efficiently understand and navigate the VA system. In turn, such a resource could be designed to facilitate substantive research and long-term relationships with compatible external partners. Targeted to launch in April 2020, the Partnered Research Program (PRP) was expedited to respond to the pandemic.
During the pandemic, new VA efforts included the creation of the VA CoronavirUs Research and Efficacy Studies (VA CURES) network, initially established as a clinical trial master protocol framework to support and maximize VA-funded COVID-19 trial efficiency.5 VA CURES joined the consortium of trials networks funded by the National Heart, Lung, and Blood Institute. It began treatment trials under Accelerating COVID-19 Therapeutic Interventions and Vaccination (ACTIV), specifically ACTIV-4. The VA also partnered with the National Institutes of Allergy and Infectious Diseases (NIAID) by organizing the VA International Coordinating Center (VA ICC) for other ACTIV trials (ACTIV-2 and -3). When approached to startup studies that included veterans and the VA health care system, these capabilities comprised the VA research response.
A Need for a New Approach
As the impact of the pandemic expanded and the need for effective treatments and vaccines grew, national calls were made to assess the capabilities and readiness of available clinical trials networks. Additionally, the US Department of Health and Human Services Biomedical Advanced Research and Development Authority, ACTIV, NIAID Division of Clinical Research and Division of AIDS, and many pharmaceutical companies were starting to roll out trials of new therapeutics and vaccines. These groups approached the VA to help evaluate the safety and efficacy of several therapeutics and vaccines because they recognized several advantages of the VA enterprise, including its position as the nation’s largest integrated health care system, its diverse patient population, and its expertise in conducting clinical trials.
Although the VA was well positioned as an important player in a collaborative investigational approach to COVID-19 research, these trials required startup approaches that were significantly different from those it had employed in traditional, prepandemic, clinical research. Despite the VA being a single federal agency, each VAMC conducting research establishes its own practices to address both operational and regulatory requirements. This structure results in individual units that operate under different standard operating procedures. Efforts must be taken centrally to organize them into a singular network for the entire health care system. During a national crisis, when there was a need for rapid trial startup to answer safety and efficacy questions and participate under a common approach to protocol execution, this variability was neither manageable nor acceptable. Additionally, the intense resource demands associated with such research, coupled with frequent reporting requirements by VA leaders, Congress, and the White House, required that VAMCs function more like a single unit. Therefore, the ORD needed to develop VAMCs’ abilities to work collectively toward a common goal, share knowledge and experience, and capitalize on potential efficiencies concerning legal, regulatory, and operational processes.
Beginning August 2020, 39 VAMCs joined 7 large-scale collaborative COVID-19 therapeutic and vaccine trials. Through its COVID-19 Research Response Team, the ORD identified, engaged, and directed appropriate resources to support the VAMC under a centralized framework for study management (Table). Centralized management not only afforded VAMCs the opportunity to work more collectively and efficiently but also provided an important advantage by enabling the VA to collect and organize its experiences (and on occasion data) to provide a base for continual learning and improvement efforts. While others have described efforts undertaken across networks to advance learning health systems, the VA’s national scope and integration of research and clinical care allow greater opportunities to learn in a practical setting.6
Challenges and Best Practices
Using surveys, webinars, interviews, and observation from site and VA Central Office personnel, the ORD identified specific variables that prevented the VAMCs from quickly starting up as a clinical trial site. We also documented strategies, solutions, and recommendations for improving startup time lines. These were organized into 8 categories: (1) site infrastructure needs and capabilities; (2) study management roles and responsibilities; (3) educational resources and training; (4) local review requirements and procedures; (5) study design demands; (6) contracting and budgeting; (7) central-level systems and processes; and (8) communication between external partners and within the VA.
Site Infrastructure Needs and Capabilities
A primary impediment to rapid study startup was a lack of basic infrastructure, including staff, space, and the agility necessary for the changing demands of high-priority, high-enrolling trials. This observation is not unique to the VA.7 Initially, certain facilities located in hot spots where COVID-19 was more prevalent became high-interest targets for study placement, despite varying degrees of available research infrastructure. Furthermore, pandemic shutdowns and quarantines permitted fewer employees onsite. This resulted in inadequate staffing in personnel needed to support required startup activities and those needed to handle the high volume of study participants who were being recruited, screened, enrolled, and followed. Additionally, as clinical care needs and infection control practices were prioritized, clinical research space was often appropriated for these needs, making it difficult to find the space to conduct trials. Lastly, supply chain issues also posed unique challenges, sometimes making it difficult for participating VAMCs to obtain needed materials, such as IV solution bags of specific sizes and contents, safety injection needles, and IV line filters.
The VA was able to use central purchasing/contracting at coordinating centers or the VA Central Office to support investigators and assist with finding supplies and clinical research space. VAMCs with research operating budgets to cover startup costs were better positioned to handle funding delays. During the pandemic, the ORD further contracted to supply administrative support to research offices to address regulatory and other requirements needed for startup activities. The ability to expand such central contracts to procure clinical research staff and outpatient clinical research space may also prove useful in meeting key needs at a site.
Management Roles and Responsibilities
Ambiguous and variable roles and responsibilities among the various partners and stakeholders represented a challenge given the large-scale, national, or international operations involved in the trials. VA attempts to operate uniformly were further limited given that each sponsor or group had preferred methods for operating and/or organizing work under urgent time lines. For example, one trial involved a coordinating center, a contract research organization, and federal partners that each worked with individual sites. Consequently, VA study teams would receive messages that were conflicting or unclear.
The VA learned that studies need a single “source of truth” and/or central command structure in times of urgency. To mitigate conflicting messages, vaccine trials relied on a clearinghouse through the PRP to interpret requirements or work on behalf of all sites before key actions were taken. For studies with the NIAID, the VA relied on experienced staff at the CSP coordinating center at the Perry Point, Maryland, VAMC before beginning. This approach especially helped with the challenges of understaffing and sites’ lack of familiarity with complex platform trial designs and already-established network practices within the ACTIV-2 and ACTIV-3 studies.
Educational Resources and Training
Since VA participation in externally sponsored, multisite clinical trials traditionally relies on an individual VAMC study team and its local resources, transitioning to centralized approaches for COVID-19 multisite studies created barriers. Many VAMCs were unfamiliar with newer capabilities for more rapid regulatory reviews and approvals involving commercial institutional review boards (IRBs) and central VA information security and privacy reviews. While tools and resources were available to facilitate these processes, real-time use had not been fully tested. As a result, everyone had to learn as they went along.
The simultaneous establishment of workflows required the ORD to centralize operations and provide training and guidance to field personnel. Although many principal investigators and clinical research coordinators had trial experience, training required unlearning previous understandings of requirements to meet urgent time lines. ORD enterprise road maps, central tools, and training materials also were made available on a study-by-study basis. Open communication was vital to train on central study materials while opportunities to discuss, question, and share experiences and ideas were promoted. The ORD also sent regular emails to prepare for upcoming work and/or raise awareness of identified challenges.
Local Review Requirements/Procedures
The clinical trials were impacted by varying VAMC review requirements and approval processes. Although VA policy defines standard requirements, the timing and procedures are left to the individual facility to determine any local factors to accommodate and/or resource availability. While such an approach is well understood within the VA, external sponsors were not as familiar and assumed a more uniform approach across all sites. In response, some VAMCs established ad hoc research and development committee review procedures, allowing study teams to obtain the necessary reviews in a timely fashion. However, not all VAMCs had the infrastructure (especially when clinical personnel had been redeployed to other priorities) to respond with such agility. One critical role of the VA Central Office coordinating entities was to communicate and manage external sponsor and group expectations surrounding individual site review time lines. However, establishing policies and procedures that focus on streamlining local review processes helped to broadly mitigate the COVID-19 trial challenges.
Study Design Demands
The design of COVID-19 studies combined with the uncertainty of the pandemic required rapid protocol changes and adaptations that were often difficult to deliver. The multinetwork trials that the VA collaborated on were platform or master protocol designs. These designs emphasized overall goals (eg, treating patients requiring intensive care unit care). However, because this trial strategy also introduces complexities that may impact review and execution among those unfamiliar with it, there is a need for increased discussion and understanding of this methodology.8 For example, there can be shared control groups, reliance on specific criteria for halting because of safety or futility concerns, or continuation and expansion applied through an external review board. Delays may arise when changes to study protocols occur rapidly or frequently and necessitate new regulatory reviews, negotiation of new agreements, modifications to contracts, changes to entry criteria, etc.
While the VA has adopted a quality by design framework, VA investigators noted many missed opportunities related to looking at outcomes with new diagnostics, studies of serology, outcomes related to vaccinations, and understanding the natural history of disease in these trials.9 The limited opportunities for investigator input suggested that the advantages offered by platform designs were not maximized during pandemic-focused urgencies. It was unclear whether this barrier was created by a general lack of awareness by sponsors or a lack of opportunities. At the very least, quality by design approaches may help avoid redundancies in documentation or study processes at the central and site levels.
Contracting and Budgeting
Given external sponsorship of COVID-19 trials, efficient contracting and budgeting were critical for a rapid start up. The variability of processes associated with these trials created several challenges that were compounded by issues, such as site sub-agreements and budget documents that did not always go to the correct groups and individuals. Furthermore, the VA’s ability to use contracted resources (eg, tents, trailers, personnel) that external sponsors had built into their contracts was more difficult for VA as a federal agency governed by other statues and policies. This also put VAMCs at a disadvantage from a timing perspective, as the VA often required additional time to find equivalent solutions that met federal regulations.
Although the VA was able to establish contract solutions to some issues, time was still lost while working to secure initial funding. Additionally, for needs such as home phlebotomy—commended for convenience to veterans and research staff—and engaging a specialized research team in the Office of General Counsel, early awareness of protocol needs and sponsor solutions could allow VA to pursue alternatives sooner.
Central-Level Systems and Processes
Not all challenges were at the VAMC level. As the ORD explored solutions, it learned that various tools and study platforms were available but not considered. Applications, such as eConsent, and file-sharing platforms that met existing information security and privacy requirements were needed but had to comply with the Privacy Act of 1974, Federal Information Security Modernization Act, and other requirements. Using sponsor-provided devices, such as drug temperature monitoring equipment, required additional review to ensure that they met system requirements for a national health care system. In addition, the VA uses a clinical trials management review system; however, its implementation was new at the time these trials began. Furthermore, the system engaged with some commercial IRBs but not all. This resulted in additional delays as VAMCs and central resources worked to familiarize themselves with the system and procedures.
The ability to work collaboratively across the VA includes having a framework in which key startup processes are standardized. This allows for efficiency and minimizes variability. Also, all stakeholders should understand the importance of holding discussions to identify appropriate solutions, guidance, and instruction. Finally, the VA must strive to be more nimble when adapting technological, regulatory, and financial processes.
Internal and External Communication
The value of communication—both internal and external—cannot be understated. Minimizing confusion, managing expectations, and ensuring consistent messaging were essential for rapid trial execution. Despite being the second largest federal agency, the VA did not have a seat at the study leadership table for several protocols. When it joined later, several study aspects were set and/or difficult to revise. Challenges affecting time and securing resources have been noted. The ability to plan and then share expectations and responsibilities across and within the respective participating organizations early in the process was perhaps the single factor that was most addressable. The VA enterprise organization and integration with other units could accentuate key communications that would be essential in time-sensitive activities.
VA as a Partner for Future Research
Before the pandemic, the VA had already undertaken a path to enhance its ability to partner as part of the national biomedical research enterprise. The need for COVID-19 therapeutic and vaccine trials accelerated opportunities to plan and develop processes and capabilities to advance this path. As a key strength for VA scientific activities, clinical trials represent a primary medium by which to develop its partnerships. Learning and development have become part of a culture that expedites opportunities for veterans who actively seek ways to contribute to medical knowledge and treatments for their peers and the nation.
CONCLUSIONS
Challenges associated with rapid startup and completion of clinical trials have been discussed for some time. During the pandemic, needs and barriers were magnified because of the heightened urgency for evidence-based therapeutics and vaccines. While the VA faced similar problems as well as those specific to it as a health care system, it had the opportunity to learn and more systematically implement solutions to help in its partnered efforts.10 As an enterprise, the VA hopes to apply lessons learned, strategies, and best practices to further its goals to enhance veteran access to clinical trials and respond to any future need to quickly establish evidence bases in pandemics and other health emergencies that warrant the rapid implementation of research.
Acknowledgments
The activities reported here were supported by the US Department of Veterans Affairs, Office of Research and Development.
The US Department of Veterans Affairs (VA), through its Office of Research and Development (ORD), supports an extensive and experienced clinical research enterprise, including the first multisite trials in the US.1 These resources contribute to the ORD support for the largest US integrated health care system, with a primary focus on the care and well-being of veterans. While the history of VA research has facilitated the creation of an experienced and organized research enterprise, the COVID-19 pandemic challenged VA to contribute even more significantly. These challenges became pronounced given the urgency associated with standing up VA sites for both therapeutic and vaccine trials.
VA Clinical Research Enterprise
The VA recognized an early need for an organized research response not only to address operational challenges resulting from COVID-19 but also ensure that the agency would be ready to support new scientific efforts focused specifically on the virus and related outcomes.2 As a result, the ORD took decisive action first by establishing itself as a central headquarters for VA COVID-19 research activities, and second, by leveraging existing resources, initiatives, and infrastructure to develop new mechanisms that would ensure that the VA was well positioned to develop or participate in research endeavors being driven by the VA as well federal, industry, and non-VA partners.
Prior to the pandemic, the ORD, through its Cooperative Studies Program (CSP), had strategies to address challenges associated with clinical trial startup and improved efficient conduct.3 For example, the VA Network of Dedicated Enrollment Sites (NODES) is a consortium of 23 VA medical centers (VAMCs) dedicated to rapid startup and recruitment into VA-sponsored clinical trials. NODES provides site-level expertise on clinical trial management, including troubleshooting challenges that may occur during clinical research execution.4 Another initiative, Access to Clinical Trials (ACT) for Veterans, engaged industry, academic, patient advocacy, and other partners to identify potential regulatory and operational hurdles to efficient startup activities specific to externally sponsored multisite clinical trials. Under ACT for Veterans, stakeholders emphasized the importance of developing a single VA point of contact for external partners to work with to more efficiently understand and navigate the VA system. In turn, such a resource could be designed to facilitate substantive research and long-term relationships with compatible external partners. Targeted to launch in April 2020, the Partnered Research Program (PRP) was expedited to respond to the pandemic.
During the pandemic, new VA efforts included the creation of the VA CoronavirUs Research and Efficacy Studies (VA CURES) network, initially established as a clinical trial master protocol framework to support and maximize VA-funded COVID-19 trial efficiency.5 VA CURES joined the consortium of trials networks funded by the National Heart, Lung, and Blood Institute. It began treatment trials under Accelerating COVID-19 Therapeutic Interventions and Vaccination (ACTIV), specifically ACTIV-4. The VA also partnered with the National Institutes of Allergy and Infectious Diseases (NIAID) by organizing the VA International Coordinating Center (VA ICC) for other ACTIV trials (ACTIV-2 and -3). When approached to startup studies that included veterans and the VA health care system, these capabilities comprised the VA research response.
A Need for a New Approach
As the impact of the pandemic expanded and the need for effective treatments and vaccines grew, national calls were made to assess the capabilities and readiness of available clinical trials networks. Additionally, the US Department of Health and Human Services Biomedical Advanced Research and Development Authority, ACTIV, NIAID Division of Clinical Research and Division of AIDS, and many pharmaceutical companies were starting to roll out trials of new therapeutics and vaccines. These groups approached the VA to help evaluate the safety and efficacy of several therapeutics and vaccines because they recognized several advantages of the VA enterprise, including its position as the nation’s largest integrated health care system, its diverse patient population, and its expertise in conducting clinical trials.
Although the VA was well positioned as an important player in a collaborative investigational approach to COVID-19 research, these trials required startup approaches that were significantly different from those it had employed in traditional, prepandemic, clinical research. Despite the VA being a single federal agency, each VAMC conducting research establishes its own practices to address both operational and regulatory requirements. This structure results in individual units that operate under different standard operating procedures. Efforts must be taken centrally to organize them into a singular network for the entire health care system. During a national crisis, when there was a need for rapid trial startup to answer safety and efficacy questions and participate under a common approach to protocol execution, this variability was neither manageable nor acceptable. Additionally, the intense resource demands associated with such research, coupled with frequent reporting requirements by VA leaders, Congress, and the White House, required that VAMCs function more like a single unit. Therefore, the ORD needed to develop VAMCs’ abilities to work collectively toward a common goal, share knowledge and experience, and capitalize on potential efficiencies concerning legal, regulatory, and operational processes.
Beginning August 2020, 39 VAMCs joined 7 large-scale collaborative COVID-19 therapeutic and vaccine trials. Through its COVID-19 Research Response Team, the ORD identified, engaged, and directed appropriate resources to support the VAMC under a centralized framework for study management (Table). Centralized management not only afforded VAMCs the opportunity to work more collectively and efficiently but also provided an important advantage by enabling the VA to collect and organize its experiences (and on occasion data) to provide a base for continual learning and improvement efforts. While others have described efforts undertaken across networks to advance learning health systems, the VA’s national scope and integration of research and clinical care allow greater opportunities to learn in a practical setting.6
Challenges and Best Practices
Using surveys, webinars, interviews, and observation from site and VA Central Office personnel, the ORD identified specific variables that prevented the VAMCs from quickly starting up as a clinical trial site. We also documented strategies, solutions, and recommendations for improving startup time lines. These were organized into 8 categories: (1) site infrastructure needs and capabilities; (2) study management roles and responsibilities; (3) educational resources and training; (4) local review requirements and procedures; (5) study design demands; (6) contracting and budgeting; (7) central-level systems and processes; and (8) communication between external partners and within the VA.
Site Infrastructure Needs and Capabilities
A primary impediment to rapid study startup was a lack of basic infrastructure, including staff, space, and the agility necessary for the changing demands of high-priority, high-enrolling trials. This observation is not unique to the VA.7 Initially, certain facilities located in hot spots where COVID-19 was more prevalent became high-interest targets for study placement, despite varying degrees of available research infrastructure. Furthermore, pandemic shutdowns and quarantines permitted fewer employees onsite. This resulted in inadequate staffing in personnel needed to support required startup activities and those needed to handle the high volume of study participants who were being recruited, screened, enrolled, and followed. Additionally, as clinical care needs and infection control practices were prioritized, clinical research space was often appropriated for these needs, making it difficult to find the space to conduct trials. Lastly, supply chain issues also posed unique challenges, sometimes making it difficult for participating VAMCs to obtain needed materials, such as IV solution bags of specific sizes and contents, safety injection needles, and IV line filters.
The VA was able to use central purchasing/contracting at coordinating centers or the VA Central Office to support investigators and assist with finding supplies and clinical research space. VAMCs with research operating budgets to cover startup costs were better positioned to handle funding delays. During the pandemic, the ORD further contracted to supply administrative support to research offices to address regulatory and other requirements needed for startup activities. The ability to expand such central contracts to procure clinical research staff and outpatient clinical research space may also prove useful in meeting key needs at a site.
Management Roles and Responsibilities
Ambiguous and variable roles and responsibilities among the various partners and stakeholders represented a challenge given the large-scale, national, or international operations involved in the trials. VA attempts to operate uniformly were further limited given that each sponsor or group had preferred methods for operating and/or organizing work under urgent time lines. For example, one trial involved a coordinating center, a contract research organization, and federal partners that each worked with individual sites. Consequently, VA study teams would receive messages that were conflicting or unclear.
The VA learned that studies need a single “source of truth” and/or central command structure in times of urgency. To mitigate conflicting messages, vaccine trials relied on a clearinghouse through the PRP to interpret requirements or work on behalf of all sites before key actions were taken. For studies with the NIAID, the VA relied on experienced staff at the CSP coordinating center at the Perry Point, Maryland, VAMC before beginning. This approach especially helped with the challenges of understaffing and sites’ lack of familiarity with complex platform trial designs and already-established network practices within the ACTIV-2 and ACTIV-3 studies.
Educational Resources and Training
Since VA participation in externally sponsored, multisite clinical trials traditionally relies on an individual VAMC study team and its local resources, transitioning to centralized approaches for COVID-19 multisite studies created barriers. Many VAMCs were unfamiliar with newer capabilities for more rapid regulatory reviews and approvals involving commercial institutional review boards (IRBs) and central VA information security and privacy reviews. While tools and resources were available to facilitate these processes, real-time use had not been fully tested. As a result, everyone had to learn as they went along.
The simultaneous establishment of workflows required the ORD to centralize operations and provide training and guidance to field personnel. Although many principal investigators and clinical research coordinators had trial experience, training required unlearning previous understandings of requirements to meet urgent time lines. ORD enterprise road maps, central tools, and training materials also were made available on a study-by-study basis. Open communication was vital to train on central study materials while opportunities to discuss, question, and share experiences and ideas were promoted. The ORD also sent regular emails to prepare for upcoming work and/or raise awareness of identified challenges.
Local Review Requirements/Procedures
The clinical trials were impacted by varying VAMC review requirements and approval processes. Although VA policy defines standard requirements, the timing and procedures are left to the individual facility to determine any local factors to accommodate and/or resource availability. While such an approach is well understood within the VA, external sponsors were not as familiar and assumed a more uniform approach across all sites. In response, some VAMCs established ad hoc research and development committee review procedures, allowing study teams to obtain the necessary reviews in a timely fashion. However, not all VAMCs had the infrastructure (especially when clinical personnel had been redeployed to other priorities) to respond with such agility. One critical role of the VA Central Office coordinating entities was to communicate and manage external sponsor and group expectations surrounding individual site review time lines. However, establishing policies and procedures that focus on streamlining local review processes helped to broadly mitigate the COVID-19 trial challenges.
Study Design Demands
The design of COVID-19 studies combined with the uncertainty of the pandemic required rapid protocol changes and adaptations that were often difficult to deliver. The multinetwork trials that the VA collaborated on were platform or master protocol designs. These designs emphasized overall goals (eg, treating patients requiring intensive care unit care). However, because this trial strategy also introduces complexities that may impact review and execution among those unfamiliar with it, there is a need for increased discussion and understanding of this methodology.8 For example, there can be shared control groups, reliance on specific criteria for halting because of safety or futility concerns, or continuation and expansion applied through an external review board. Delays may arise when changes to study protocols occur rapidly or frequently and necessitate new regulatory reviews, negotiation of new agreements, modifications to contracts, changes to entry criteria, etc.
While the VA has adopted a quality by design framework, VA investigators noted many missed opportunities related to looking at outcomes with new diagnostics, studies of serology, outcomes related to vaccinations, and understanding the natural history of disease in these trials.9 The limited opportunities for investigator input suggested that the advantages offered by platform designs were not maximized during pandemic-focused urgencies. It was unclear whether this barrier was created by a general lack of awareness by sponsors or a lack of opportunities. At the very least, quality by design approaches may help avoid redundancies in documentation or study processes at the central and site levels.
Contracting and Budgeting
Given external sponsorship of COVID-19 trials, efficient contracting and budgeting were critical for a rapid start up. The variability of processes associated with these trials created several challenges that were compounded by issues, such as site sub-agreements and budget documents that did not always go to the correct groups and individuals. Furthermore, the VA’s ability to use contracted resources (eg, tents, trailers, personnel) that external sponsors had built into their contracts was more difficult for VA as a federal agency governed by other statues and policies. This also put VAMCs at a disadvantage from a timing perspective, as the VA often required additional time to find equivalent solutions that met federal regulations.
Although the VA was able to establish contract solutions to some issues, time was still lost while working to secure initial funding. Additionally, for needs such as home phlebotomy—commended for convenience to veterans and research staff—and engaging a specialized research team in the Office of General Counsel, early awareness of protocol needs and sponsor solutions could allow VA to pursue alternatives sooner.
Central-Level Systems and Processes
Not all challenges were at the VAMC level. As the ORD explored solutions, it learned that various tools and study platforms were available but not considered. Applications, such as eConsent, and file-sharing platforms that met existing information security and privacy requirements were needed but had to comply with the Privacy Act of 1974, Federal Information Security Modernization Act, and other requirements. Using sponsor-provided devices, such as drug temperature monitoring equipment, required additional review to ensure that they met system requirements for a national health care system. In addition, the VA uses a clinical trials management review system; however, its implementation was new at the time these trials began. Furthermore, the system engaged with some commercial IRBs but not all. This resulted in additional delays as VAMCs and central resources worked to familiarize themselves with the system and procedures.
The ability to work collaboratively across the VA includes having a framework in which key startup processes are standardized. This allows for efficiency and minimizes variability. Also, all stakeholders should understand the importance of holding discussions to identify appropriate solutions, guidance, and instruction. Finally, the VA must strive to be more nimble when adapting technological, regulatory, and financial processes.
Internal and External Communication
The value of communication—both internal and external—cannot be understated. Minimizing confusion, managing expectations, and ensuring consistent messaging were essential for rapid trial execution. Despite being the second largest federal agency, the VA did not have a seat at the study leadership table for several protocols. When it joined later, several study aspects were set and/or difficult to revise. Challenges affecting time and securing resources have been noted. The ability to plan and then share expectations and responsibilities across and within the respective participating organizations early in the process was perhaps the single factor that was most addressable. The VA enterprise organization and integration with other units could accentuate key communications that would be essential in time-sensitive activities.
VA as a Partner for Future Research
Before the pandemic, the VA had already undertaken a path to enhance its ability to partner as part of the national biomedical research enterprise. The need for COVID-19 therapeutic and vaccine trials accelerated opportunities to plan and develop processes and capabilities to advance this path. As a key strength for VA scientific activities, clinical trials represent a primary medium by which to develop its partnerships. Learning and development have become part of a culture that expedites opportunities for veterans who actively seek ways to contribute to medical knowledge and treatments for their peers and the nation.
CONCLUSIONS
Challenges associated with rapid startup and completion of clinical trials have been discussed for some time. During the pandemic, needs and barriers were magnified because of the heightened urgency for evidence-based therapeutics and vaccines. While the VA faced similar problems as well as those specific to it as a health care system, it had the opportunity to learn and more systematically implement solutions to help in its partnered efforts.10 As an enterprise, the VA hopes to apply lessons learned, strategies, and best practices to further its goals to enhance veteran access to clinical trials and respond to any future need to quickly establish evidence bases in pandemics and other health emergencies that warrant the rapid implementation of research.
Acknowledgments
The activities reported here were supported by the US Department of Veterans Affairs, Office of Research and Development.
1. Hays MT; US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development. A historical look at the establishment of the Department of Veterans Affairs Research & Development Program. Accessed August 28, 2023. https://www.research.va.gov/pubs/docs/ORD-85yrHistory.pdf
2. Garcia AP, Huang GD, Arnheim L, Ramoni R, Clancy C. The VA research enterprise: a platform for national partnerships toward evidence building and scientific innovation. Fed Pract. 2023;40(suppl 5):S12-S17. doi:10.12788/fp.0425
3. Johnston SC, Lewis-Hall F, Bajpai A, et al. It’s time to harmonize clinical trial site standards. NAM Perspectives. October 9, 2017. Accessed August 28, 2023. https://nam.edu/wp-content/uploads/2017/10/Its-Time-to-Harmonize-Clinical-Trial-Site-1.pdf
4. Condon DL, Beck D, Kenworthy-Heinige T, et al. A cross-cutting approach to enhancing clinical trial site success: the Department of Veterans Affairs’ Network of Dedicated Enrollment Sites (NODES) model. Contemp Clin Trials Commun. 2017;6:78-84. Published 2017 Mar 29. doi:10.1016/j.conctc.2017.03.006
5. US Food and Drug Administration. Master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry. March 2022. Accessed August 23, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/master-protocols-efficient-clinical-trial-design-strategies-expedite-development-oncology-drugs-and
6. IOM Roundtable on Value & Science-Driven Care; Institute of Medicine. Continuous learning and improvement in health care. In: Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. National Academies Press (US); 2015:chap 2. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK284654 7. Institute of Medicine (US). Building an infrastructure to support clinical trials. In: Envisioning a Transformed Clinical Trials Enterprise in the United States. National Academies Press (US); 2012:chap 5. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK114656
8. Park JJH, Harari O, Dron L, Lester RT, Thorlund K, Mills EJ. An overview of platform trials with a checklist for clinical readers. J Clin Epidemiol. 2020;125:1-8. doi:10.1016/j.jclinepi.2020.04.025
9. Meeker-O’Connell A, Glessner C, Behm M, et al. Enhancing clinical evidence by proactively building quality into clinical trials. Clin Trials. 2016;13(4):439-444. doi:10.1177/1740774516643491
10. McClure J, Asghar A, Krajec A, et al. Clinical trial facilitators: a novel approach to support the execution of clinical research at the study site level. Contemp Clin Trials Commun. 2023;33:101106. doi:10.1016/j.conctc.2023.101106
1. Hays MT; US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development. A historical look at the establishment of the Department of Veterans Affairs Research & Development Program. Accessed August 28, 2023. https://www.research.va.gov/pubs/docs/ORD-85yrHistory.pdf
2. Garcia AP, Huang GD, Arnheim L, Ramoni R, Clancy C. The VA research enterprise: a platform for national partnerships toward evidence building and scientific innovation. Fed Pract. 2023;40(suppl 5):S12-S17. doi:10.12788/fp.0425
3. Johnston SC, Lewis-Hall F, Bajpai A, et al. It’s time to harmonize clinical trial site standards. NAM Perspectives. October 9, 2017. Accessed August 28, 2023. https://nam.edu/wp-content/uploads/2017/10/Its-Time-to-Harmonize-Clinical-Trial-Site-1.pdf
4. Condon DL, Beck D, Kenworthy-Heinige T, et al. A cross-cutting approach to enhancing clinical trial site success: the Department of Veterans Affairs’ Network of Dedicated Enrollment Sites (NODES) model. Contemp Clin Trials Commun. 2017;6:78-84. Published 2017 Mar 29. doi:10.1016/j.conctc.2017.03.006
5. US Food and Drug Administration. Master protocols: efficient clinical trial design strategies to expedite development of oncology drugs and biologics guidance for industry. March 2022. Accessed August 23, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/master-protocols-efficient-clinical-trial-design-strategies-expedite-development-oncology-drugs-and
6. IOM Roundtable on Value & Science-Driven Care; Institute of Medicine. Continuous learning and improvement in health care. In: Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. National Academies Press (US); 2015:chap 2. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK284654 7. Institute of Medicine (US). Building an infrastructure to support clinical trials. In: Envisioning a Transformed Clinical Trials Enterprise in the United States. National Academies Press (US); 2012:chap 5. Accessed August 28, 2023. https://www.ncbi.nlm.nih.gov/books/NBK114656
8. Park JJH, Harari O, Dron L, Lester RT, Thorlund K, Mills EJ. An overview of platform trials with a checklist for clinical readers. J Clin Epidemiol. 2020;125:1-8. doi:10.1016/j.jclinepi.2020.04.025
9. Meeker-O’Connell A, Glessner C, Behm M, et al. Enhancing clinical evidence by proactively building quality into clinical trials. Clin Trials. 2016;13(4):439-444. doi:10.1177/1740774516643491
10. McClure J, Asghar A, Krajec A, et al. Clinical trial facilitators: a novel approach to support the execution of clinical research at the study site level. Contemp Clin Trials Commun. 2023;33:101106. doi:10.1016/j.conctc.2023.101106