Original Research

Spinal cord injuries (SCI) are common in veteran populations.¹ Veterans with spinal cord injuries and disorders (SCI/D) also may have concurrent sleep disturbances. Spinal cord injury typically causes spasticity.^2,3 Hypersensitivity of the flexor reflex pathways is believed to cause painful muscle spasms in patients with SCI.⁴ Neuropathic pain at or below the level of the lesion also is common.

Restless legs syndrome (RLS) is a common sleep disorder that affects sleep quality and can occur concomitantly with spinal cord lesions.⁵ In about 80% of RLS cases, involuntary movements of legs across hip, knee, and ankle joints during sleep, known as periodic limb movement during sleep (PLMS), occurs.⁶ Several studies showed increased prevalence of PLMS in patients with SCI, and some case reports suggest an increased prevalence of RLS in this population.^7,8 One small study showed that 100% of patients with SCI had symptoms of RLS.⁶ Another study found that SCI could trigger PLMS.⁸

The pathophysiology of RLS and PLMS in patients with SCI is not fully understood, but case reports describing PLM in SCI patients points to a possible role of central pattern generators and the flexor reflex afferents in the pathophysiology of PLMS.^9,10 Changes of the tissue microstructure in the midbrain and upper cervical spinal cord have been described in patients with RLS.¹¹The objective of this study was to assess the prevalence of RLS in a veteran population with SCI/D and to determine possible neuroanatomical patterns involved in RLS and SCI/D.

Methods

The institutional review and ethical approval boards of the Minneapolis VA Health Care System approved the study. Within the VA system, 666 patients with SCI/D were identified using a national database. Of the 666 people, 316 were excluded, 199 were included, and 151 were deceased.

Patients aged between 18 and 65 years were included in the study. Charts of patients who had been discharged with the diagnosis of SCI from 2002 to 2008 were studied. All patients met the inclusion criteria of the International Restless Legs Syndrome Study Group diagnosis.

Exclusion criteria were as follows: Patients with evidence of brain pathology (eg, stroke), concurrent neurologic condition associated with RLS (Parkinson disease, spinocerebellar ataxia, peripheral neuropathy), concurrent psychiatric condition within the setting of treatment with dopamine antagonists, secondary causes of RLS (renal failure/uremia, iron deficiency, rheumatoid arthritis, and pregnancy) and a recent history of alcohol or drug misuse or current evidence of substance use of < 1 year.

A patient list was compiled that included the etiology of the SCI (vascular injury, multiple sclerosis [MS], trauma, unknown, and other), the level(s) and completeness of the SCI per radiology report, RLS pharmacotherapies, and pertinent medical history.

Axial T2-weighted images on magnetic resonance imaging (MRI) scans were retrospectively reviewed. Sagittal T1/T2-weighted and axial T2-weighted sequences were performed routinely on all patients with spinal cord lesions. The analysis included the extension of the lesion on both sagittal and axial distributions. The anatomic location of the cord lesion was categorized by the following: (1) pure gray matter (central cord); (2) white matter (dorsal [D], dorsolateral [DL], ventral [V], ventrolateral areas [VL]).

A questionnaire using standard diagnostic criteria for RLS was mailed to the 199 patients who met the inclusion criteria (Appendix A).

All analyses were carried out using StataCorp STATA 13 (College Station, TX). Descriptive statistics were used. The analyses were carried out using chi-square and Fisher exact tests. Differences between the groups were considered statistically significant at P < .05. The data were analyzed to obtain point prevalence among patients with SCI, and comparisons were made among the different subgroups.

Results

Of the 162 patients who chose to participate in the study, the sleep specialists confirmed 31 (19%) to have RLS, 112 (69%) were confirmed negative for RLS, and an additional 19 (12%) screened positive for RLS but were not confirmed to have RLS by the sleep specialists (Figure 1).

The etiology of SCI was subdivided into 4 groups: MS, trauma, vascular, and other/unknown. Within each group (– RLS vs + RLS), MS and trauma were the most common etiologies with 55% MS and 36% trauma in the + RLS group.

When comparing RLS among the spinal cord levels (cervical, thoracic, lumbar and cervical + thoracic), only the cervical + thoracic subgroup (18% + RLS vs 5% – RLS) showed a significant difference (Figure 2).

There was no significant difference found with the prevalence of RLS in the axial plane of the spinal cord lesions (ventral/ventro-lateral/central cord vs dorsal/dorsolateral) or by the completeness of spinal cord lesions, P = .76. There was a higher prevalence of incomplete cord injury, however, within each subgroup of RLS.

The Mann-Whitney test was used to analyze the burden of disease in both groups (+ RLS vs – RLS). Moderate level of burden was most frequently reported with a higher prevalence within the + RLS group. Of those receiving treatment for RLS, 71% were + RLS vs 46% – RLS with a P value of .01. Symptoms of RLS after cord injury were 89% + RLS vs 55% – RLS with a P value of .03.

Discussion

This study represents one of the first studies to determine the prevalence of RLS in veterans with spinal cord disease. Research in this area is important to raise awareness of RLS among the veteran population with and without SCI and disorders. Restless legs syndrome often escapes diagnosis because of difficulty understanding the patient’s descriptions of their sensations. In addition, RLS may cause debilitating symptoms of sleep deprivation, daytime sleepiness, discomfort, and fatigue, which often results in decreased quality of life (QOL). Proper screening and treatment may improve QOL.

A study by Kumru and colleagues showed a similar rate of RLS in patients with SCI and RLS symptoms presented in the first year after SCI as did this study (18% vs 19%, respectively).⁴ In that study, RLS was more common in patients with lesions in lumbosacral area. Kumru and colleagues also showed that a dopaminergic medication improved symptoms of RLS in this population, whereas this study did not explore treatment outcomes.⁴

The pathogenesis of RLS is not fully known, but hereditary factors, iron metabolism, and the brain dopaminergic system are thought to be involved.¹¹ It is hypothesized that spinal cord lesions allow the appearance of RLS symptoms and spinal leg movement generator by blocking descending inhibitory spinal pathways.¹² One hypothesis is that damage to A11 nuclei (the main source of dopamine in the spinal cord or its diencephalospinal tract in animals) causes hyperexcitability of the spinal cord and leads to PLM and RLS symptoms.¹³ As the axons of A11 nuclei are present along the whole span of the spinal cord, SCI/D in patients with RLS might interrupt this dopaminergic tract and produce the RLS symptoms.

Limitations

This study included only veterans, so the prevalence may not apply to the nonveteran SCI population. Also, the population mainly was male, and there was no accurate information on race. Ferritin levels of the patients were not checked and is a major factor in RLS. The reported onset of RLS after the SCI could be due to recall bias.

Conclusion

The prevalence of RLS in veterans with SCI is above that reported in the general population (19% vs 10%, respectively). Furthermore, those with RLS have symptoms that often started after the SCI (suggesting causality) and required therapy due to their level of RLS symptom burden. A spectrum of severity of symptoms is present among those with RLS, with 83% having moderate-to-severe RLS affecting their QOL.

Although there was not a statistically significant relationship between RLS and spinal cord lesion level, there was a slightly higher prevalence of RLS at the cervical and thoracic levels, which may be relevant for future studies. There was no difference found between the RLS subgroups with respect to the location of the lesion within the spinal cord; however, a larger sample size may be needed to determine whether this would reach statistical significance. Prompt search for symptoms of RLS in veterans with SCI is warranted to provide adequate treatment to improve sleep health and QOL in this population.

Spinal cord injuries (SCI) are common in veteran populations.¹ Veterans with spinal cord injuries and disorders (SCI/D) also may have concurrent sleep disturbances. Spinal cord injury typically causes spasticity.^2,3 Hypersensitivity of the flexor reflex pathways is believed to cause painful muscle spasms in patients with SCI.⁴ Neuropathic pain at or below the level of the lesion also is common.

Restless legs syndrome (RLS) is a common sleep disorder that affects sleep quality and can occur concomitantly with spinal cord lesions.⁵ In about 80% of RLS cases, involuntary movements of legs across hip, knee, and ankle joints during sleep, known as periodic limb movement during sleep (PLMS), occurs.⁶ Several studies showed increased prevalence of PLMS in patients with SCI, and some case reports suggest an increased prevalence of RLS in this population.^7,8 One small study showed that 100% of patients with SCI had symptoms of RLS.⁶ Another study found that SCI could trigger PLMS.⁸

The pathophysiology of RLS and PLMS in patients with SCI is not fully understood, but case reports describing PLM in SCI patients points to a possible role of central pattern generators and the flexor reflex afferents in the pathophysiology of PLMS.^9,10 Changes of the tissue microstructure in the midbrain and upper cervical spinal cord have been described in patients with RLS.¹¹The objective of this study was to assess the prevalence of RLS in a veteran population with SCI/D and to determine possible neuroanatomical patterns involved in RLS and SCI/D.

Methods

The institutional review and ethical approval boards of the Minneapolis VA Health Care System approved the study. Within the VA system, 666 patients with SCI/D were identified using a national database. Of the 666 people, 316 were excluded, 199 were included, and 151 were deceased.

Patients aged between 18 and 65 years were included in the study. Charts of patients who had been discharged with the diagnosis of SCI from 2002 to 2008 were studied. All patients met the inclusion criteria of the International Restless Legs Syndrome Study Group diagnosis.

Exclusion criteria were as follows: Patients with evidence of brain pathology (eg, stroke), concurrent neurologic condition associated with RLS (Parkinson disease, spinocerebellar ataxia, peripheral neuropathy), concurrent psychiatric condition within the setting of treatment with dopamine antagonists, secondary causes of RLS (renal failure/uremia, iron deficiency, rheumatoid arthritis, and pregnancy) and a recent history of alcohol or drug misuse or current evidence of substance use of < 1 year.

A patient list was compiled that included the etiology of the SCI (vascular injury, multiple sclerosis [MS], trauma, unknown, and other), the level(s) and completeness of the SCI per radiology report, RLS pharmacotherapies, and pertinent medical history.

Axial T2-weighted images on magnetic resonance imaging (MRI) scans were retrospectively reviewed. Sagittal T1/T2-weighted and axial T2-weighted sequences were performed routinely on all patients with spinal cord lesions. The analysis included the extension of the lesion on both sagittal and axial distributions. The anatomic location of the cord lesion was categorized by the following: (1) pure gray matter (central cord); (2) white matter (dorsal [D], dorsolateral [DL], ventral [V], ventrolateral areas [VL]).

A questionnaire using standard diagnostic criteria for RLS was mailed to the 199 patients who met the inclusion criteria (Appendix A).

All analyses were carried out using StataCorp STATA 13 (College Station, TX). Descriptive statistics were used. The analyses were carried out using chi-square and Fisher exact tests. Differences between the groups were considered statistically significant at P < .05. The data were analyzed to obtain point prevalence among patients with SCI, and comparisons were made among the different subgroups.

Results

Of the 162 patients who chose to participate in the study, the sleep specialists confirmed 31 (19%) to have RLS, 112 (69%) were confirmed negative for RLS, and an additional 19 (12%) screened positive for RLS but were not confirmed to have RLS by the sleep specialists (Figure 1).

The etiology of SCI was subdivided into 4 groups: MS, trauma, vascular, and other/unknown. Within each group (– RLS vs + RLS), MS and trauma were the most common etiologies with 55% MS and 36% trauma in the + RLS group.

When comparing RLS among the spinal cord levels (cervical, thoracic, lumbar and cervical + thoracic), only the cervical + thoracic subgroup (18% + RLS vs 5% – RLS) showed a significant difference (Figure 2).

There was no significant difference found with the prevalence of RLS in the axial plane of the spinal cord lesions (ventral/ventro-lateral/central cord vs dorsal/dorsolateral) or by the completeness of spinal cord lesions, P = .76. There was a higher prevalence of incomplete cord injury, however, within each subgroup of RLS.

The Mann-Whitney test was used to analyze the burden of disease in both groups (+ RLS vs – RLS). Moderate level of burden was most frequently reported with a higher prevalence within the + RLS group. Of those receiving treatment for RLS, 71% were + RLS vs 46% – RLS with a P value of .01. Symptoms of RLS after cord injury were 89% + RLS vs 55% – RLS with a P value of .03.

Discussion

This study represents one of the first studies to determine the prevalence of RLS in veterans with spinal cord disease. Research in this area is important to raise awareness of RLS among the veteran population with and without SCI and disorders. Restless legs syndrome often escapes diagnosis because of difficulty understanding the patient’s descriptions of their sensations. In addition, RLS may cause debilitating symptoms of sleep deprivation, daytime sleepiness, discomfort, and fatigue, which often results in decreased quality of life (QOL). Proper screening and treatment may improve QOL.

A study by Kumru and colleagues showed a similar rate of RLS in patients with SCI and RLS symptoms presented in the first year after SCI as did this study (18% vs 19%, respectively).⁴ In that study, RLS was more common in patients with lesions in lumbosacral area. Kumru and colleagues also showed that a dopaminergic medication improved symptoms of RLS in this population, whereas this study did not explore treatment outcomes.⁴

The pathogenesis of RLS is not fully known, but hereditary factors, iron metabolism, and the brain dopaminergic system are thought to be involved.¹¹ It is hypothesized that spinal cord lesions allow the appearance of RLS symptoms and spinal leg movement generator by blocking descending inhibitory spinal pathways.¹² One hypothesis is that damage to A11 nuclei (the main source of dopamine in the spinal cord or its diencephalospinal tract in animals) causes hyperexcitability of the spinal cord and leads to PLM and RLS symptoms.¹³ As the axons of A11 nuclei are present along the whole span of the spinal cord, SCI/D in patients with RLS might interrupt this dopaminergic tract and produce the RLS symptoms.

Limitations

This study included only veterans, so the prevalence may not apply to the nonveteran SCI population. Also, the population mainly was male, and there was no accurate information on race. Ferritin levels of the patients were not checked and is a major factor in RLS. The reported onset of RLS after the SCI could be due to recall bias.

Conclusion

The prevalence of RLS in veterans with SCI is above that reported in the general population (19% vs 10%, respectively). Furthermore, those with RLS have symptoms that often started after the SCI (suggesting causality) and required therapy due to their level of RLS symptom burden. A spectrum of severity of symptoms is present among those with RLS, with 83% having moderate-to-severe RLS affecting their QOL.

Although there was not a statistically significant relationship between RLS and spinal cord lesion level, there was a slightly higher prevalence of RLS at the cervical and thoracic levels, which may be relevant for future studies. There was no difference found between the RLS subgroups with respect to the location of the lesion within the spinal cord; however, a larger sample size may be needed to determine whether this would reach statistical significance. Prompt search for symptoms of RLS in veterans with SCI is warranted to provide adequate treatment to improve sleep health and QOL in this population.

Spinal cord injuries (SCI) are common in veteran populations.¹ Veterans with spinal cord injuries and disorders (SCI/D) also may have concurrent sleep disturbances. Spinal cord injury typically causes spasticity.^2,3 Hypersensitivity of the flexor reflex pathways is believed to cause painful muscle spasms in patients with SCI.⁴ Neuropathic pain at or below the level of the lesion also is common.

Restless legs syndrome (RLS) is a common sleep disorder that affects sleep quality and can occur concomitantly with spinal cord lesions.⁵ In about 80% of RLS cases, involuntary movements of legs across hip, knee, and ankle joints during sleep, known as periodic limb movement during sleep (PLMS), occurs.⁶ Several studies showed increased prevalence of PLMS in patients with SCI, and some case reports suggest an increased prevalence of RLS in this population.^7,8 One small study showed that 100% of patients with SCI had symptoms of RLS.⁶ Another study found that SCI could trigger PLMS.⁸

The pathophysiology of RLS and PLMS in patients with SCI is not fully understood, but case reports describing PLM in SCI patients points to a possible role of central pattern generators and the flexor reflex afferents in the pathophysiology of PLMS.^9,10 Changes of the tissue microstructure in the midbrain and upper cervical spinal cord have been described in patients with RLS.¹¹The objective of this study was to assess the prevalence of RLS in a veteran population with SCI/D and to determine possible neuroanatomical patterns involved in RLS and SCI/D.

Methods

The institutional review and ethical approval boards of the Minneapolis VA Health Care System approved the study. Within the VA system, 666 patients with SCI/D were identified using a national database. Of the 666 people, 316 were excluded, 199 were included, and 151 were deceased.

Patients aged between 18 and 65 years were included in the study. Charts of patients who had been discharged with the diagnosis of SCI from 2002 to 2008 were studied. All patients met the inclusion criteria of the International Restless Legs Syndrome Study Group diagnosis.

Exclusion criteria were as follows: Patients with evidence of brain pathology (eg, stroke), concurrent neurologic condition associated with RLS (Parkinson disease, spinocerebellar ataxia, peripheral neuropathy), concurrent psychiatric condition within the setting of treatment with dopamine antagonists, secondary causes of RLS (renal failure/uremia, iron deficiency, rheumatoid arthritis, and pregnancy) and a recent history of alcohol or drug misuse or current evidence of substance use of < 1 year.

A patient list was compiled that included the etiology of the SCI (vascular injury, multiple sclerosis [MS], trauma, unknown, and other), the level(s) and completeness of the SCI per radiology report, RLS pharmacotherapies, and pertinent medical history.

Axial T2-weighted images on magnetic resonance imaging (MRI) scans were retrospectively reviewed. Sagittal T1/T2-weighted and axial T2-weighted sequences were performed routinely on all patients with spinal cord lesions. The analysis included the extension of the lesion on both sagittal and axial distributions. The anatomic location of the cord lesion was categorized by the following: (1) pure gray matter (central cord); (2) white matter (dorsal [D], dorsolateral [DL], ventral [V], ventrolateral areas [VL]).

A questionnaire using standard diagnostic criteria for RLS was mailed to the 199 patients who met the inclusion criteria (Appendix A).

All analyses were carried out using StataCorp STATA 13 (College Station, TX). Descriptive statistics were used. The analyses were carried out using chi-square and Fisher exact tests. Differences between the groups were considered statistically significant at P < .05. The data were analyzed to obtain point prevalence among patients with SCI, and comparisons were made among the different subgroups.

Results

Of the 162 patients who chose to participate in the study, the sleep specialists confirmed 31 (19%) to have RLS, 112 (69%) were confirmed negative for RLS, and an additional 19 (12%) screened positive for RLS but were not confirmed to have RLS by the sleep specialists (Figure 1).

The etiology of SCI was subdivided into 4 groups: MS, trauma, vascular, and other/unknown. Within each group (– RLS vs + RLS), MS and trauma were the most common etiologies with 55% MS and 36% trauma in the + RLS group.

When comparing RLS among the spinal cord levels (cervical, thoracic, lumbar and cervical + thoracic), only the cervical + thoracic subgroup (18% + RLS vs 5% – RLS) showed a significant difference (Figure 2).

There was no significant difference found with the prevalence of RLS in the axial plane of the spinal cord lesions (ventral/ventro-lateral/central cord vs dorsal/dorsolateral) or by the completeness of spinal cord lesions, P = .76. There was a higher prevalence of incomplete cord injury, however, within each subgroup of RLS.

The Mann-Whitney test was used to analyze the burden of disease in both groups (+ RLS vs – RLS). Moderate level of burden was most frequently reported with a higher prevalence within the + RLS group. Of those receiving treatment for RLS, 71% were + RLS vs 46% – RLS with a P value of .01. Symptoms of RLS after cord injury were 89% + RLS vs 55% – RLS with a P value of .03.

Discussion

This study represents one of the first studies to determine the prevalence of RLS in veterans with spinal cord disease. Research in this area is important to raise awareness of RLS among the veteran population with and without SCI and disorders. Restless legs syndrome often escapes diagnosis because of difficulty understanding the patient’s descriptions of their sensations. In addition, RLS may cause debilitating symptoms of sleep deprivation, daytime sleepiness, discomfort, and fatigue, which often results in decreased quality of life (QOL). Proper screening and treatment may improve QOL.

A study by Kumru and colleagues showed a similar rate of RLS in patients with SCI and RLS symptoms presented in the first year after SCI as did this study (18% vs 19%, respectively).⁴ In that study, RLS was more common in patients with lesions in lumbosacral area. Kumru and colleagues also showed that a dopaminergic medication improved symptoms of RLS in this population, whereas this study did not explore treatment outcomes.⁴

The pathogenesis of RLS is not fully known, but hereditary factors, iron metabolism, and the brain dopaminergic system are thought to be involved.¹¹ It is hypothesized that spinal cord lesions allow the appearance of RLS symptoms and spinal leg movement generator by blocking descending inhibitory spinal pathways.¹² One hypothesis is that damage to A11 nuclei (the main source of dopamine in the spinal cord or its diencephalospinal tract in animals) causes hyperexcitability of the spinal cord and leads to PLM and RLS symptoms.¹³ As the axons of A11 nuclei are present along the whole span of the spinal cord, SCI/D in patients with RLS might interrupt this dopaminergic tract and produce the RLS symptoms.

Limitations

This study included only veterans, so the prevalence may not apply to the nonveteran SCI population. Also, the population mainly was male, and there was no accurate information on race. Ferritin levels of the patients were not checked and is a major factor in RLS. The reported onset of RLS after the SCI could be due to recall bias.

Conclusion

The prevalence of RLS in veterans with SCI is above that reported in the general population (19% vs 10%, respectively). Furthermore, those with RLS have symptoms that often started after the SCI (suggesting causality) and required therapy due to their level of RLS symptom burden. A spectrum of severity of symptoms is present among those with RLS, with 83% having moderate-to-severe RLS affecting their QOL.

Although there was not a statistically significant relationship between RLS and spinal cord lesion level, there was a slightly higher prevalence of RLS at the cervical and thoracic levels, which may be relevant for future studies. There was no difference found between the RLS subgroups with respect to the location of the lesion within the spinal cord; however, a larger sample size may be needed to determine whether this would reach statistical significance. Prompt search for symptoms of RLS in veterans with SCI is warranted to provide adequate treatment to improve sleep health and QOL in this population.

Sézary syndrome (SS) is an advanced leukemic form of cutaneous T-cell lymphoma (CTCL) that is characterized by generalized erythroderma and T-cell leukemia. Skin changes can include erythroderma, keratosis pilaris–like lesions, keratoderma, ectropion, alopecia, and nail changes.¹ Nail changes in SS patients frequently are overlooked and underreported; they vary greatly from patient to patient, and their incidence has not been widely evaluated in the literature.

In this retrospective study, we reviewed medical records from a previously collected CTCL clinic database at the University of Texas MD Anderson Cancer Center (Houston, Texas) and found nail abnormalities in 36 of 83 (43.4%) patients with a diagnosis of SS. Findings for 2 select cases are described in more detail; they were compared to prior case reports from the literature to establish a comprehensive list of nail irregularities that have been associated with SS.

Methods

We examined records from a previously collected CTCL clinic database at the University of Texas MD Anderson Cancer Center. This database was part of an institutional review board–approved protocol to prospectively collect data from patients with CTCL. Our search yielded 83 patients with SS who were seen between 2007 and 2014.

Results

Of the 83 cases reviewed from the CTCL database, 36 (43.4%) SS patients reported at least 1 nail abnormality on the fingernails or toenails. Patients ranged in age from 59 to 85 years and included 27 (75%) men and 9 (25%) women. Nail irregularities noted on physical examination are summarized in Table 1. More than half of the patients presented with nail thickening (58.3% [21/36]), dystrophy (55.6% [20/36]), or yellowing (55.6% [20/36]) of 1 or more nails. Other findings included 15 (41.7%) patients with subungual hyperkeratosis, 3 (8.3%) with Beau lines, and 1 (2.8%) with multiple oil spots consistent with salmon patches. Five (13.9%) patients had only 1 reported nail irregularity, and 1 (2.8%) patient had 6 irregularities. The average number of nail abnormalities per patient was 2.88 (range, 1–6). We selected 2 patients with extensive nail findings who represent the spectrum of nail findings in patients with SS.

Patient 1
A 71-year-old white man presented with a papular rash of 30 years’ duration. The eruption first occurred on the soles of the feet but progressed to generalized erythroderma. He was found to be colonized with methicillin-resistant Staphylococcus aureus. Over the next 9 months, the patient was diagnosed with SS at an outside institution and was treated with cyclophosphamide, hydroxydaunorubicin, vincristine, prednisone, gemcitabine, etoposide, methylprednisolone, cytarabine, cisplatin, topical steroids, and intravenous methotrexate with no apparent improvement. At presentation to our institution, physical examination revealed pruritus; alopecia; generalized lymphadenopathy; erythroderma; and irregular nail findings, including yellowing, thickened fingernails and toenails with subungual debris, and splinter hemorrhage (Figure 1). A thick plaque with perioral distribution as well as erosions on the face and feet were noted. The total body surface area (BSA) affected was 100% (patches, 91%; plaques, 9%).

At diagnosis at our institution, the patient’s white blood cell (WBC) count was 17,800/µL (reference range, 4000–11,000/µL), with 11% Sézary cells noted. Biopsy of a lymph node from the inguinal area indicated T-cell lymphoma with clonal T-cell receptor (TCR) β gene rearrangement. Biopsy of lesional skin in the right groin area showed an atypical T-cell lymphocytic infiltrate with a CD4:CD8 ratio of 2.9:1 and partial loss of CD7 expression, consistent with mycosis fungoides (MF)/SS stage IVA. At presentation to our institution, the WBC count was 12,700/µL with a neutrophil count of 47% (reference range, 42%–66%), lymphocyte count of 36% (reference range, 24%–44%), monocyte count of 4% (reference range, 2%–7%), platelet count of 427,000/µL (reference range, 150,000–350,000/µL), hemoglobin of 9.9 g/dL (reference range, 14.0–17.5 g/dL), and lactate dehydrogenase of 733 U/L (reference range, 135–214 U/L). Lymphocytes were positive for CD2, CD3, CD4, CD5, CD25, CD52, TCRα, TCRβ, and TCR VB17; partial for CD26; and negative for CD7, CD8, and CD57. At follow-up 1 month later, the CD4⁺CD26⁻ T-cell population was 56%, which was consistent with SS T-cell lymphoma.

Skin scrapings from the generalized keratoderma on the patient’s feet were positive for fungal hyphae under potassium hydroxide examination. Nail clippings showed compact keratin with periodic acid–Schiff–positive small yeast forms admixed with bacterial organisms, consistent with onychomycosis. At our institution, the patient received extracorporeal photopheresis, whirlpool therapy (a type of hydrotherapy), steroid wet wraps, and intravenous vancomycin for methicillin-resistant S aureus. He also received bexarotene, levothyroxine sodium, and fenofibrate. After antibiotics and 2 sessions of photopheresis, the total BSA improved from 100% to 33%. The feet and nails were treated with ciclopirox gel and terbinafine, but neither the keratoderma nor the nails improved.

Patient 2
An 84-year-old white man with B-cell chronic lymphocytic leukemia also was diagnosed with SS at an outside institution. One year later, he presented to our institution with mild pruritus and swelling of the lower left leg, which was diagnosed as deep vein thrombosis. There was bilateral scaling of the palms, with fissures present on the left palm. The fingernails showed dystrophy with Beau lines, and the toenails were dystrophic with onycholysis on the bilateral great toes (Figure 2). Patches were noted on most of the body, including the feet, with plaques limited to the hands; the total BSA affected was 80%. Flow cytometry showed an elevated Sézary cell count (CD4⁺CD26⁻) of 4700 cells/µL. Complete blood cell count with differential included a hemoglobin level of 11.4 g/dL, hematocrit level of 35.3% (reference range, 37%–47%), a platelet count of 217,000/µL, and a WBC count of 17,700/µL, of which 29% were neutrophils, 63% were lymphocytes, 6% were monocytes, and 2% were eosinophils. The lactate dehydrogenase level was elevated at 829 U/L. The patient had not been treated for chronic lymphocytic leukemia in the last 11 months due to adverse reactions to rituximab after 2 cycles of fludarabine, cyclophosphamide, and rituximab chemotherapy. First-line therapy for the patient was photopheresis for SS.

Comment

Nail changes are found in many cases of advanced-stage SS but rarely have been reported in the literature. A literature review of PubMed articles indexed for MEDLINE was conducted using the search terms Sézary, nail, onychodystrophy, cutaneous T-cell lymphoma, and CTCL. All results were reviewed for original reported cases of SS with at least 1 reported nail finding. A total of 7 reports^2-8 met these requirements with a total of 43 SS patients with reported nail findings, which are summarized in Tables 2 and 3.

Our findings are generally consistent with those previously described in the literature. Nail thickening, yellowing, subungual hyperkeratosis, dystrophy, and onycholysis are consistently some of the most common nail findings in patients with SS. In 2012, Martin and Duvic⁹ found that 52.9% (45/85) of SS patients with keratoderma on physical examination were positive for dermatophyte hyphae when skin scrapings were done under potassium hydroxide examination, a considerably greater incidence than in the general population (10%–20%). The nail changes seen in our SS patients were identical to those found in dermatophyte infections, including discoloration, subungual debris, nail thickening, onycholysis, and dystrophy.¹⁰ In patient 1, nail clippings were positive for onychomycosis, a common nail condition that is especially prevalent in older or immunocompromised patients.^9,10

Interestingly, findings not observed in the literature included salmon patches and Beau lines. Beau lines are horizontal depressions in the nail plate and often are indicative of temporary interruption of nail growth, such as due to an underlying disease process, severe illness, and/or chemotherapy.^11,12 In our review, patient 2 had clinical findings of Beau lines. Because the average time for fingernail regrowth is 3 to 6 months,¹³ it is reasonable to assume that physical findings associated with fludarabine, cyclophosphamide, and rituximab chemotherapy treatment would no longer be demonstrated 11 months after completion of therapy. On the other hand, paronychia was frequently observed by Damasco et al⁸ (63.2% [12/19] of their cases), yet it was not found in our database or the other literature reports we reviewed. Perhaps these differences are due to differences in patient populations and/or available therapies, lack of documentation, or small sample size and limited reports in the literature.

A common question is: Are the nail irregularities caused by the physical symptoms of advanced CTCL or by the underlying disease process in response to the atypical T cells? Erythroderma has been speculated to cause many of the clinical findings of nail abnormalities found in CTCL patients.^2,3 However, Fleming et al¹⁴ described an MF patient who experienced onychomadesis without erythroderma, which suggests that a different mechanism may cause these nail changes. The wide range of nail abnormalities in CTCL can cause problems with diagnosing the specific cause underlying the nail alteration.

To further complicate the issue, numerous therapies for CTCL also may cause nail changes, such as the previously described Beau lines. In 2010, Parmentier et al⁴ reported a patient with nail alterations that had been present for more than 1 year, with 9 of 10 fingernails demonstrating anonychia, onychomadesis, subungual distal hyperkeratosis, and onycholysis. In this case report, the authors were able to exclude phototherapy as the cause of onycholysis (visible separation of the nail plate from the nail bed) and other clinical nail findings in the SS patient based on the onset of nail changes prior to beginning psoralen plus UVA therapy and complete sparing of 1 finger.⁴ The findings in our patient 1, who had no history of psoralen plus UVA therapy at the time the irregular nail findings presented, supports this observation. Total skin electron beam therapy for MF also has been reported to cause temporary nail stasis and thus must be taken into account when considering nail changes in patients with MF/SS.¹⁵

A nail matrix biopsy may provide clues to the definitive cause of the clinically observed nail changes; however, this procedure typically is not performed due to patient concerns of postoperative complications including pain and nail dystrophy.¹⁶ Histopathology features were similar in reported nail biopsies of 2 SS patients.^3,4 Tosti et al³ reported that longitudinal biopsy showed a dense lymphocytic infiltrate of atypical lymphocytes with involuted nuclei and notable epidermotropism. Parmentier et al⁴ reported a longitudinal nail biopsy in an SS patient that presented with atypical lymphocytes, epidermotropism, and Pautrier microabscess formation. Immunostaining showed CD3 positivity within the distal nail matrix, nail bed, and hyponychium. One-third of the cells stained positive for CD4, while the majority stained positive for CD8. Most notably, the skin, nails, and blood showed identical clonal rearrangement of TCRγ.⁴ Nail matrix biopsies in MF patients rarely have been reported in the literature, but those that are available show similar features to those seen in SS patients. Harland et al¹⁷ summarized the findings of 4 case reports of CTCL patients that included nail biopsies by stating, “[a]ll histopathologic findings from nail biopsies showed a dense subepithelial infiltrate of lymphocytes with marked epitheliotropism.” These histopathologic abnormalities are akin to skin biopsies in MF patients, thus providing an essential link to the disease state of MF and the nail abnormalities found within SS patients.

Treatment of the nail problems found within SS is challenging due to limited research. Parmentier et al⁴ noted an SS patient who was treated with topical mechlorethamine applied directly to the nail. In this case, topical mechlorethamine was effective at treating onychomadesis, subungual distal hyperkeratosis, and onycholysis within 6 months.⁴ Another SS patient, who presented with thickening and yellowing of the nail, had reported a proximal nail plate that resolved after chemotherapy. The patient did not survive long enough to note complete improvement of the nail.³ In our study, patient 1 was treated with ciclopirox gel and terbinafine, which did not result in nail improvement. Nail treatments in SS patients have yet to show much improvement and thus need more research and focus in the literature.

Conclusion

Sézary syndrome is a rare CTCL that can present with clinical features that may be mistaken for other diseases. Nail abnormalities in SS patients may be related to fungal involvement, medical therapy, or the underlying disease process of SS. We report one of the largest populations of SS patients with specific reported nail abnormalities, thus expanding the possibilities of nail changes that accompany the disease. Continued research and studies involving SS can provide a better understanding of nail involvement and successful treatment of these clinical findings.

Sézary syndrome (SS) is an advanced leukemic form of cutaneous T-cell lymphoma (CTCL) that is characterized by generalized erythroderma and T-cell leukemia. Skin changes can include erythroderma, keratosis pilaris–like lesions, keratoderma, ectropion, alopecia, and nail changes.¹ Nail changes in SS patients frequently are overlooked and underreported; they vary greatly from patient to patient, and their incidence has not been widely evaluated in the literature.

In this retrospective study, we reviewed medical records from a previously collected CTCL clinic database at the University of Texas MD Anderson Cancer Center (Houston, Texas) and found nail abnormalities in 36 of 83 (43.4%) patients with a diagnosis of SS. Findings for 2 select cases are described in more detail; they were compared to prior case reports from the literature to establish a comprehensive list of nail irregularities that have been associated with SS.

Methods

We examined records from a previously collected CTCL clinic database at the University of Texas MD Anderson Cancer Center. This database was part of an institutional review board–approved protocol to prospectively collect data from patients with CTCL. Our search yielded 83 patients with SS who were seen between 2007 and 2014.

Results

Of the 83 cases reviewed from the CTCL database, 36 (43.4%) SS patients reported at least 1 nail abnormality on the fingernails or toenails. Patients ranged in age from 59 to 85 years and included 27 (75%) men and 9 (25%) women. Nail irregularities noted on physical examination are summarized in Table 1. More than half of the patients presented with nail thickening (58.3% [21/36]), dystrophy (55.6% [20/36]), or yellowing (55.6% [20/36]) of 1 or more nails. Other findings included 15 (41.7%) patients with subungual hyperkeratosis, 3 (8.3%) with Beau lines, and 1 (2.8%) with multiple oil spots consistent with salmon patches. Five (13.9%) patients had only 1 reported nail irregularity, and 1 (2.8%) patient had 6 irregularities. The average number of nail abnormalities per patient was 2.88 (range, 1–6). We selected 2 patients with extensive nail findings who represent the spectrum of nail findings in patients with SS.

Patient 1
A 71-year-old white man presented with a papular rash of 30 years’ duration. The eruption first occurred on the soles of the feet but progressed to generalized erythroderma. He was found to be colonized with methicillin-resistant Staphylococcus aureus. Over the next 9 months, the patient was diagnosed with SS at an outside institution and was treated with cyclophosphamide, hydroxydaunorubicin, vincristine, prednisone, gemcitabine, etoposide, methylprednisolone, cytarabine, cisplatin, topical steroids, and intravenous methotrexate with no apparent improvement. At presentation to our institution, physical examination revealed pruritus; alopecia; generalized lymphadenopathy; erythroderma; and irregular nail findings, including yellowing, thickened fingernails and toenails with subungual debris, and splinter hemorrhage (Figure 1). A thick plaque with perioral distribution as well as erosions on the face and feet were noted. The total body surface area (BSA) affected was 100% (patches, 91%; plaques, 9%).

At diagnosis at our institution, the patient’s white blood cell (WBC) count was 17,800/µL (reference range, 4000–11,000/µL), with 11% Sézary cells noted. Biopsy of a lymph node from the inguinal area indicated T-cell lymphoma with clonal T-cell receptor (TCR) β gene rearrangement. Biopsy of lesional skin in the right groin area showed an atypical T-cell lymphocytic infiltrate with a CD4:CD8 ratio of 2.9:1 and partial loss of CD7 expression, consistent with mycosis fungoides (MF)/SS stage IVA. At presentation to our institution, the WBC count was 12,700/µL with a neutrophil count of 47% (reference range, 42%–66%), lymphocyte count of 36% (reference range, 24%–44%), monocyte count of 4% (reference range, 2%–7%), platelet count of 427,000/µL (reference range, 150,000–350,000/µL), hemoglobin of 9.9 g/dL (reference range, 14.0–17.5 g/dL), and lactate dehydrogenase of 733 U/L (reference range, 135–214 U/L). Lymphocytes were positive for CD2, CD3, CD4, CD5, CD25, CD52, TCRα, TCRβ, and TCR VB17; partial for CD26; and negative for CD7, CD8, and CD57. At follow-up 1 month later, the CD4⁺CD26⁻ T-cell population was 56%, which was consistent with SS T-cell lymphoma.

Skin scrapings from the generalized keratoderma on the patient’s feet were positive for fungal hyphae under potassium hydroxide examination. Nail clippings showed compact keratin with periodic acid–Schiff–positive small yeast forms admixed with bacterial organisms, consistent with onychomycosis. At our institution, the patient received extracorporeal photopheresis, whirlpool therapy (a type of hydrotherapy), steroid wet wraps, and intravenous vancomycin for methicillin-resistant S aureus. He also received bexarotene, levothyroxine sodium, and fenofibrate. After antibiotics and 2 sessions of photopheresis, the total BSA improved from 100% to 33%. The feet and nails were treated with ciclopirox gel and terbinafine, but neither the keratoderma nor the nails improved.

Patient 2
An 84-year-old white man with B-cell chronic lymphocytic leukemia also was diagnosed with SS at an outside institution. One year later, he presented to our institution with mild pruritus and swelling of the lower left leg, which was diagnosed as deep vein thrombosis. There was bilateral scaling of the palms, with fissures present on the left palm. The fingernails showed dystrophy with Beau lines, and the toenails were dystrophic with onycholysis on the bilateral great toes (Figure 2). Patches were noted on most of the body, including the feet, with plaques limited to the hands; the total BSA affected was 80%. Flow cytometry showed an elevated Sézary cell count (CD4⁺CD26⁻) of 4700 cells/µL. Complete blood cell count with differential included a hemoglobin level of 11.4 g/dL, hematocrit level of 35.3% (reference range, 37%–47%), a platelet count of 217,000/µL, and a WBC count of 17,700/µL, of which 29% were neutrophils, 63% were lymphocytes, 6% were monocytes, and 2% were eosinophils. The lactate dehydrogenase level was elevated at 829 U/L. The patient had not been treated for chronic lymphocytic leukemia in the last 11 months due to adverse reactions to rituximab after 2 cycles of fludarabine, cyclophosphamide, and rituximab chemotherapy. First-line therapy for the patient was photopheresis for SS.

Comment

Nail changes are found in many cases of advanced-stage SS but rarely have been reported in the literature. A literature review of PubMed articles indexed for MEDLINE was conducted using the search terms Sézary, nail, onychodystrophy, cutaneous T-cell lymphoma, and CTCL. All results were reviewed for original reported cases of SS with at least 1 reported nail finding. A total of 7 reports^2-8 met these requirements with a total of 43 SS patients with reported nail findings, which are summarized in Tables 2 and 3.

Our findings are generally consistent with those previously described in the literature. Nail thickening, yellowing, subungual hyperkeratosis, dystrophy, and onycholysis are consistently some of the most common nail findings in patients with SS. In 2012, Martin and Duvic⁹ found that 52.9% (45/85) of SS patients with keratoderma on physical examination were positive for dermatophyte hyphae when skin scrapings were done under potassium hydroxide examination, a considerably greater incidence than in the general population (10%–20%). The nail changes seen in our SS patients were identical to those found in dermatophyte infections, including discoloration, subungual debris, nail thickening, onycholysis, and dystrophy.¹⁰ In patient 1, nail clippings were positive for onychomycosis, a common nail condition that is especially prevalent in older or immunocompromised patients.^9,10

Interestingly, findings not observed in the literature included salmon patches and Beau lines. Beau lines are horizontal depressions in the nail plate and often are indicative of temporary interruption of nail growth, such as due to an underlying disease process, severe illness, and/or chemotherapy.^11,12 In our review, patient 2 had clinical findings of Beau lines. Because the average time for fingernail regrowth is 3 to 6 months,¹³ it is reasonable to assume that physical findings associated with fludarabine, cyclophosphamide, and rituximab chemotherapy treatment would no longer be demonstrated 11 months after completion of therapy. On the other hand, paronychia was frequently observed by Damasco et al⁸ (63.2% [12/19] of their cases), yet it was not found in our database or the other literature reports we reviewed. Perhaps these differences are due to differences in patient populations and/or available therapies, lack of documentation, or small sample size and limited reports in the literature.

A common question is: Are the nail irregularities caused by the physical symptoms of advanced CTCL or by the underlying disease process in response to the atypical T cells? Erythroderma has been speculated to cause many of the clinical findings of nail abnormalities found in CTCL patients.^2,3 However, Fleming et al¹⁴ described an MF patient who experienced onychomadesis without erythroderma, which suggests that a different mechanism may cause these nail changes. The wide range of nail abnormalities in CTCL can cause problems with diagnosing the specific cause underlying the nail alteration.

To further complicate the issue, numerous therapies for CTCL also may cause nail changes, such as the previously described Beau lines. In 2010, Parmentier et al⁴ reported a patient with nail alterations that had been present for more than 1 year, with 9 of 10 fingernails demonstrating anonychia, onychomadesis, subungual distal hyperkeratosis, and onycholysis. In this case report, the authors were able to exclude phototherapy as the cause of onycholysis (visible separation of the nail plate from the nail bed) and other clinical nail findings in the SS patient based on the onset of nail changes prior to beginning psoralen plus UVA therapy and complete sparing of 1 finger.⁴ The findings in our patient 1, who had no history of psoralen plus UVA therapy at the time the irregular nail findings presented, supports this observation. Total skin electron beam therapy for MF also has been reported to cause temporary nail stasis and thus must be taken into account when considering nail changes in patients with MF/SS.¹⁵

A nail matrix biopsy may provide clues to the definitive cause of the clinically observed nail changes; however, this procedure typically is not performed due to patient concerns of postoperative complications including pain and nail dystrophy.¹⁶ Histopathology features were similar in reported nail biopsies of 2 SS patients.^3,4 Tosti et al³ reported that longitudinal biopsy showed a dense lymphocytic infiltrate of atypical lymphocytes with involuted nuclei and notable epidermotropism. Parmentier et al⁴ reported a longitudinal nail biopsy in an SS patient that presented with atypical lymphocytes, epidermotropism, and Pautrier microabscess formation. Immunostaining showed CD3 positivity within the distal nail matrix, nail bed, and hyponychium. One-third of the cells stained positive for CD4, while the majority stained positive for CD8. Most notably, the skin, nails, and blood showed identical clonal rearrangement of TCRγ.⁴ Nail matrix biopsies in MF patients rarely have been reported in the literature, but those that are available show similar features to those seen in SS patients. Harland et al¹⁷ summarized the findings of 4 case reports of CTCL patients that included nail biopsies by stating, “[a]ll histopathologic findings from nail biopsies showed a dense subepithelial infiltrate of lymphocytes with marked epitheliotropism.” These histopathologic abnormalities are akin to skin biopsies in MF patients, thus providing an essential link to the disease state of MF and the nail abnormalities found within SS patients.

Treatment of the nail problems found within SS is challenging due to limited research. Parmentier et al⁴ noted an SS patient who was treated with topical mechlorethamine applied directly to the nail. In this case, topical mechlorethamine was effective at treating onychomadesis, subungual distal hyperkeratosis, and onycholysis within 6 months.⁴ Another SS patient, who presented with thickening and yellowing of the nail, had reported a proximal nail plate that resolved after chemotherapy. The patient did not survive long enough to note complete improvement of the nail.³ In our study, patient 1 was treated with ciclopirox gel and terbinafine, which did not result in nail improvement. Nail treatments in SS patients have yet to show much improvement and thus need more research and focus in the literature.

Conclusion

Sézary syndrome is a rare CTCL that can present with clinical features that may be mistaken for other diseases. Nail abnormalities in SS patients may be related to fungal involvement, medical therapy, or the underlying disease process of SS. We report one of the largest populations of SS patients with specific reported nail abnormalities, thus expanding the possibilities of nail changes that accompany the disease. Continued research and studies involving SS can provide a better understanding of nail involvement and successful treatment of these clinical findings.

Sézary syndrome (SS) is an advanced leukemic form of cutaneous T-cell lymphoma (CTCL) that is characterized by generalized erythroderma and T-cell leukemia. Skin changes can include erythroderma, keratosis pilaris–like lesions, keratoderma, ectropion, alopecia, and nail changes.¹ Nail changes in SS patients frequently are overlooked and underreported; they vary greatly from patient to patient, and their incidence has not been widely evaluated in the literature.

In this retrospective study, we reviewed medical records from a previously collected CTCL clinic database at the University of Texas MD Anderson Cancer Center (Houston, Texas) and found nail abnormalities in 36 of 83 (43.4%) patients with a diagnosis of SS. Findings for 2 select cases are described in more detail; they were compared to prior case reports from the literature to establish a comprehensive list of nail irregularities that have been associated with SS.

Methods

We examined records from a previously collected CTCL clinic database at the University of Texas MD Anderson Cancer Center. This database was part of an institutional review board–approved protocol to prospectively collect data from patients with CTCL. Our search yielded 83 patients with SS who were seen between 2007 and 2014.

Results

Of the 83 cases reviewed from the CTCL database, 36 (43.4%) SS patients reported at least 1 nail abnormality on the fingernails or toenails. Patients ranged in age from 59 to 85 years and included 27 (75%) men and 9 (25%) women. Nail irregularities noted on physical examination are summarized in Table 1. More than half of the patients presented with nail thickening (58.3% [21/36]), dystrophy (55.6% [20/36]), or yellowing (55.6% [20/36]) of 1 or more nails. Other findings included 15 (41.7%) patients with subungual hyperkeratosis, 3 (8.3%) with Beau lines, and 1 (2.8%) with multiple oil spots consistent with salmon patches. Five (13.9%) patients had only 1 reported nail irregularity, and 1 (2.8%) patient had 6 irregularities. The average number of nail abnormalities per patient was 2.88 (range, 1–6). We selected 2 patients with extensive nail findings who represent the spectrum of nail findings in patients with SS.

Patient 1
A 71-year-old white man presented with a papular rash of 30 years’ duration. The eruption first occurred on the soles of the feet but progressed to generalized erythroderma. He was found to be colonized with methicillin-resistant Staphylococcus aureus. Over the next 9 months, the patient was diagnosed with SS at an outside institution and was treated with cyclophosphamide, hydroxydaunorubicin, vincristine, prednisone, gemcitabine, etoposide, methylprednisolone, cytarabine, cisplatin, topical steroids, and intravenous methotrexate with no apparent improvement. At presentation to our institution, physical examination revealed pruritus; alopecia; generalized lymphadenopathy; erythroderma; and irregular nail findings, including yellowing, thickened fingernails and toenails with subungual debris, and splinter hemorrhage (Figure 1). A thick plaque with perioral distribution as well as erosions on the face and feet were noted. The total body surface area (BSA) affected was 100% (patches, 91%; plaques, 9%).

At diagnosis at our institution, the patient’s white blood cell (WBC) count was 17,800/µL (reference range, 4000–11,000/µL), with 11% Sézary cells noted. Biopsy of a lymph node from the inguinal area indicated T-cell lymphoma with clonal T-cell receptor (TCR) β gene rearrangement. Biopsy of lesional skin in the right groin area showed an atypical T-cell lymphocytic infiltrate with a CD4:CD8 ratio of 2.9:1 and partial loss of CD7 expression, consistent with mycosis fungoides (MF)/SS stage IVA. At presentation to our institution, the WBC count was 12,700/µL with a neutrophil count of 47% (reference range, 42%–66%), lymphocyte count of 36% (reference range, 24%–44%), monocyte count of 4% (reference range, 2%–7%), platelet count of 427,000/µL (reference range, 150,000–350,000/µL), hemoglobin of 9.9 g/dL (reference range, 14.0–17.5 g/dL), and lactate dehydrogenase of 733 U/L (reference range, 135–214 U/L). Lymphocytes were positive for CD2, CD3, CD4, CD5, CD25, CD52, TCRα, TCRβ, and TCR VB17; partial for CD26; and negative for CD7, CD8, and CD57. At follow-up 1 month later, the CD4⁺CD26⁻ T-cell population was 56%, which was consistent with SS T-cell lymphoma.

Skin scrapings from the generalized keratoderma on the patient’s feet were positive for fungal hyphae under potassium hydroxide examination. Nail clippings showed compact keratin with periodic acid–Schiff–positive small yeast forms admixed with bacterial organisms, consistent with onychomycosis. At our institution, the patient received extracorporeal photopheresis, whirlpool therapy (a type of hydrotherapy), steroid wet wraps, and intravenous vancomycin for methicillin-resistant S aureus. He also received bexarotene, levothyroxine sodium, and fenofibrate. After antibiotics and 2 sessions of photopheresis, the total BSA improved from 100% to 33%. The feet and nails were treated with ciclopirox gel and terbinafine, but neither the keratoderma nor the nails improved.

Patient 2
An 84-year-old white man with B-cell chronic lymphocytic leukemia also was diagnosed with SS at an outside institution. One year later, he presented to our institution with mild pruritus and swelling of the lower left leg, which was diagnosed as deep vein thrombosis. There was bilateral scaling of the palms, with fissures present on the left palm. The fingernails showed dystrophy with Beau lines, and the toenails were dystrophic with onycholysis on the bilateral great toes (Figure 2). Patches were noted on most of the body, including the feet, with plaques limited to the hands; the total BSA affected was 80%. Flow cytometry showed an elevated Sézary cell count (CD4⁺CD26⁻) of 4700 cells/µL. Complete blood cell count with differential included a hemoglobin level of 11.4 g/dL, hematocrit level of 35.3% (reference range, 37%–47%), a platelet count of 217,000/µL, and a WBC count of 17,700/µL, of which 29% were neutrophils, 63% were lymphocytes, 6% were monocytes, and 2% were eosinophils. The lactate dehydrogenase level was elevated at 829 U/L. The patient had not been treated for chronic lymphocytic leukemia in the last 11 months due to adverse reactions to rituximab after 2 cycles of fludarabine, cyclophosphamide, and rituximab chemotherapy. First-line therapy for the patient was photopheresis for SS.

Comment

Nail changes are found in many cases of advanced-stage SS but rarely have been reported in the literature. A literature review of PubMed articles indexed for MEDLINE was conducted using the search terms Sézary, nail, onychodystrophy, cutaneous T-cell lymphoma, and CTCL. All results were reviewed for original reported cases of SS with at least 1 reported nail finding. A total of 7 reports^2-8 met these requirements with a total of 43 SS patients with reported nail findings, which are summarized in Tables 2 and 3.

Our findings are generally consistent with those previously described in the literature. Nail thickening, yellowing, subungual hyperkeratosis, dystrophy, and onycholysis are consistently some of the most common nail findings in patients with SS. In 2012, Martin and Duvic⁹ found that 52.9% (45/85) of SS patients with keratoderma on physical examination were positive for dermatophyte hyphae when skin scrapings were done under potassium hydroxide examination, a considerably greater incidence than in the general population (10%–20%). The nail changes seen in our SS patients were identical to those found in dermatophyte infections, including discoloration, subungual debris, nail thickening, onycholysis, and dystrophy.¹⁰ In patient 1, nail clippings were positive for onychomycosis, a common nail condition that is especially prevalent in older or immunocompromised patients.^9,10

Interestingly, findings not observed in the literature included salmon patches and Beau lines. Beau lines are horizontal depressions in the nail plate and often are indicative of temporary interruption of nail growth, such as due to an underlying disease process, severe illness, and/or chemotherapy.^11,12 In our review, patient 2 had clinical findings of Beau lines. Because the average time for fingernail regrowth is 3 to 6 months,¹³ it is reasonable to assume that physical findings associated with fludarabine, cyclophosphamide, and rituximab chemotherapy treatment would no longer be demonstrated 11 months after completion of therapy. On the other hand, paronychia was frequently observed by Damasco et al⁸ (63.2% [12/19] of their cases), yet it was not found in our database or the other literature reports we reviewed. Perhaps these differences are due to differences in patient populations and/or available therapies, lack of documentation, or small sample size and limited reports in the literature.

A common question is: Are the nail irregularities caused by the physical symptoms of advanced CTCL or by the underlying disease process in response to the atypical T cells? Erythroderma has been speculated to cause many of the clinical findings of nail abnormalities found in CTCL patients.^2,3 However, Fleming et al¹⁴ described an MF patient who experienced onychomadesis without erythroderma, which suggests that a different mechanism may cause these nail changes. The wide range of nail abnormalities in CTCL can cause problems with diagnosing the specific cause underlying the nail alteration.

To further complicate the issue, numerous therapies for CTCL also may cause nail changes, such as the previously described Beau lines. In 2010, Parmentier et al⁴ reported a patient with nail alterations that had been present for more than 1 year, with 9 of 10 fingernails demonstrating anonychia, onychomadesis, subungual distal hyperkeratosis, and onycholysis. In this case report, the authors were able to exclude phototherapy as the cause of onycholysis (visible separation of the nail plate from the nail bed) and other clinical nail findings in the SS patient based on the onset of nail changes prior to beginning psoralen plus UVA therapy and complete sparing of 1 finger.⁴ The findings in our patient 1, who had no history of psoralen plus UVA therapy at the time the irregular nail findings presented, supports this observation. Total skin electron beam therapy for MF also has been reported to cause temporary nail stasis and thus must be taken into account when considering nail changes in patients with MF/SS.¹⁵

A nail matrix biopsy may provide clues to the definitive cause of the clinically observed nail changes; however, this procedure typically is not performed due to patient concerns of postoperative complications including pain and nail dystrophy.¹⁶ Histopathology features were similar in reported nail biopsies of 2 SS patients.^3,4 Tosti et al³ reported that longitudinal biopsy showed a dense lymphocytic infiltrate of atypical lymphocytes with involuted nuclei and notable epidermotropism. Parmentier et al⁴ reported a longitudinal nail biopsy in an SS patient that presented with atypical lymphocytes, epidermotropism, and Pautrier microabscess formation. Immunostaining showed CD3 positivity within the distal nail matrix, nail bed, and hyponychium. One-third of the cells stained positive for CD4, while the majority stained positive for CD8. Most notably, the skin, nails, and blood showed identical clonal rearrangement of TCRγ.⁴ Nail matrix biopsies in MF patients rarely have been reported in the literature, but those that are available show similar features to those seen in SS patients. Harland et al¹⁷ summarized the findings of 4 case reports of CTCL patients that included nail biopsies by stating, “[a]ll histopathologic findings from nail biopsies showed a dense subepithelial infiltrate of lymphocytes with marked epitheliotropism.” These histopathologic abnormalities are akin to skin biopsies in MF patients, thus providing an essential link to the disease state of MF and the nail abnormalities found within SS patients.

Treatment of the nail problems found within SS is challenging due to limited research. Parmentier et al⁴ noted an SS patient who was treated with topical mechlorethamine applied directly to the nail. In this case, topical mechlorethamine was effective at treating onychomadesis, subungual distal hyperkeratosis, and onycholysis within 6 months.⁴ Another SS patient, who presented with thickening and yellowing of the nail, had reported a proximal nail plate that resolved after chemotherapy. The patient did not survive long enough to note complete improvement of the nail.³ In our study, patient 1 was treated with ciclopirox gel and terbinafine, which did not result in nail improvement. Nail treatments in SS patients have yet to show much improvement and thus need more research and focus in the literature.

Conclusion

Sézary syndrome is a rare CTCL that can present with clinical features that may be mistaken for other diseases. Nail abnormalities in SS patients may be related to fungal involvement, medical therapy, or the underlying disease process of SS. We report one of the largest populations of SS patients with specific reported nail abnormalities, thus expanding the possibilities of nail changes that accompany the disease. Continued research and studies involving SS can provide a better understanding of nail involvement and successful treatment of these clinical findings.

In recent years, driven by accelerating health care costs and desire for improved health care value, major specialty group guidelines have incorporated resource utilization and value calculations into their recommendations. High-value care has the characteristics of enhancing outcomes, safety, and patient satisfaction at a reasonable cost. As one example, the American College of Cardiology (ACC) recently published a consensus statement on its clinical practice guidelines with a specific focus on cost and value.¹ This guideline acknowledges the difficulty in incorporating value into clinical decision making but stresses a need for increased transparency and consistency to boost value in everyday practice.

Chest pain and related symptoms were listed as the second leading principle reasons for emergency department visits in the US in 2011 with 14% of patients undergoing cardiac enzyme testing.² The ACC guidelines advocate use of troponin as the preferred laboratory test for the initial evaluation of acute coronary syndrome (ACS). Fractionated creatine kinase (CK-MB) is an acceptable alternative only when a cardiac troponin test is not available.³ Furthermore, troponins should be obtained no more than 3 times for the initial evaluation of a single event, and further trending provides no additional benefit or prognostic information.

A recent study from an academic hospital showed that process improvement interventions focused on eliminating unnecessary cardiac enzyme testing led to a 1-year cost savings of $1.25 million while increasing the rate of ACS diagnosis.⁴ Common clinical practice at Naval Medical Center Portsmouth (NMCP) in Virginia still routinely includes both troponin as well as a CK panel comprised of CK, CK-MB, and a calculated CK-MB/CK index. Our study focuses on the implementation of quality improvement efforts described by Larochelle and colleagues at NMCP.⁴ The study aimed to determine the impact of implementing interventions designed to improve the ordering practices and reduce the cost of cardiac enzyme testing.

Methods

The primary focus of the intervention was on ordering practices of the emergency medicine department (EMD), internal medicine (IM) inpatient services, and cardiology inpatient services. Specific interventions were: (1) removal of the CK panel from the chest pain order set in the EMD electronic health record (EHR); (2) removal of the CK panel from the inpatient cardiology order set; (3) education of staff on the changes in CK panel utility via direct communication during IM academic seminars; (4) education of nursing staff ordering laboratory results on behalf of physicians on the cardiology service at the morning and evening huddles; and (5) addition of “max of 3 tests indicated” comment to the inpatient EHR ordering page of the troponin test. Acknowledging that the CK-MB has some utility to interventional cardiologists in the setting of confirmed ACS, the laboratory instituted an automated, reflexive order of the CK-MB panel only if the troponin tests were positive. This test was automatically run on the same vial originally sent to the lab to mitigate any additional delay in determining results.

Data Source

The process improvement interventions were considered exempt from institutional review board (IRB) approval; however, we obtained expedited IRB approval with waiver of consent for the research aspect of the project. We obtained clinical administrative data from the Military Health System Data Repository (MDR). We identified all adult patients aged ≥ 18 years who had a troponin test, CK-MB, or both drawn at NMCP on the following services: the EMD, IM, and cardiology. A troponin or CK-MB test was defined using Current Procedural Terminology (CPT) codes and unique Logical Observation Identifiers Names and Codes (LOINC).

Measures

The study was divided into 3 periods: the preintervention period from August 1, 2013 to July 31, 2014; the intervention period from August 1, 2014 to January 31, 2015; and the postintervention period February 1, 2015 to January 31, 2016.

The primary outcomes measured were the frequency of guideline concordance and total costs for tests ordered per month using the Centers for Medicare and Medicaid Services (CMS) clinical laboratory fee schedule of $13.40 for troponin and $16.17 for CK-MB.⁵Concordance was defined as ≤ 3 troponin tests and no CK-MB tests ordered during 1 encounter for a patient without an ACS diagnosis in the preceding 7 days. Due to faster cellular release kinetics of CK-MB compared with that of troponin, this test has utility in evaluating new or worsening chest pain in the setting of a recent myocardial infarction (MI). Therefore, we excluded any patient who had a MI within the preceding 7 days of an order for either CK-MB or troponin tests. Additionally, the number of tests, both CK-MB and troponin, ordered per patient encounter (hereafter referred to as an episode) were measured. Finally, we measured the monthly prevalence of ACS diagnosis and percentage of visits having that diagnosis.

Data Analysis

Descriptive statistics were used to calculate population demographics of age group, sex, beneficiary category, sponsor service, and clinical setting. Monthly data were grouped into the preintervention and postintervention periods. The analysis was performed using t tests to compare mean values and CIs before and after the intervention. Simple linear regression with attention to correlation was used to create best fit lines with confidence bands before and after the intervention. Interrupted time series (ITS) regression was used to describe all data points throughout the study. Consistency between these various methods was verified. Mean values and CIs were reported from the t tests. Statistical significance was reported when appropriate. Equations and confidence predictions on the simple linear regressions were produced and reported. These were used to identify values at the start, midpoint, and end of the pre- and postintervention periods.

Results

There were a total of 6,281 patients in the study population. More patients were seen during the postintervention period than in the preintervention period. The mean age of patients was slightly higher during the preintervention period (Table 1).

Guideline Concordance

To determine whether ordering practices for cardiac enzyme testing improved, we assessed the changes in the frequency of guideline concordance during the pre- and postintervention period. On average during the preintervention year, the percentage of tests ordered that met guideline concordance was 10.1% (95% CI, 7.4%-12.9%), increasing by 0.80% (95% CI, 0.17%-1.42%) each month. 

Costs

We assessed changes in total dollars spent on cardiac enzyme testing during the pre- and postintervention periods. During the preintervention year, $9,400 (95% CI, $8,700-$10,100) was spent on average each month, which did not change significantly throughout the period. During the postintervention year, the cost was stable at $5,000 (95% CI, $4,600-$5,300) on average each month, a reduction of $4,400 (95% CI, $3,700-$5,100) (Figure 2).

CK-MB and Troponin Tests per Patient

To further assess ordering practices for cardiac enzyme testing, we compared the changes in the monthly number of tests and the average number of CK-MB and troponin tests ordered per episode pre- and postintervention. On average during the preintervention year, 297 tests (95% CI, 278-315) were run per month, with an average of 1.21 CK tests (95% CI, 1.15-1.27) per episode (Table 2, Figure 3). 

The changes in troponin testing were not as dramatic. The counts of tests each month remained similar, with a preintervention year average of 341 (95% CI, 306-377) and postintervention year average of 310 (95% CI, 287-332), which were not statistically different. However, there was a statistically significant decrease in the number of tests per episode. During the preintervention year, 1.38 troponin tests (95% CI, 1.31-1.45) were ordered per patient on average. This dropped by 0.17 (95% CI, 0.09-0.24) to the postintervention average of 1.21 (95% CI, 1.17-1.25) (Table 2, Figure 4). 

ACS Prevalence

To determine whether there was an impact on ACS diagnoses, we looked at the numbers of ACS diagnoses and their prevalence among visits before and after the intervention. During the preintervention year, the average monthly number of diagnoses was 29.7 (95% CI, 26.1-33.2), and prevalence of ACS was 0.56% (95% CI, 0.48%-0.63%) of all episodes. Although the monthly rate was statistically decreasing by 0.022% (95% CI, 0.003-0.41), this has little meaning since the level of correlation (r² = 0.2522, not displayed) was poor due to the essentially nonexistent correlation in number of visits each month (r² = 0.0112, not displayed). During the postintervention year, the average number of diagnoses was 32.2 (95% CI, 27.9-36.6), and the prevalence of ACS was 0.62% (95% CI, 0.54-0.65). Neither of these values changed significantly between the pre- and postintervention period. All ICD-9 and ICD-10 diagnosis codes used for the analysis are available upon request from the authors.

Discussion

Our data demonstrate the ability of simple process improvement interventions to decrease unnecessary testing in the workup of ACS, increasing the rate of guideline concordant testing by > 70% at a single military treatment facility (MTF). In particular, with the now widespread use of EHR, the order set presents a high-yield target for process improvement in an easily implemented, durable fashion. We had expected to see some decrease in the efficacy of the intervention at a time of staff turnover in the summer of 2015 because ongoing dedicated teaching sessions were not performed. Despite that, the intervention remained effective without further dedicated teaching sessions. This outcome was certainly attributable to the hardwired interventions made (mainly via order sets), but possibly indicates an institutional memory that can take hold after an initial concerted effort is made.

We reduced the estimated preintervention annual cost of $113,000 by $53,000 (95% CI, $42,000-$64,000). Although on a much smaller scale than the study by Larochelle, our study represents a nearly 50% reduction in the total cost of initial testing for possible ACS and a > 80% reduction in unnecessary CK-MB testing.⁴ This result was achieved with no statistical change in the prevalence of ACS. The cost reduction does not account for the labor costs to clinically follow-up and address additional unnecessary lab results. The estimated cost of intervention was limited to the time required to educate residents, interns, and nursing staff as well as the implementation of the automated, reflexive laboratory results ordering process.

Unique to our study, we also demonstrated an intervention that satisfied all the major stakeholders in the ordering of these laboratory results. By instituting the reflexive ordering of CK-MB tests for positive troponins, we obtained the support of the facility’s interventional cardiology department, which finds value in that data. Appreciating the time-sensitive nature of an ACS diagnosis, the reflexive ordering minimized the delay in receiving these data while still greatly reducing the number of tests performed. That being said, if the current trend away from CK-MB in favor of exclusively testing troponin continues, removing the reflexive ordering for positive laboratory results protocol would be an easy follow-on intervention.

Limitations

Our study presented several limitations. First, reporting errors due to improper or insufficient medical coding as well as data entry errors may exist within the MDR; therefore, the results of this analysis may be over- or underestimated. Specifically, CPT codes for troponin and CK-MB were available only in 1 of the 2 data sets used for this study, which primarily contains outpatient patient encounters. For this reason, most of the laboratory testing comes from the EMD rather than from inpatient services. However, because we excluded all patients who eventually had an ACS diagnosis (patients who likely had more inpatient time and better indication for repeat troponin), we feel that our intervention was still thoroughly investigated. Second, the number of tests drawn per patient was significantly < 2, the expected minimum number of tests to rule out ACS in patients with appropriate symptoms.

This study was not designed to answer the source of variation from guidelines. Many patients had only 1 test, which we feel represents an opportunity for future study to identify other ways cardiac enzyme testing is being used clinically. These tests might be used for patients without convincing symptoms and signs of coronary syndromes or for patients with other primary problems. Third, by using the ITS analysis, we assumed that the outcome during each intervention period follows a linear pattern. However, changes may follow a nonlinear pattern over a long period. Finally, our intervention was limited to only a single MTF, which may limit generalizability to other facilities across military medicine. However, we feel this study should serve as a guide for other MTFs as well as US Department of Veterans Affairs facilities that could institute similar process improvements.

Conclusion

We made easily implemented and durable process improvement interventions that changed institution-wide ordering practices. These changes dramatically increased the rate of guideline-concordant testing, decreasing cost and furthering the goal of high-value medical care.

In recent years, driven by accelerating health care costs and desire for improved health care value, major specialty group guidelines have incorporated resource utilization and value calculations into their recommendations. High-value care has the characteristics of enhancing outcomes, safety, and patient satisfaction at a reasonable cost. As one example, the American College of Cardiology (ACC) recently published a consensus statement on its clinical practice guidelines with a specific focus on cost and value.¹ This guideline acknowledges the difficulty in incorporating value into clinical decision making but stresses a need for increased transparency and consistency to boost value in everyday practice.

Chest pain and related symptoms were listed as the second leading principle reasons for emergency department visits in the US in 2011 with 14% of patients undergoing cardiac enzyme testing.² The ACC guidelines advocate use of troponin as the preferred laboratory test for the initial evaluation of acute coronary syndrome (ACS). Fractionated creatine kinase (CK-MB) is an acceptable alternative only when a cardiac troponin test is not available.³ Furthermore, troponins should be obtained no more than 3 times for the initial evaluation of a single event, and further trending provides no additional benefit or prognostic information.

A recent study from an academic hospital showed that process improvement interventions focused on eliminating unnecessary cardiac enzyme testing led to a 1-year cost savings of $1.25 million while increasing the rate of ACS diagnosis.⁴ Common clinical practice at Naval Medical Center Portsmouth (NMCP) in Virginia still routinely includes both troponin as well as a CK panel comprised of CK, CK-MB, and a calculated CK-MB/CK index. Our study focuses on the implementation of quality improvement efforts described by Larochelle and colleagues at NMCP.⁴ The study aimed to determine the impact of implementing interventions designed to improve the ordering practices and reduce the cost of cardiac enzyme testing.

Methods

The primary focus of the intervention was on ordering practices of the emergency medicine department (EMD), internal medicine (IM) inpatient services, and cardiology inpatient services. Specific interventions were: (1) removal of the CK panel from the chest pain order set in the EMD electronic health record (EHR); (2) removal of the CK panel from the inpatient cardiology order set; (3) education of staff on the changes in CK panel utility via direct communication during IM academic seminars; (4) education of nursing staff ordering laboratory results on behalf of physicians on the cardiology service at the morning and evening huddles; and (5) addition of “max of 3 tests indicated” comment to the inpatient EHR ordering page of the troponin test. Acknowledging that the CK-MB has some utility to interventional cardiologists in the setting of confirmed ACS, the laboratory instituted an automated, reflexive order of the CK-MB panel only if the troponin tests were positive. This test was automatically run on the same vial originally sent to the lab to mitigate any additional delay in determining results.

Data Source

The process improvement interventions were considered exempt from institutional review board (IRB) approval; however, we obtained expedited IRB approval with waiver of consent for the research aspect of the project. We obtained clinical administrative data from the Military Health System Data Repository (MDR). We identified all adult patients aged ≥ 18 years who had a troponin test, CK-MB, or both drawn at NMCP on the following services: the EMD, IM, and cardiology. A troponin or CK-MB test was defined using Current Procedural Terminology (CPT) codes and unique Logical Observation Identifiers Names and Codes (LOINC).

Measures

The study was divided into 3 periods: the preintervention period from August 1, 2013 to July 31, 2014; the intervention period from August 1, 2014 to January 31, 2015; and the postintervention period February 1, 2015 to January 31, 2016.

The primary outcomes measured were the frequency of guideline concordance and total costs for tests ordered per month using the Centers for Medicare and Medicaid Services (CMS) clinical laboratory fee schedule of $13.40 for troponin and $16.17 for CK-MB.⁵Concordance was defined as ≤ 3 troponin tests and no CK-MB tests ordered during 1 encounter for a patient without an ACS diagnosis in the preceding 7 days. Due to faster cellular release kinetics of CK-MB compared with that of troponin, this test has utility in evaluating new or worsening chest pain in the setting of a recent myocardial infarction (MI). Therefore, we excluded any patient who had a MI within the preceding 7 days of an order for either CK-MB or troponin tests. Additionally, the number of tests, both CK-MB and troponin, ordered per patient encounter (hereafter referred to as an episode) were measured. Finally, we measured the monthly prevalence of ACS diagnosis and percentage of visits having that diagnosis.

Data Analysis

Descriptive statistics were used to calculate population demographics of age group, sex, beneficiary category, sponsor service, and clinical setting. Monthly data were grouped into the preintervention and postintervention periods. The analysis was performed using t tests to compare mean values and CIs before and after the intervention. Simple linear regression with attention to correlation was used to create best fit lines with confidence bands before and after the intervention. Interrupted time series (ITS) regression was used to describe all data points throughout the study. Consistency between these various methods was verified. Mean values and CIs were reported from the t tests. Statistical significance was reported when appropriate. Equations and confidence predictions on the simple linear regressions were produced and reported. These were used to identify values at the start, midpoint, and end of the pre- and postintervention periods.

Results

There were a total of 6,281 patients in the study population. More patients were seen during the postintervention period than in the preintervention period. The mean age of patients was slightly higher during the preintervention period (Table 1).

Guideline Concordance

To determine whether ordering practices for cardiac enzyme testing improved, we assessed the changes in the frequency of guideline concordance during the pre- and postintervention period. On average during the preintervention year, the percentage of tests ordered that met guideline concordance was 10.1% (95% CI, 7.4%-12.9%), increasing by 0.80% (95% CI, 0.17%-1.42%) each month. 

Costs

We assessed changes in total dollars spent on cardiac enzyme testing during the pre- and postintervention periods. During the preintervention year, $9,400 (95% CI, $8,700-$10,100) was spent on average each month, which did not change significantly throughout the period. During the postintervention year, the cost was stable at $5,000 (95% CI, $4,600-$5,300) on average each month, a reduction of $4,400 (95% CI, $3,700-$5,100) (Figure 2).

CK-MB and Troponin Tests per Patient

To further assess ordering practices for cardiac enzyme testing, we compared the changes in the monthly number of tests and the average number of CK-MB and troponin tests ordered per episode pre- and postintervention. On average during the preintervention year, 297 tests (95% CI, 278-315) were run per month, with an average of 1.21 CK tests (95% CI, 1.15-1.27) per episode (Table 2, Figure 3). 

The changes in troponin testing were not as dramatic. The counts of tests each month remained similar, with a preintervention year average of 341 (95% CI, 306-377) and postintervention year average of 310 (95% CI, 287-332), which were not statistically different. However, there was a statistically significant decrease in the number of tests per episode. During the preintervention year, 1.38 troponin tests (95% CI, 1.31-1.45) were ordered per patient on average. This dropped by 0.17 (95% CI, 0.09-0.24) to the postintervention average of 1.21 (95% CI, 1.17-1.25) (Table 2, Figure 4). 

ACS Prevalence

To determine whether there was an impact on ACS diagnoses, we looked at the numbers of ACS diagnoses and their prevalence among visits before and after the intervention. During the preintervention year, the average monthly number of diagnoses was 29.7 (95% CI, 26.1-33.2), and prevalence of ACS was 0.56% (95% CI, 0.48%-0.63%) of all episodes. Although the monthly rate was statistically decreasing by 0.022% (95% CI, 0.003-0.41), this has little meaning since the level of correlation (r² = 0.2522, not displayed) was poor due to the essentially nonexistent correlation in number of visits each month (r² = 0.0112, not displayed). During the postintervention year, the average number of diagnoses was 32.2 (95% CI, 27.9-36.6), and the prevalence of ACS was 0.62% (95% CI, 0.54-0.65). Neither of these values changed significantly between the pre- and postintervention period. All ICD-9 and ICD-10 diagnosis codes used for the analysis are available upon request from the authors.

Discussion

Our data demonstrate the ability of simple process improvement interventions to decrease unnecessary testing in the workup of ACS, increasing the rate of guideline concordant testing by > 70% at a single military treatment facility (MTF). In particular, with the now widespread use of EHR, the order set presents a high-yield target for process improvement in an easily implemented, durable fashion. We had expected to see some decrease in the efficacy of the intervention at a time of staff turnover in the summer of 2015 because ongoing dedicated teaching sessions were not performed. Despite that, the intervention remained effective without further dedicated teaching sessions. This outcome was certainly attributable to the hardwired interventions made (mainly via order sets), but possibly indicates an institutional memory that can take hold after an initial concerted effort is made.

We reduced the estimated preintervention annual cost of $113,000 by $53,000 (95% CI, $42,000-$64,000). Although on a much smaller scale than the study by Larochelle, our study represents a nearly 50% reduction in the total cost of initial testing for possible ACS and a > 80% reduction in unnecessary CK-MB testing.⁴ This result was achieved with no statistical change in the prevalence of ACS. The cost reduction does not account for the labor costs to clinically follow-up and address additional unnecessary lab results. The estimated cost of intervention was limited to the time required to educate residents, interns, and nursing staff as well as the implementation of the automated, reflexive laboratory results ordering process.

Unique to our study, we also demonstrated an intervention that satisfied all the major stakeholders in the ordering of these laboratory results. By instituting the reflexive ordering of CK-MB tests for positive troponins, we obtained the support of the facility’s interventional cardiology department, which finds value in that data. Appreciating the time-sensitive nature of an ACS diagnosis, the reflexive ordering minimized the delay in receiving these data while still greatly reducing the number of tests performed. That being said, if the current trend away from CK-MB in favor of exclusively testing troponin continues, removing the reflexive ordering for positive laboratory results protocol would be an easy follow-on intervention.

Limitations

Our study presented several limitations. First, reporting errors due to improper or insufficient medical coding as well as data entry errors may exist within the MDR; therefore, the results of this analysis may be over- or underestimated. Specifically, CPT codes for troponin and CK-MB were available only in 1 of the 2 data sets used for this study, which primarily contains outpatient patient encounters. For this reason, most of the laboratory testing comes from the EMD rather than from inpatient services. However, because we excluded all patients who eventually had an ACS diagnosis (patients who likely had more inpatient time and better indication for repeat troponin), we feel that our intervention was still thoroughly investigated. Second, the number of tests drawn per patient was significantly < 2, the expected minimum number of tests to rule out ACS in patients with appropriate symptoms.

This study was not designed to answer the source of variation from guidelines. Many patients had only 1 test, which we feel represents an opportunity for future study to identify other ways cardiac enzyme testing is being used clinically. These tests might be used for patients without convincing symptoms and signs of coronary syndromes or for patients with other primary problems. Third, by using the ITS analysis, we assumed that the outcome during each intervention period follows a linear pattern. However, changes may follow a nonlinear pattern over a long period. Finally, our intervention was limited to only a single MTF, which may limit generalizability to other facilities across military medicine. However, we feel this study should serve as a guide for other MTFs as well as US Department of Veterans Affairs facilities that could institute similar process improvements.

Conclusion

We made easily implemented and durable process improvement interventions that changed institution-wide ordering practices. These changes dramatically increased the rate of guideline-concordant testing, decreasing cost and furthering the goal of high-value medical care.

In recent years, driven by accelerating health care costs and desire for improved health care value, major specialty group guidelines have incorporated resource utilization and value calculations into their recommendations. High-value care has the characteristics of enhancing outcomes, safety, and patient satisfaction at a reasonable cost. As one example, the American College of Cardiology (ACC) recently published a consensus statement on its clinical practice guidelines with a specific focus on cost and value.¹ This guideline acknowledges the difficulty in incorporating value into clinical decision making but stresses a need for increased transparency and consistency to boost value in everyday practice.

Chest pain and related symptoms were listed as the second leading principle reasons for emergency department visits in the US in 2011 with 14% of patients undergoing cardiac enzyme testing.² The ACC guidelines advocate use of troponin as the preferred laboratory test for the initial evaluation of acute coronary syndrome (ACS). Fractionated creatine kinase (CK-MB) is an acceptable alternative only when a cardiac troponin test is not available.³ Furthermore, troponins should be obtained no more than 3 times for the initial evaluation of a single event, and further trending provides no additional benefit or prognostic information.

A recent study from an academic hospital showed that process improvement interventions focused on eliminating unnecessary cardiac enzyme testing led to a 1-year cost savings of $1.25 million while increasing the rate of ACS diagnosis.⁴ Common clinical practice at Naval Medical Center Portsmouth (NMCP) in Virginia still routinely includes both troponin as well as a CK panel comprised of CK, CK-MB, and a calculated CK-MB/CK index. Our study focuses on the implementation of quality improvement efforts described by Larochelle and colleagues at NMCP.⁴ The study aimed to determine the impact of implementing interventions designed to improve the ordering practices and reduce the cost of cardiac enzyme testing.

Methods

The primary focus of the intervention was on ordering practices of the emergency medicine department (EMD), internal medicine (IM) inpatient services, and cardiology inpatient services. Specific interventions were: (1) removal of the CK panel from the chest pain order set in the EMD electronic health record (EHR); (2) removal of the CK panel from the inpatient cardiology order set; (3) education of staff on the changes in CK panel utility via direct communication during IM academic seminars; (4) education of nursing staff ordering laboratory results on behalf of physicians on the cardiology service at the morning and evening huddles; and (5) addition of “max of 3 tests indicated” comment to the inpatient EHR ordering page of the troponin test. Acknowledging that the CK-MB has some utility to interventional cardiologists in the setting of confirmed ACS, the laboratory instituted an automated, reflexive order of the CK-MB panel only if the troponin tests were positive. This test was automatically run on the same vial originally sent to the lab to mitigate any additional delay in determining results.

Data Source

The process improvement interventions were considered exempt from institutional review board (IRB) approval; however, we obtained expedited IRB approval with waiver of consent for the research aspect of the project. We obtained clinical administrative data from the Military Health System Data Repository (MDR). We identified all adult patients aged ≥ 18 years who had a troponin test, CK-MB, or both drawn at NMCP on the following services: the EMD, IM, and cardiology. A troponin or CK-MB test was defined using Current Procedural Terminology (CPT) codes and unique Logical Observation Identifiers Names and Codes (LOINC).

Measures

The study was divided into 3 periods: the preintervention period from August 1, 2013 to July 31, 2014; the intervention period from August 1, 2014 to January 31, 2015; and the postintervention period February 1, 2015 to January 31, 2016.

The primary outcomes measured were the frequency of guideline concordance and total costs for tests ordered per month using the Centers for Medicare and Medicaid Services (CMS) clinical laboratory fee schedule of $13.40 for troponin and $16.17 for CK-MB.⁵Concordance was defined as ≤ 3 troponin tests and no CK-MB tests ordered during 1 encounter for a patient without an ACS diagnosis in the preceding 7 days. Due to faster cellular release kinetics of CK-MB compared with that of troponin, this test has utility in evaluating new or worsening chest pain in the setting of a recent myocardial infarction (MI). Therefore, we excluded any patient who had a MI within the preceding 7 days of an order for either CK-MB or troponin tests. Additionally, the number of tests, both CK-MB and troponin, ordered per patient encounter (hereafter referred to as an episode) were measured. Finally, we measured the monthly prevalence of ACS diagnosis and percentage of visits having that diagnosis.

Data Analysis

Descriptive statistics were used to calculate population demographics of age group, sex, beneficiary category, sponsor service, and clinical setting. Monthly data were grouped into the preintervention and postintervention periods. The analysis was performed using t tests to compare mean values and CIs before and after the intervention. Simple linear regression with attention to correlation was used to create best fit lines with confidence bands before and after the intervention. Interrupted time series (ITS) regression was used to describe all data points throughout the study. Consistency between these various methods was verified. Mean values and CIs were reported from the t tests. Statistical significance was reported when appropriate. Equations and confidence predictions on the simple linear regressions were produced and reported. These were used to identify values at the start, midpoint, and end of the pre- and postintervention periods.

Results

There were a total of 6,281 patients in the study population. More patients were seen during the postintervention period than in the preintervention period. The mean age of patients was slightly higher during the preintervention period (Table 1).

Guideline Concordance

To determine whether ordering practices for cardiac enzyme testing improved, we assessed the changes in the frequency of guideline concordance during the pre- and postintervention period. On average during the preintervention year, the percentage of tests ordered that met guideline concordance was 10.1% (95% CI, 7.4%-12.9%), increasing by 0.80% (95% CI, 0.17%-1.42%) each month. 

Costs

We assessed changes in total dollars spent on cardiac enzyme testing during the pre- and postintervention periods. During the preintervention year, $9,400 (95% CI, $8,700-$10,100) was spent on average each month, which did not change significantly throughout the period. During the postintervention year, the cost was stable at $5,000 (95% CI, $4,600-$5,300) on average each month, a reduction of $4,400 (95% CI, $3,700-$5,100) (Figure 2).

CK-MB and Troponin Tests per Patient

To further assess ordering practices for cardiac enzyme testing, we compared the changes in the monthly number of tests and the average number of CK-MB and troponin tests ordered per episode pre- and postintervention. On average during the preintervention year, 297 tests (95% CI, 278-315) were run per month, with an average of 1.21 CK tests (95% CI, 1.15-1.27) per episode (Table 2, Figure 3). 

The changes in troponin testing were not as dramatic. The counts of tests each month remained similar, with a preintervention year average of 341 (95% CI, 306-377) and postintervention year average of 310 (95% CI, 287-332), which were not statistically different. However, there was a statistically significant decrease in the number of tests per episode. During the preintervention year, 1.38 troponin tests (95% CI, 1.31-1.45) were ordered per patient on average. This dropped by 0.17 (95% CI, 0.09-0.24) to the postintervention average of 1.21 (95% CI, 1.17-1.25) (Table 2, Figure 4). 

ACS Prevalence

To determine whether there was an impact on ACS diagnoses, we looked at the numbers of ACS diagnoses and their prevalence among visits before and after the intervention. During the preintervention year, the average monthly number of diagnoses was 29.7 (95% CI, 26.1-33.2), and prevalence of ACS was 0.56% (95% CI, 0.48%-0.63%) of all episodes. Although the monthly rate was statistically decreasing by 0.022% (95% CI, 0.003-0.41), this has little meaning since the level of correlation (r² = 0.2522, not displayed) was poor due to the essentially nonexistent correlation in number of visits each month (r² = 0.0112, not displayed). During the postintervention year, the average number of diagnoses was 32.2 (95% CI, 27.9-36.6), and the prevalence of ACS was 0.62% (95% CI, 0.54-0.65). Neither of these values changed significantly between the pre- and postintervention period. All ICD-9 and ICD-10 diagnosis codes used for the analysis are available upon request from the authors.

Discussion

Our data demonstrate the ability of simple process improvement interventions to decrease unnecessary testing in the workup of ACS, increasing the rate of guideline concordant testing by > 70% at a single military treatment facility (MTF). In particular, with the now widespread use of EHR, the order set presents a high-yield target for process improvement in an easily implemented, durable fashion. We had expected to see some decrease in the efficacy of the intervention at a time of staff turnover in the summer of 2015 because ongoing dedicated teaching sessions were not performed. Despite that, the intervention remained effective without further dedicated teaching sessions. This outcome was certainly attributable to the hardwired interventions made (mainly via order sets), but possibly indicates an institutional memory that can take hold after an initial concerted effort is made.

We reduced the estimated preintervention annual cost of $113,000 by $53,000 (95% CI, $42,000-$64,000). Although on a much smaller scale than the study by Larochelle, our study represents a nearly 50% reduction in the total cost of initial testing for possible ACS and a > 80% reduction in unnecessary CK-MB testing.⁴ This result was achieved with no statistical change in the prevalence of ACS. The cost reduction does not account for the labor costs to clinically follow-up and address additional unnecessary lab results. The estimated cost of intervention was limited to the time required to educate residents, interns, and nursing staff as well as the implementation of the automated, reflexive laboratory results ordering process.

Unique to our study, we also demonstrated an intervention that satisfied all the major stakeholders in the ordering of these laboratory results. By instituting the reflexive ordering of CK-MB tests for positive troponins, we obtained the support of the facility’s interventional cardiology department, which finds value in that data. Appreciating the time-sensitive nature of an ACS diagnosis, the reflexive ordering minimized the delay in receiving these data while still greatly reducing the number of tests performed. That being said, if the current trend away from CK-MB in favor of exclusively testing troponin continues, removing the reflexive ordering for positive laboratory results protocol would be an easy follow-on intervention.

Limitations

Our study presented several limitations. First, reporting errors due to improper or insufficient medical coding as well as data entry errors may exist within the MDR; therefore, the results of this analysis may be over- or underestimated. Specifically, CPT codes for troponin and CK-MB were available only in 1 of the 2 data sets used for this study, which primarily contains outpatient patient encounters. For this reason, most of the laboratory testing comes from the EMD rather than from inpatient services. However, because we excluded all patients who eventually had an ACS diagnosis (patients who likely had more inpatient time and better indication for repeat troponin), we feel that our intervention was still thoroughly investigated. Second, the number of tests drawn per patient was significantly < 2, the expected minimum number of tests to rule out ACS in patients with appropriate symptoms.

This study was not designed to answer the source of variation from guidelines. Many patients had only 1 test, which we feel represents an opportunity for future study to identify other ways cardiac enzyme testing is being used clinically. These tests might be used for patients without convincing symptoms and signs of coronary syndromes or for patients with other primary problems. Third, by using the ITS analysis, we assumed that the outcome during each intervention period follows a linear pattern. However, changes may follow a nonlinear pattern over a long period. Finally, our intervention was limited to only a single MTF, which may limit generalizability to other facilities across military medicine. However, we feel this study should serve as a guide for other MTFs as well as US Department of Veterans Affairs facilities that could institute similar process improvements.

Conclusion

We made easily implemented and durable process improvement interventions that changed institution-wide ordering practices. These changes dramatically increased the rate of guideline-concordant testing, decreasing cost and furthering the goal of high-value medical care.

Postgraduate training for physician assistants (PAs) and nurse practitioners (NPs) is a rapidly evolving field. It has been estimated that the number of these advanced practice providers (APPs) almost doubled between 2000 and 2016 (from 15.3 to 28.2 per 100 physicians) and is expected to double again by 2030.¹ As APPs continue to become a progressively larger part of the healthcare workforce, medical organizations are seeking more comprehensive strategies to train and mentor them.² This has led to the development of formal postgraduate programs, often called APP fellowships.

Historically, postgraduate APP fellowships have functioned to help bridge the gap in clinical practice experience between physicians and APPs.³ This gap is evident in hours of clinical training. Whereas NPs are generally expected to complete 500-1,500 hours of clinical practice before graduating,⁴ and PAs are expected to complete 2,000 hours,⁵ most physicians will complete over 15,000 hours of clinical training by the end of residency.⁶ As increasing patient complexity continues to challenge the healthcare workforce,⁷ both the NP and the PA leadership have recommended increased training of graduates and outcome studies of formal postgraduate fellowships.^8,9 In 2007, there were over 60 of these programs in the United States,¹⁰ most of them offering training in surgical specialties.

First described in 2010 by the Mayo Clinic,¹¹ APP fellowships in hospital medicine are also being developed. These programs are built to improve the training of nonphysician hospitalists, who often work independently¹² and manage medically complex patients.¹³ However, little is known about the number or structure of these fellowships. The limited understanding of the current APP fellowship environment is partly due to the lack of an administrative body overseeing these programs.¹⁴ The Accreditation Review Commission on Education for the Physician Assistant (ARC-PA) pioneered a model in 2007 for postgraduate PA programs, but it has been held in abeyance since 2014.¹⁵ Both the American Nurses Credentialing Center and the National Nurse Practitioner Residency and Fellowship Training Consortium have fellowship accreditation review processes, but they are not specific to hospital medicine.¹⁶ The Society of Hospital Medicine (SHM) has several resources for the training of APPs;¹⁷ however, it neither reviews nor accredits fellowship programs. Without standards, guidelines, or active accrediting bodies, APP fellowships in hospital medicine are poorly understood and are of unknown efficacy. The purpose of this study was to identify and describe the active APP fellowships in hospital medicine.

This was a cross-sectional study of all APP adult and pediatric fellowships in hospital medicine, in the United States, that were identifiable through May 2018. Multiple methods were used to identify all active fellowships. First, all training programs offering a Hospital Medicine Fellowship in the ARC-PA and Association of Postgraduate PA Programs databases were noted. Second, questionnaires were given out at the NP/PA forum at the national SHM conference in 2018 to gather information on existing APP fellowships. Third, similar online requests to identify known programs were posted to the SHM web forum Hospital Medicine Exchange (HMX). Fourth, Internet searches were used to discover additional programs. Once those fellowships were identified, surveys were sent to their program directors (PDs). These surveys not only asked the PDs about their fellowship but also asked them to identify additional APP fellowships beyond those that we had captured. Once additional programs were identified, a second round of surveys was sent to their PDs. This was performed in an iterative fashion until no additional fellowships were discovered.

The survey tool was developed and validated internally in the AAMC Survey Development style¹⁸ and was influenced by prior validated surveys of postgraduate medical fellowships.^10,19-21 Each question was developed by a team that had expertise in survey design (Wright and Tackett), and two survey design team members were themselves PDs of APP fellowships in hospital medicine (Kisuule and Franco). The survey was revised iteratively by the team on the basis of meetings and pilot testing with PDs of other programs. All qualitative or descriptive questions had a free response option available to allow PDs to answer the survey accurately and exhaustively. The final version of the survey was approved by consensus of all authors. It consisted of 25 multiple choice questions which were created to gather information about the following key areas of APP hospital medicine fellowships: fellowship and learner characteristics, program rationales, curricula, and methods of fellow assessment.

A web-based survey format (Qualtrics) was used to distribute the questionnaire e-mail to the PDs. Follow up e-mail reminders were sent to all nonresponders to encourage full participation. Survey completion was voluntary; no financial incentives or gifts were offered. IRB approval was obtained at Johns Hopkins Bayview (IRB number 00181629). Descriptive statistics (proportions, means, and ranges as appropriate) were calculated for all variables. Stata 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas. StataCorp LP) was used for data analysis.

In total, 11 fellowships were identified using our multimethod approach. We found four (36%) programs by utilizing existing online databases, two (18%) through the SHM questionnaire and HMX forum, three (27%) through internet searches, and the remaining two (18%) were referred to us by the other PDs who were surveyed. Of the programs surveyed, 10 were adult programs and one was a pediatric program. Surveys were sent to the PDs of the 11 fellowships, and all but one of them (10/11, 91%) responded. Respondent programs were given alphabetical designations A through J (Table). 

Fellowship and Individual Characteristics

Most programs have been in existence for five years or fewer. Eighty percent of the programs are about one year in duration; two outlier programs have fellowship lengths of six months and 18 months. The main hospital where training occurs has a mean of 496 beds (range 213 to 900). Ninety percent of the hospitals also have physician residency training programs. Sixty percent of programs enroll two to four fellows per year while 40% enroll five or more. The salary range paid by the programs is $55,000 to >$70,000, and half the programs pay more than $65,000.

The majority of fellows accepted into APP fellowships in hospital medicine are women. Eighty percent of fellows are 26-30 years old, and 90% of fellows have been out of NP or PA school for one year or less. Both NP and PA applicants are accepted in 80% of fellowships.

Program Rationales

All programs reported that training and retaining applicants is the main driver for developing their fellowship, and 50% of them offer financial incentives for retention upon successful completion of the program. Forty percent of PDs stated that there is an implicit or explicit understanding that successful completion of the fellowship would result in further employment. Over the last five years, 89% (range: 71%-100%) of graduates were asked to remain for a full-time position after program completion.

In addition to training and retention, building an interprofessional team (50%), managing patient volume (30%), and reducing overhead (20%) were also reported as rationales for program development. The majority of programs (80%) have fellows bill for clinical services, and five of those eight programs do so after their fellows become more clinically competent.

Curricula

Of the nine adult programs, 67% teach explicitly to SHM core competencies and 33% send their fellows to the SHM NP/PA Boot Camp. Thirty percent of fellowships partner formally with either a physician residency or a local PA program to develop educational content. Six of the nine programs with active physician residencies, including the pediatric fellowship, offer shared educational experiences for the residents and APPs.

There are notable differences in clinical rotations between the programs (Figure 1). No single rotation is universally required, although general hospital internal medicine is required in all adult fellowships. The majority (80%) of programs offer at least one elective. Six programs reported mandatory rotations outside the department of medicine, most commonly neurology or the stroke service (four programs). Only one program reported only general medicine rotations, with no subspecialty electives.

Methods of Fellow Assessment

Each program surveyed has a unique method of fellow assessment. Ninety percent of the programs use more than one method to assess their fellows. Faculty reviews are most commonly used and are conducted in all rotations in 80% of fellowships. Both self-assessment exercises and written examinations are used in some rotations by the majority of programs. Capstone projects are required infrequently (30%).

We found several commonalities between the fellowships surveyed. Many of the program characteristics, such as years in operation, salary, duration, and lack of accreditation, are quite similar. Most fellowships also have a similar rationale for building their programs and use resources from the SHM to inform their curricula. Fellows, on average, share several demographic characteristics, such as age, gender, and time out of schooling. Conversely, we found wide variability in clinical rotations, the general teaching structure, and methods of fellow evaluation.

There have been several publications detailing successful individual APP fellowships in medical subspecialties,²² psychiatry,²³ and surgical specialties,²⁴ all of which describe the benefits to the institution. One study found that physician hospitalists have a poor understanding of the training PAs undergo and would favor a standardized curriculum for PA hospitalists.²⁵ Another study compared all PA postgraduate training programs in emergency medicine;¹⁹ it also described a small number of relatively young programs with variable curricula and a need for standardization. Yet another paper¹⁰ surveyed postgraduate PA programs across all specialties; however, that study only captured two hospital medicine programs, and it was not focused on several key areas studied in this paper—such as the program rationale, curricular elements, and assessment.

It is noteworthy that every program surveyed was created with training and retention in mind, rather than other factors like decreasing overhead or managing patient volume. Training one’s own APPs so that they can learn on the job, come to understand expectations within a group, and witness the culture is extremely valuable. From a patient safety standpoint, it has been documented that physician hospitalists straight out of residency have a higher patient mortality compared with more experienced providers.²⁶ Given the findings that on a national level, the majority of hospitalist NPs and PAs practice autonomously or somewhat autonomously,¹² it is reasonable to assume that similar trends of more experienced providers delivering safer care would be expected for APPs, but this remains speculative. From a retention standpoint, it has been well described that high APP turnover is often due to decreased feelings of competence and confidence during their transition from trainees to medical providers.²⁷ APPs who have completed fellowships feel more confident and able to succeed in their field.²⁸ To this point, in one survey of hospitalist PAs, almost all reported that they would have been interested in completing a fellowship, even it meant a lower initial salary.²⁹Despite having the same general goals and using similar national resources, our study reveals that APP fellows are trained and assessed very differently between programs. This might represent an area of future growth in the field of hospitalist APP education. For physician learning, competency-based medical education (CBME) has emerged as a learner centric, outcomes-based model of teaching and assessment that emphasizes mastery of skills and progression through milestones.³⁰ Both the ACGME³¹ and the SHM³² have described core competencies that provide a framework within CBME for determining readiness for independent practice. While we were not surprised to find that each fellowship has its own unique method of determining readiness for practice, these findings suggest that graduates from different programs likely have very different skill sets and aptitude levels. In the future, an active accrediting body could offer guidance in defining hospitalist APP core competencies and help standardize education.

Several limitations to this study should be considered. While we used multiple strategies to locate as many fellowships as possible, it is unlikely that we successfully captured all existing programs, and new programs are being developed annually. We also relied on self-reported data from PDs. While we would expect PDs to provide accurate data, we could not externally validate their answers. Additionally, although our survey tool was reviewed extensively and validated internally, it was developed de novo for this study.

APP fellowships in hospital medicine have experienced marked growth since the first program was described in 2010. The majority of programs are 12 months long, operate in existing teaching centers, and are intended to further enhance the training and retention of newly graduated PAs and NPs. Despite their similarities, fellowships have striking variability in their methods of teaching and assessing their learners. Best practices have yet to be identified, and further study is required to determine how to standardize curricula across the board.

Acknowledgments

Disclosures

The authors report no conflicts of interest.

Funding

This project was supported by the Johns Hopkins School of Medicine Biostatistics, Epidemiology and Data Management (BEAD) Core. Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine, which is supported through the Johns Hopkins’ Center for Innovative Medicine.

Postgraduate training for physician assistants (PAs) and nurse practitioners (NPs) is a rapidly evolving field. It has been estimated that the number of these advanced practice providers (APPs) almost doubled between 2000 and 2016 (from 15.3 to 28.2 per 100 physicians) and is expected to double again by 2030.¹ As APPs continue to become a progressively larger part of the healthcare workforce, medical organizations are seeking more comprehensive strategies to train and mentor them.² This has led to the development of formal postgraduate programs, often called APP fellowships.

Historically, postgraduate APP fellowships have functioned to help bridge the gap in clinical practice experience between physicians and APPs.³ This gap is evident in hours of clinical training. Whereas NPs are generally expected to complete 500-1,500 hours of clinical practice before graduating,⁴ and PAs are expected to complete 2,000 hours,⁵ most physicians will complete over 15,000 hours of clinical training by the end of residency.⁶ As increasing patient complexity continues to challenge the healthcare workforce,⁷ both the NP and the PA leadership have recommended increased training of graduates and outcome studies of formal postgraduate fellowships.^8,9 In 2007, there were over 60 of these programs in the United States,¹⁰ most of them offering training in surgical specialties.

First described in 2010 by the Mayo Clinic,¹¹ APP fellowships in hospital medicine are also being developed. These programs are built to improve the training of nonphysician hospitalists, who often work independently¹² and manage medically complex patients.¹³ However, little is known about the number or structure of these fellowships. The limited understanding of the current APP fellowship environment is partly due to the lack of an administrative body overseeing these programs.¹⁴ The Accreditation Review Commission on Education for the Physician Assistant (ARC-PA) pioneered a model in 2007 for postgraduate PA programs, but it has been held in abeyance since 2014.¹⁵ Both the American Nurses Credentialing Center and the National Nurse Practitioner Residency and Fellowship Training Consortium have fellowship accreditation review processes, but they are not specific to hospital medicine.¹⁶ The Society of Hospital Medicine (SHM) has several resources for the training of APPs;¹⁷ however, it neither reviews nor accredits fellowship programs. Without standards, guidelines, or active accrediting bodies, APP fellowships in hospital medicine are poorly understood and are of unknown efficacy. The purpose of this study was to identify and describe the active APP fellowships in hospital medicine.

This was a cross-sectional study of all APP adult and pediatric fellowships in hospital medicine, in the United States, that were identifiable through May 2018. Multiple methods were used to identify all active fellowships. First, all training programs offering a Hospital Medicine Fellowship in the ARC-PA and Association of Postgraduate PA Programs databases were noted. Second, questionnaires were given out at the NP/PA forum at the national SHM conference in 2018 to gather information on existing APP fellowships. Third, similar online requests to identify known programs were posted to the SHM web forum Hospital Medicine Exchange (HMX). Fourth, Internet searches were used to discover additional programs. Once those fellowships were identified, surveys were sent to their program directors (PDs). These surveys not only asked the PDs about their fellowship but also asked them to identify additional APP fellowships beyond those that we had captured. Once additional programs were identified, a second round of surveys was sent to their PDs. This was performed in an iterative fashion until no additional fellowships were discovered.

The survey tool was developed and validated internally in the AAMC Survey Development style¹⁸ and was influenced by prior validated surveys of postgraduate medical fellowships.^10,19-21 Each question was developed by a team that had expertise in survey design (Wright and Tackett), and two survey design team members were themselves PDs of APP fellowships in hospital medicine (Kisuule and Franco). The survey was revised iteratively by the team on the basis of meetings and pilot testing with PDs of other programs. All qualitative or descriptive questions had a free response option available to allow PDs to answer the survey accurately and exhaustively. The final version of the survey was approved by consensus of all authors. It consisted of 25 multiple choice questions which were created to gather information about the following key areas of APP hospital medicine fellowships: fellowship and learner characteristics, program rationales, curricula, and methods of fellow assessment.

A web-based survey format (Qualtrics) was used to distribute the questionnaire e-mail to the PDs. Follow up e-mail reminders were sent to all nonresponders to encourage full participation. Survey completion was voluntary; no financial incentives or gifts were offered. IRB approval was obtained at Johns Hopkins Bayview (IRB number 00181629). Descriptive statistics (proportions, means, and ranges as appropriate) were calculated for all variables. Stata 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas. StataCorp LP) was used for data analysis.

In total, 11 fellowships were identified using our multimethod approach. We found four (36%) programs by utilizing existing online databases, two (18%) through the SHM questionnaire and HMX forum, three (27%) through internet searches, and the remaining two (18%) were referred to us by the other PDs who were surveyed. Of the programs surveyed, 10 were adult programs and one was a pediatric program. Surveys were sent to the PDs of the 11 fellowships, and all but one of them (10/11, 91%) responded. Respondent programs were given alphabetical designations A through J (Table). 

Fellowship and Individual Characteristics

Most programs have been in existence for five years or fewer. Eighty percent of the programs are about one year in duration; two outlier programs have fellowship lengths of six months and 18 months. The main hospital where training occurs has a mean of 496 beds (range 213 to 900). Ninety percent of the hospitals also have physician residency training programs. Sixty percent of programs enroll two to four fellows per year while 40% enroll five or more. The salary range paid by the programs is $55,000 to >$70,000, and half the programs pay more than $65,000.

The majority of fellows accepted into APP fellowships in hospital medicine are women. Eighty percent of fellows are 26-30 years old, and 90% of fellows have been out of NP or PA school for one year or less. Both NP and PA applicants are accepted in 80% of fellowships.

Program Rationales

All programs reported that training and retaining applicants is the main driver for developing their fellowship, and 50% of them offer financial incentives for retention upon successful completion of the program. Forty percent of PDs stated that there is an implicit or explicit understanding that successful completion of the fellowship would result in further employment. Over the last five years, 89% (range: 71%-100%) of graduates were asked to remain for a full-time position after program completion.

In addition to training and retention, building an interprofessional team (50%), managing patient volume (30%), and reducing overhead (20%) were also reported as rationales for program development. The majority of programs (80%) have fellows bill for clinical services, and five of those eight programs do so after their fellows become more clinically competent.

Curricula

Of the nine adult programs, 67% teach explicitly to SHM core competencies and 33% send their fellows to the SHM NP/PA Boot Camp. Thirty percent of fellowships partner formally with either a physician residency or a local PA program to develop educational content. Six of the nine programs with active physician residencies, including the pediatric fellowship, offer shared educational experiences for the residents and APPs.

There are notable differences in clinical rotations between the programs (Figure 1). No single rotation is universally required, although general hospital internal medicine is required in all adult fellowships. The majority (80%) of programs offer at least one elective. Six programs reported mandatory rotations outside the department of medicine, most commonly neurology or the stroke service (four programs). Only one program reported only general medicine rotations, with no subspecialty electives.

Methods of Fellow Assessment

Each program surveyed has a unique method of fellow assessment. Ninety percent of the programs use more than one method to assess their fellows. Faculty reviews are most commonly used and are conducted in all rotations in 80% of fellowships. Both self-assessment exercises and written examinations are used in some rotations by the majority of programs. Capstone projects are required infrequently (30%).

We found several commonalities between the fellowships surveyed. Many of the program characteristics, such as years in operation, salary, duration, and lack of accreditation, are quite similar. Most fellowships also have a similar rationale for building their programs and use resources from the SHM to inform their curricula. Fellows, on average, share several demographic characteristics, such as age, gender, and time out of schooling. Conversely, we found wide variability in clinical rotations, the general teaching structure, and methods of fellow evaluation.

There have been several publications detailing successful individual APP fellowships in medical subspecialties,²² psychiatry,²³ and surgical specialties,²⁴ all of which describe the benefits to the institution. One study found that physician hospitalists have a poor understanding of the training PAs undergo and would favor a standardized curriculum for PA hospitalists.²⁵ Another study compared all PA postgraduate training programs in emergency medicine;¹⁹ it also described a small number of relatively young programs with variable curricula and a need for standardization. Yet another paper¹⁰ surveyed postgraduate PA programs across all specialties; however, that study only captured two hospital medicine programs, and it was not focused on several key areas studied in this paper—such as the program rationale, curricular elements, and assessment.

It is noteworthy that every program surveyed was created with training and retention in mind, rather than other factors like decreasing overhead or managing patient volume. Training one’s own APPs so that they can learn on the job, come to understand expectations within a group, and witness the culture is extremely valuable. From a patient safety standpoint, it has been documented that physician hospitalists straight out of residency have a higher patient mortality compared with more experienced providers.²⁶ Given the findings that on a national level, the majority of hospitalist NPs and PAs practice autonomously or somewhat autonomously,¹² it is reasonable to assume that similar trends of more experienced providers delivering safer care would be expected for APPs, but this remains speculative. From a retention standpoint, it has been well described that high APP turnover is often due to decreased feelings of competence and confidence during their transition from trainees to medical providers.²⁷ APPs who have completed fellowships feel more confident and able to succeed in their field.²⁸ To this point, in one survey of hospitalist PAs, almost all reported that they would have been interested in completing a fellowship, even it meant a lower initial salary.²⁹Despite having the same general goals and using similar national resources, our study reveals that APP fellows are trained and assessed very differently between programs. This might represent an area of future growth in the field of hospitalist APP education. For physician learning, competency-based medical education (CBME) has emerged as a learner centric, outcomes-based model of teaching and assessment that emphasizes mastery of skills and progression through milestones.³⁰ Both the ACGME³¹ and the SHM³² have described core competencies that provide a framework within CBME for determining readiness for independent practice. While we were not surprised to find that each fellowship has its own unique method of determining readiness for practice, these findings suggest that graduates from different programs likely have very different skill sets and aptitude levels. In the future, an active accrediting body could offer guidance in defining hospitalist APP core competencies and help standardize education.

Several limitations to this study should be considered. While we used multiple strategies to locate as many fellowships as possible, it is unlikely that we successfully captured all existing programs, and new programs are being developed annually. We also relied on self-reported data from PDs. While we would expect PDs to provide accurate data, we could not externally validate their answers. Additionally, although our survey tool was reviewed extensively and validated internally, it was developed de novo for this study.

APP fellowships in hospital medicine have experienced marked growth since the first program was described in 2010. The majority of programs are 12 months long, operate in existing teaching centers, and are intended to further enhance the training and retention of newly graduated PAs and NPs. Despite their similarities, fellowships have striking variability in their methods of teaching and assessing their learners. Best practices have yet to be identified, and further study is required to determine how to standardize curricula across the board.

Acknowledgments

Disclosures

The authors report no conflicts of interest.

Funding

This project was supported by the Johns Hopkins School of Medicine Biostatistics, Epidemiology and Data Management (BEAD) Core. Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine, which is supported through the Johns Hopkins’ Center for Innovative Medicine.

Postgraduate training for physician assistants (PAs) and nurse practitioners (NPs) is a rapidly evolving field. It has been estimated that the number of these advanced practice providers (APPs) almost doubled between 2000 and 2016 (from 15.3 to 28.2 per 100 physicians) and is expected to double again by 2030.¹ As APPs continue to become a progressively larger part of the healthcare workforce, medical organizations are seeking more comprehensive strategies to train and mentor them.² This has led to the development of formal postgraduate programs, often called APP fellowships.

Historically, postgraduate APP fellowships have functioned to help bridge the gap in clinical practice experience between physicians and APPs.³ This gap is evident in hours of clinical training. Whereas NPs are generally expected to complete 500-1,500 hours of clinical practice before graduating,⁴ and PAs are expected to complete 2,000 hours,⁵ most physicians will complete over 15,000 hours of clinical training by the end of residency.⁶ As increasing patient complexity continues to challenge the healthcare workforce,⁷ both the NP and the PA leadership have recommended increased training of graduates and outcome studies of formal postgraduate fellowships.^8,9 In 2007, there were over 60 of these programs in the United States,¹⁰ most of them offering training in surgical specialties.

First described in 2010 by the Mayo Clinic,¹¹ APP fellowships in hospital medicine are also being developed. These programs are built to improve the training of nonphysician hospitalists, who often work independently¹² and manage medically complex patients.¹³ However, little is known about the number or structure of these fellowships. The limited understanding of the current APP fellowship environment is partly due to the lack of an administrative body overseeing these programs.¹⁴ The Accreditation Review Commission on Education for the Physician Assistant (ARC-PA) pioneered a model in 2007 for postgraduate PA programs, but it has been held in abeyance since 2014.¹⁵ Both the American Nurses Credentialing Center and the National Nurse Practitioner Residency and Fellowship Training Consortium have fellowship accreditation review processes, but they are not specific to hospital medicine.¹⁶ The Society of Hospital Medicine (SHM) has several resources for the training of APPs;¹⁷ however, it neither reviews nor accredits fellowship programs. Without standards, guidelines, or active accrediting bodies, APP fellowships in hospital medicine are poorly understood and are of unknown efficacy. The purpose of this study was to identify and describe the active APP fellowships in hospital medicine.

This was a cross-sectional study of all APP adult and pediatric fellowships in hospital medicine, in the United States, that were identifiable through May 2018. Multiple methods were used to identify all active fellowships. First, all training programs offering a Hospital Medicine Fellowship in the ARC-PA and Association of Postgraduate PA Programs databases were noted. Second, questionnaires were given out at the NP/PA forum at the national SHM conference in 2018 to gather information on existing APP fellowships. Third, similar online requests to identify known programs were posted to the SHM web forum Hospital Medicine Exchange (HMX). Fourth, Internet searches were used to discover additional programs. Once those fellowships were identified, surveys were sent to their program directors (PDs). These surveys not only asked the PDs about their fellowship but also asked them to identify additional APP fellowships beyond those that we had captured. Once additional programs were identified, a second round of surveys was sent to their PDs. This was performed in an iterative fashion until no additional fellowships were discovered.

The survey tool was developed and validated internally in the AAMC Survey Development style¹⁸ and was influenced by prior validated surveys of postgraduate medical fellowships.^10,19-21 Each question was developed by a team that had expertise in survey design (Wright and Tackett), and two survey design team members were themselves PDs of APP fellowships in hospital medicine (Kisuule and Franco). The survey was revised iteratively by the team on the basis of meetings and pilot testing with PDs of other programs. All qualitative or descriptive questions had a free response option available to allow PDs to answer the survey accurately and exhaustively. The final version of the survey was approved by consensus of all authors. It consisted of 25 multiple choice questions which were created to gather information about the following key areas of APP hospital medicine fellowships: fellowship and learner characteristics, program rationales, curricula, and methods of fellow assessment.

A web-based survey format (Qualtrics) was used to distribute the questionnaire e-mail to the PDs. Follow up e-mail reminders were sent to all nonresponders to encourage full participation. Survey completion was voluntary; no financial incentives or gifts were offered. IRB approval was obtained at Johns Hopkins Bayview (IRB number 00181629). Descriptive statistics (proportions, means, and ranges as appropriate) were calculated for all variables. Stata 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas. StataCorp LP) was used for data analysis.

In total, 11 fellowships were identified using our multimethod approach. We found four (36%) programs by utilizing existing online databases, two (18%) through the SHM questionnaire and HMX forum, three (27%) through internet searches, and the remaining two (18%) were referred to us by the other PDs who were surveyed. Of the programs surveyed, 10 were adult programs and one was a pediatric program. Surveys were sent to the PDs of the 11 fellowships, and all but one of them (10/11, 91%) responded. Respondent programs were given alphabetical designations A through J (Table). 

Fellowship and Individual Characteristics

Most programs have been in existence for five years or fewer. Eighty percent of the programs are about one year in duration; two outlier programs have fellowship lengths of six months and 18 months. The main hospital where training occurs has a mean of 496 beds (range 213 to 900). Ninety percent of the hospitals also have physician residency training programs. Sixty percent of programs enroll two to four fellows per year while 40% enroll five or more. The salary range paid by the programs is $55,000 to >$70,000, and half the programs pay more than $65,000.

The majority of fellows accepted into APP fellowships in hospital medicine are women. Eighty percent of fellows are 26-30 years old, and 90% of fellows have been out of NP or PA school for one year or less. Both NP and PA applicants are accepted in 80% of fellowships.

Program Rationales

All programs reported that training and retaining applicants is the main driver for developing their fellowship, and 50% of them offer financial incentives for retention upon successful completion of the program. Forty percent of PDs stated that there is an implicit or explicit understanding that successful completion of the fellowship would result in further employment. Over the last five years, 89% (range: 71%-100%) of graduates were asked to remain for a full-time position after program completion.

In addition to training and retention, building an interprofessional team (50%), managing patient volume (30%), and reducing overhead (20%) were also reported as rationales for program development. The majority of programs (80%) have fellows bill for clinical services, and five of those eight programs do so after their fellows become more clinically competent.

Curricula

Of the nine adult programs, 67% teach explicitly to SHM core competencies and 33% send their fellows to the SHM NP/PA Boot Camp. Thirty percent of fellowships partner formally with either a physician residency or a local PA program to develop educational content. Six of the nine programs with active physician residencies, including the pediatric fellowship, offer shared educational experiences for the residents and APPs.

There are notable differences in clinical rotations between the programs (Figure 1). No single rotation is universally required, although general hospital internal medicine is required in all adult fellowships. The majority (80%) of programs offer at least one elective. Six programs reported mandatory rotations outside the department of medicine, most commonly neurology or the stroke service (four programs). Only one program reported only general medicine rotations, with no subspecialty electives.

Methods of Fellow Assessment

Each program surveyed has a unique method of fellow assessment. Ninety percent of the programs use more than one method to assess their fellows. Faculty reviews are most commonly used and are conducted in all rotations in 80% of fellowships. Both self-assessment exercises and written examinations are used in some rotations by the majority of programs. Capstone projects are required infrequently (30%).

We found several commonalities between the fellowships surveyed. Many of the program characteristics, such as years in operation, salary, duration, and lack of accreditation, are quite similar. Most fellowships also have a similar rationale for building their programs and use resources from the SHM to inform their curricula. Fellows, on average, share several demographic characteristics, such as age, gender, and time out of schooling. Conversely, we found wide variability in clinical rotations, the general teaching structure, and methods of fellow evaluation.

There have been several publications detailing successful individual APP fellowships in medical subspecialties,²² psychiatry,²³ and surgical specialties,²⁴ all of which describe the benefits to the institution. One study found that physician hospitalists have a poor understanding of the training PAs undergo and would favor a standardized curriculum for PA hospitalists.²⁵ Another study compared all PA postgraduate training programs in emergency medicine;¹⁹ it also described a small number of relatively young programs with variable curricula and a need for standardization. Yet another paper¹⁰ surveyed postgraduate PA programs across all specialties; however, that study only captured two hospital medicine programs, and it was not focused on several key areas studied in this paper—such as the program rationale, curricular elements, and assessment.

It is noteworthy that every program surveyed was created with training and retention in mind, rather than other factors like decreasing overhead or managing patient volume. Training one’s own APPs so that they can learn on the job, come to understand expectations within a group, and witness the culture is extremely valuable. From a patient safety standpoint, it has been documented that physician hospitalists straight out of residency have a higher patient mortality compared with more experienced providers.²⁶ Given the findings that on a national level, the majority of hospitalist NPs and PAs practice autonomously or somewhat autonomously,¹² it is reasonable to assume that similar trends of more experienced providers delivering safer care would be expected for APPs, but this remains speculative. From a retention standpoint, it has been well described that high APP turnover is often due to decreased feelings of competence and confidence during their transition from trainees to medical providers.²⁷ APPs who have completed fellowships feel more confident and able to succeed in their field.²⁸ To this point, in one survey of hospitalist PAs, almost all reported that they would have been interested in completing a fellowship, even it meant a lower initial salary.²⁹Despite having the same general goals and using similar national resources, our study reveals that APP fellows are trained and assessed very differently between programs. This might represent an area of future growth in the field of hospitalist APP education. For physician learning, competency-based medical education (CBME) has emerged as a learner centric, outcomes-based model of teaching and assessment that emphasizes mastery of skills and progression through milestones.³⁰ Both the ACGME³¹ and the SHM³² have described core competencies that provide a framework within CBME for determining readiness for independent practice. While we were not surprised to find that each fellowship has its own unique method of determining readiness for practice, these findings suggest that graduates from different programs likely have very different skill sets and aptitude levels. In the future, an active accrediting body could offer guidance in defining hospitalist APP core competencies and help standardize education.

Several limitations to this study should be considered. While we used multiple strategies to locate as many fellowships as possible, it is unlikely that we successfully captured all existing programs, and new programs are being developed annually. We also relied on self-reported data from PDs. While we would expect PDs to provide accurate data, we could not externally validate their answers. Additionally, although our survey tool was reviewed extensively and validated internally, it was developed de novo for this study.

APP fellowships in hospital medicine have experienced marked growth since the first program was described in 2010. The majority of programs are 12 months long, operate in existing teaching centers, and are intended to further enhance the training and retention of newly graduated PAs and NPs. Despite their similarities, fellowships have striking variability in their methods of teaching and assessing their learners. Best practices have yet to be identified, and further study is required to determine how to standardize curricula across the board.

Acknowledgments

Disclosures

The authors report no conflicts of interest.

Funding

This project was supported by the Johns Hopkins School of Medicine Biostatistics, Epidemiology and Data Management (BEAD) Core. Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine, which is supported through the Johns Hopkins’ Center for Innovative Medicine.

Inadequate bowel preparation (IBP) at the time of inpatient colonoscopy is common and associated with increased length of stay and cost of care.¹ The factors that contribute to IBP can be categorized into those that are modifiable and those that are nonmodifiable. While many factors have been associated with IBP, studies have been limited by small sample size or have combined inpatient/outpatient populations, thus limiting generalizability.^1-5 Moreover, most factors associated with IBP, such as socioeconomic status, male gender, increased age, and comorbidities, are nonmodifiable. No studies have explicitly focused on modifiable risk factors, such as medication use, colonoscopy timing, or assessed the potential impact of modifying these factors.

In a large, multihospital system, we examine the frequency of IBP among inpatients undergoing colonoscopy along with factors associated with IBP. We attempted to identify modifiable risk factors that were associated with IBP.

Potential Predictors of IBP

Demographic data such as patient age, gender, ethnicity, body mass index (BMI), and insurance/payor status were obtained from the electronic health record (EHR). International Classification of Disease 9^th and 10^th revision, Clinical Modifications (ICD-9/10-CM) codes were used to obtain patient comorbidities including diabetes, coronary artery disease, heart failure, cirrhosis, gastroparesis, hypothyroidism, inflammatory bowel disease, constipation, stroke, dementia, dysphagia, and nausea/vomiting. Use of opioid medications within three days before colonoscopy was extracted from the medication administration record. These variables were chosen as biologically plausible modifiers of bowel preparation or had previously been assessed in the literature.^1-6 The name and volume, classified as 4 L (GoLytely^®) and < 4 liters (MoviPrep^®) of bowel preparation, time of day when colonoscopy was performed, solid diet the day prior to colonoscopy, type of sedation used (conscious sedation or general anesthesia), and total colonoscopy time (defined as the time from scope insertion to removal) was recorded. Hospitalization-related variables, including the number of hospitalizations in the year before the current hospitalization, the year in which the colonoscopy was performed, and the number of days from admission to colonoscopy, were also obtained from the EHR.

Outcome Measures

An internally validated natural language algorithm, using Structured Queried Language was used to search through colonoscopy reports to identify adequacy of bowel preparation. ProVation^® software allows the gastroenterologist to use some terms to describe bowel preparation in a drop-down menu format. In addition to the Aronchik scale (which allows the gastroenterologist to rate bowel preparation on a five-point scale: “excellent,” “good,” “fair,” “poor,” and “inadequate”) it also allows the provider to use terms such as “adequate” or “adequate to detect polyps >5 mm” as well as “unsatisfactory.”⁷ Mirroring prior literature, bowel preparation quality was classified into “adequate” and “inadequate”; “good” and “excellent” on the Aronchik scale were categorized as adequate as was the term “adequate” in any form; “fair,” “poor,” or “inadequate” on the Aronchik scale were classified as inadequate as was the term “unsatisfactory.” We evaluated the hospital length of stay (LOS) as a secondary outcome measure.

Statistical Analysis

After describing the frequency of IBP, the quality of bowel preparation (adequate vs inadequate) was compared based on the predictors described above. Categorical variables were reported as frequencies with percentages and continuous variables were reported as medians with 25^th-75^th percentile values. The significance of the difference between the proportion or median values of those who had inadequate versus adequate bowel preparation was assessed. Two-sided chi-square analysis was used to assess the significance of differences between categorical variables and the Wilcoxon Rank-Sum test was used to assess the significance of differences between continuous variables.

Multivariate logistic regression analysis was performed to assess factors associated with hospital predictors and outcomes, after adjusting for all the aforementioned factors and clustering the effect based on the endoscopist. To evaluate the potential impact of modifiable factors on IBP, we performed counterfactual analysis, in which the observed distribution was compared to a hypothetical population in which all the modifiable risk factors were optimal.

Overall, 8,819 patients were included in our study population. They had a median age of 64 [53-76] years; 50.5% were female and 51% had an IBP. Patient characteristics and rates of IBP are presented in Table 1.

In unadjusted analyses, with regards to modifiable factors, opiate use within three days of colonoscopy was associated with a higher rate of IBP (55.4% vs 47.3%, P <.001), as was a lower volume (<4L) bowel preparation (55.3% vs 50.4%, P = .003). IBP was less frequent when colonoscopy was performed before noon vs afternoon (50.3% vs 57.4%, P < .001), and when patients were documented to receive a clear liquid diet or nil per os vs a solid diet the day prior to colonoscopy (50.3% vs 57.4%, P < .001). Overall bowel preparation quality improved over time (Figure 1). Median LOS was five [3-11] days. Patients who had IBP on their initial colonoscopy had a LOS one day longer than patients without IBP (six days vs five days, P < .001).

Multivariate Analysis

Table 2 shows the results of the multivariate analysis. The following modifiable factors were associated with IBP: opiate used within three days of the procedure (OR 1.31; 95% CI 1.8, 1.45), having the colonoscopy performed after12:00 pm (OR, 1.25; 95% CI, 1.10, 1.41), and consuming a solid diet the day prior to the colonoscopy (OR, 1.37; 95% CI, 1.18, 1.59). However, the volume of bowel preparation was not associated with IBP. The selected nonmodifiable factors that were found to be associated with IBP included age (increment of five years; OR, 1.04; 95% CI, 1.02, 1.05), male gender (OR, 1.33; 95% CI, 1.23, 1.44), Medicare insurance (OR, 1.17; 95% CI, 1.07, 1.28), Medicaid insurance (OR, 1.34; 95% CI, 1.07, 1.28), gastroparesis (OR, 1.62; 95% CI, 1.16, 2.27), nausea/vomiting (OR 1.21; 95% CI, 1.09, 1.34), and dysphagia (OR, 1.16; 95% CI, 1.01, 1.34).

Potential Impact of Modifiable Variables

We conducted a counterfactual analysis based on a multivariate model to assess the impact of each modifiable risk factor on the IBP rate (Figure 1). In the included study population, 44.9% received an opiate, 39.3% had a colonoscopy after 12:00 pm, and 9.1% received solid food the day prior to the procedure. Holding all other factors constant, if all patients were not prescribed opiates within three days of the procedure a 2.9% reduction in IBP would be expected. Similarly, if all patients underwent colonoscopy before noon, a 2.1% reduction in IBP rate would be expected. A 0.7% reduction would be expected if all patients were maintained on a liquid diet or nil per os. Combined, instituting all these changes (no opiates or solid diet before colonoscopy and performing all colonoscopies before noon) would produce a 5.6% reduction in IBP rate.

In this large, multihospital cohort, IBP was documented in half (51%) of 8,819 inpatient colonoscopies performed. Nonmodifiable patient characteristics independently associated with IBP were age, male gender, white race, Medicare and Medicaid insurance, nausea/vomiting, dysphagia, and gastroparesis. Modifiable factors included not consuming opiates within three days of colonoscopy, avoidance of a solid diet the day prior to colonoscopy and performing the colonoscopy before noon. The volume of bowel preparation consumed was not associated with IBP. In a counterfactual analysis, we found that if all three modifiable factors were optimized, the predicted rate of IBP would drop to 45%.

Many studies, including our analysis, have shown significant differences between the frequency of IBP in inpatient versus outpatient bowel preparations.^8-11 Therefore, it is crucial to study IBP in these settings separately. Three single-institution studies, including a total of 898 patients, have identified risk factors for inpatient IBP. Individual studies ranged in size from 130 to 524 patients with rates of IBP ranging from 22%-57%.^1-3 They found IBP to be associated with increasing age, lower income, ASA Grade >3, diabetes, coronary artery disease (CAD), nausea or vomiting, BMI >25, and chronic constipation. Modifiable factors included opiates, afternoon procedures, and runway times >6 hours.

We also found IBP to be associated with increasing age and male gender. However, we found no association with diabetes, chronic constipation, CAD or BMI. As we were able to adjust for a wider variety of variables, it is possible that we were able to account for residual confounding better than previous studies. For example, we found that having nausea/vomiting, dysphagia, and gastroparesis was associated with IBP. Gastroparesis with associated nausea and vomiting may be the mechanism by which diabetes increases the risk for IBP. Further studies are needed to assess if interventions or alternative bowel cleansing in these patients can result in improved IBP. Finally, in contrast to studies with smaller cohorts which found that lower volume bowel preps improved IBP in the right colon,^4,12 we found no association between IBP based and volume of bowel preparation consumed. Our impact analysis suggests that avoidance of opiates for at least three days before colonoscopy, avoidance of solid diet on the day before colonoscopy and performing all colonoscopies before noon would reduce the rate of IBP by 5.6%. While at first glance this does not appear to be a significant change, from a public health perspective with thousands of inpatient colonoscopies performed every year, it is crucial. We found that IBP was associated with an increased inpatient LOS of approximately one day. Assuming an average cost of one hospital day to be $2,000,¹³ this 5.6% improvement among our almost 9,000 patients, would translate into eliminating 494 unnecessary hospital days, or approximately $1 million in savings. More importantly, this savings comes without risk to patients and would result in an improvement in quality.

The factors mentioned above may not always be amenable to modification. For example, for patients with active gastrointestinal bleeding, postponing colonoscopy by one day for the sake of maintaining a patient on a clear diet may not be feasible. Similarly, performing colonoscopies in the morning is highly dependent on endoscopy suite availability and hospital logistics. Denying opiates to patients experiencing severe pain is not ethical. In many scenarios, however, these variables could be modified, and institutional efforts to support these practices could yield considerable savings. Future prospective studies are needed to verify the real impact of these changes.

Further discussion is needed to contextualize the finding that colonoscopies scheduled in the afternoon are associated with improved bowel preparation quality. Previous research—albeit in the outpatient setting—has demonstrated 11.8 hours as the maximum upper time limit for the time elapsed between the end of bowel preparation to colonoscopy.¹⁴ Another study found an inverse relationship between the quality of bowel preparation and the time after completion of the bowel preparation.¹⁵ This makes sense from a physiological perspective as delaying the time between completion of bowel preparation, and the procedure allows chyme from the small intestine to reaccumulate in the colon. Anecdotally, at our institution as well as at many others, the bowel preparations are ordered to start in the evening to allow the consumption of complete bowel preparation by midnight. As a result of this practice, only patients who have their colonoscopies scheduled before noon fall within the optimal period of 11.8 hours. In the outpatient setting, the use of split preparations has led to the obliteration of the difference in the quality of bowel preparation between morning and afternoon colonoscopies.¹⁶ Prospective trials are needed to evaluate the use of split preparations to improve the quality of afternoon inpatient colonoscopies.

In this large retrospective study evaluating bowel preparation quality in inpatients undergoing colonoscopy, we found that more than half of the patients have IBP and that IBP was associated with an extra day of hospitalization. Our study identifies those patients at highest risk and identifies modifiable risk factors for IBP. Specifically, we found that abstinence from opiates or solid diet before the colonoscopy, along with performing colonoscopies before noon were associated with improved outcomes. Prospective studies are needed to confirm the effects of these interventions on bowel preparation quality.

Disclosures

Carol A Burke, MD has received research funding from Ferring Pharmaceuticals. Other authors have no conflicts of interest to disclose.

Inadequate bowel preparation (IBP) at the time of inpatient colonoscopy is common and associated with increased length of stay and cost of care.¹ The factors that contribute to IBP can be categorized into those that are modifiable and those that are nonmodifiable. While many factors have been associated with IBP, studies have been limited by small sample size or have combined inpatient/outpatient populations, thus limiting generalizability.^1-5 Moreover, most factors associated with IBP, such as socioeconomic status, male gender, increased age, and comorbidities, are nonmodifiable. No studies have explicitly focused on modifiable risk factors, such as medication use, colonoscopy timing, or assessed the potential impact of modifying these factors.

In a large, multihospital system, we examine the frequency of IBP among inpatients undergoing colonoscopy along with factors associated with IBP. We attempted to identify modifiable risk factors that were associated with IBP.

Potential Predictors of IBP

Demographic data such as patient age, gender, ethnicity, body mass index (BMI), and insurance/payor status were obtained from the electronic health record (EHR). International Classification of Disease 9^th and 10^th revision, Clinical Modifications (ICD-9/10-CM) codes were used to obtain patient comorbidities including diabetes, coronary artery disease, heart failure, cirrhosis, gastroparesis, hypothyroidism, inflammatory bowel disease, constipation, stroke, dementia, dysphagia, and nausea/vomiting. Use of opioid medications within three days before colonoscopy was extracted from the medication administration record. These variables were chosen as biologically plausible modifiers of bowel preparation or had previously been assessed in the literature.^1-6 The name and volume, classified as 4 L (GoLytely^®) and < 4 liters (MoviPrep^®) of bowel preparation, time of day when colonoscopy was performed, solid diet the day prior to colonoscopy, type of sedation used (conscious sedation or general anesthesia), and total colonoscopy time (defined as the time from scope insertion to removal) was recorded. Hospitalization-related variables, including the number of hospitalizations in the year before the current hospitalization, the year in which the colonoscopy was performed, and the number of days from admission to colonoscopy, were also obtained from the EHR.

Outcome Measures

An internally validated natural language algorithm, using Structured Queried Language was used to search through colonoscopy reports to identify adequacy of bowel preparation. ProVation^® software allows the gastroenterologist to use some terms to describe bowel preparation in a drop-down menu format. In addition to the Aronchik scale (which allows the gastroenterologist to rate bowel preparation on a five-point scale: “excellent,” “good,” “fair,” “poor,” and “inadequate”) it also allows the provider to use terms such as “adequate” or “adequate to detect polyps >5 mm” as well as “unsatisfactory.”⁷ Mirroring prior literature, bowel preparation quality was classified into “adequate” and “inadequate”; “good” and “excellent” on the Aronchik scale were categorized as adequate as was the term “adequate” in any form; “fair,” “poor,” or “inadequate” on the Aronchik scale were classified as inadequate as was the term “unsatisfactory.” We evaluated the hospital length of stay (LOS) as a secondary outcome measure.

Statistical Analysis

After describing the frequency of IBP, the quality of bowel preparation (adequate vs inadequate) was compared based on the predictors described above. Categorical variables were reported as frequencies with percentages and continuous variables were reported as medians with 25^th-75^th percentile values. The significance of the difference between the proportion or median values of those who had inadequate versus adequate bowel preparation was assessed. Two-sided chi-square analysis was used to assess the significance of differences between categorical variables and the Wilcoxon Rank-Sum test was used to assess the significance of differences between continuous variables.

Multivariate logistic regression analysis was performed to assess factors associated with hospital predictors and outcomes, after adjusting for all the aforementioned factors and clustering the effect based on the endoscopist. To evaluate the potential impact of modifiable factors on IBP, we performed counterfactual analysis, in which the observed distribution was compared to a hypothetical population in which all the modifiable risk factors were optimal.

Overall, 8,819 patients were included in our study population. They had a median age of 64 [53-76] years; 50.5% were female and 51% had an IBP. Patient characteristics and rates of IBP are presented in Table 1.

In unadjusted analyses, with regards to modifiable factors, opiate use within three days of colonoscopy was associated with a higher rate of IBP (55.4% vs 47.3%, P <.001), as was a lower volume (<4L) bowel preparation (55.3% vs 50.4%, P = .003). IBP was less frequent when colonoscopy was performed before noon vs afternoon (50.3% vs 57.4%, P < .001), and when patients were documented to receive a clear liquid diet or nil per os vs a solid diet the day prior to colonoscopy (50.3% vs 57.4%, P < .001). Overall bowel preparation quality improved over time (Figure 1). Median LOS was five [3-11] days. Patients who had IBP on their initial colonoscopy had a LOS one day longer than patients without IBP (six days vs five days, P < .001).

Multivariate Analysis

Table 2 shows the results of the multivariate analysis. The following modifiable factors were associated with IBP: opiate used within three days of the procedure (OR 1.31; 95% CI 1.8, 1.45), having the colonoscopy performed after12:00 pm (OR, 1.25; 95% CI, 1.10, 1.41), and consuming a solid diet the day prior to the colonoscopy (OR, 1.37; 95% CI, 1.18, 1.59). However, the volume of bowel preparation was not associated with IBP. The selected nonmodifiable factors that were found to be associated with IBP included age (increment of five years; OR, 1.04; 95% CI, 1.02, 1.05), male gender (OR, 1.33; 95% CI, 1.23, 1.44), Medicare insurance (OR, 1.17; 95% CI, 1.07, 1.28), Medicaid insurance (OR, 1.34; 95% CI, 1.07, 1.28), gastroparesis (OR, 1.62; 95% CI, 1.16, 2.27), nausea/vomiting (OR 1.21; 95% CI, 1.09, 1.34), and dysphagia (OR, 1.16; 95% CI, 1.01, 1.34).

Potential Impact of Modifiable Variables

We conducted a counterfactual analysis based on a multivariate model to assess the impact of each modifiable risk factor on the IBP rate (Figure 1). In the included study population, 44.9% received an opiate, 39.3% had a colonoscopy after 12:00 pm, and 9.1% received solid food the day prior to the procedure. Holding all other factors constant, if all patients were not prescribed opiates within three days of the procedure a 2.9% reduction in IBP would be expected. Similarly, if all patients underwent colonoscopy before noon, a 2.1% reduction in IBP rate would be expected. A 0.7% reduction would be expected if all patients were maintained on a liquid diet or nil per os. Combined, instituting all these changes (no opiates or solid diet before colonoscopy and performing all colonoscopies before noon) would produce a 5.6% reduction in IBP rate.

In this large, multihospital cohort, IBP was documented in half (51%) of 8,819 inpatient colonoscopies performed. Nonmodifiable patient characteristics independently associated with IBP were age, male gender, white race, Medicare and Medicaid insurance, nausea/vomiting, dysphagia, and gastroparesis. Modifiable factors included not consuming opiates within three days of colonoscopy, avoidance of a solid diet the day prior to colonoscopy and performing the colonoscopy before noon. The volume of bowel preparation consumed was not associated with IBP. In a counterfactual analysis, we found that if all three modifiable factors were optimized, the predicted rate of IBP would drop to 45%.

Many studies, including our analysis, have shown significant differences between the frequency of IBP in inpatient versus outpatient bowel preparations.^8-11 Therefore, it is crucial to study IBP in these settings separately. Three single-institution studies, including a total of 898 patients, have identified risk factors for inpatient IBP. Individual studies ranged in size from 130 to 524 patients with rates of IBP ranging from 22%-57%.^1-3 They found IBP to be associated with increasing age, lower income, ASA Grade >3, diabetes, coronary artery disease (CAD), nausea or vomiting, BMI >25, and chronic constipation. Modifiable factors included opiates, afternoon procedures, and runway times >6 hours.

We also found IBP to be associated with increasing age and male gender. However, we found no association with diabetes, chronic constipation, CAD or BMI. As we were able to adjust for a wider variety of variables, it is possible that we were able to account for residual confounding better than previous studies. For example, we found that having nausea/vomiting, dysphagia, and gastroparesis was associated with IBP. Gastroparesis with associated nausea and vomiting may be the mechanism by which diabetes increases the risk for IBP. Further studies are needed to assess if interventions or alternative bowel cleansing in these patients can result in improved IBP. Finally, in contrast to studies with smaller cohorts which found that lower volume bowel preps improved IBP in the right colon,^4,12 we found no association between IBP based and volume of bowel preparation consumed. Our impact analysis suggests that avoidance of opiates for at least three days before colonoscopy, avoidance of solid diet on the day before colonoscopy and performing all colonoscopies before noon would reduce the rate of IBP by 5.6%. While at first glance this does not appear to be a significant change, from a public health perspective with thousands of inpatient colonoscopies performed every year, it is crucial. We found that IBP was associated with an increased inpatient LOS of approximately one day. Assuming an average cost of one hospital day to be $2,000,¹³ this 5.6% improvement among our almost 9,000 patients, would translate into eliminating 494 unnecessary hospital days, or approximately $1 million in savings. More importantly, this savings comes without risk to patients and would result in an improvement in quality.

The factors mentioned above may not always be amenable to modification. For example, for patients with active gastrointestinal bleeding, postponing colonoscopy by one day for the sake of maintaining a patient on a clear diet may not be feasible. Similarly, performing colonoscopies in the morning is highly dependent on endoscopy suite availability and hospital logistics. Denying opiates to patients experiencing severe pain is not ethical. In many scenarios, however, these variables could be modified, and institutional efforts to support these practices could yield considerable savings. Future prospective studies are needed to verify the real impact of these changes.

Further discussion is needed to contextualize the finding that colonoscopies scheduled in the afternoon are associated with improved bowel preparation quality. Previous research—albeit in the outpatient setting—has demonstrated 11.8 hours as the maximum upper time limit for the time elapsed between the end of bowel preparation to colonoscopy.¹⁴ Another study found an inverse relationship between the quality of bowel preparation and the time after completion of the bowel preparation.¹⁵ This makes sense from a physiological perspective as delaying the time between completion of bowel preparation, and the procedure allows chyme from the small intestine to reaccumulate in the colon. Anecdotally, at our institution as well as at many others, the bowel preparations are ordered to start in the evening to allow the consumption of complete bowel preparation by midnight. As a result of this practice, only patients who have their colonoscopies scheduled before noon fall within the optimal period of 11.8 hours. In the outpatient setting, the use of split preparations has led to the obliteration of the difference in the quality of bowel preparation between morning and afternoon colonoscopies.¹⁶ Prospective trials are needed to evaluate the use of split preparations to improve the quality of afternoon inpatient colonoscopies.

In this large retrospective study evaluating bowel preparation quality in inpatients undergoing colonoscopy, we found that more than half of the patients have IBP and that IBP was associated with an extra day of hospitalization. Our study identifies those patients at highest risk and identifies modifiable risk factors for IBP. Specifically, we found that abstinence from opiates or solid diet before the colonoscopy, along with performing colonoscopies before noon were associated with improved outcomes. Prospective studies are needed to confirm the effects of these interventions on bowel preparation quality.

Disclosures

Carol A Burke, MD has received research funding from Ferring Pharmaceuticals. Other authors have no conflicts of interest to disclose.

Inadequate bowel preparation (IBP) at the time of inpatient colonoscopy is common and associated with increased length of stay and cost of care.¹ The factors that contribute to IBP can be categorized into those that are modifiable and those that are nonmodifiable. While many factors have been associated with IBP, studies have been limited by small sample size or have combined inpatient/outpatient populations, thus limiting generalizability.^1-5 Moreover, most factors associated with IBP, such as socioeconomic status, male gender, increased age, and comorbidities, are nonmodifiable. No studies have explicitly focused on modifiable risk factors, such as medication use, colonoscopy timing, or assessed the potential impact of modifying these factors.

In a large, multihospital system, we examine the frequency of IBP among inpatients undergoing colonoscopy along with factors associated with IBP. We attempted to identify modifiable risk factors that were associated with IBP.

Potential Predictors of IBP

Demographic data such as patient age, gender, ethnicity, body mass index (BMI), and insurance/payor status were obtained from the electronic health record (EHR). International Classification of Disease 9^th and 10^th revision, Clinical Modifications (ICD-9/10-CM) codes were used to obtain patient comorbidities including diabetes, coronary artery disease, heart failure, cirrhosis, gastroparesis, hypothyroidism, inflammatory bowel disease, constipation, stroke, dementia, dysphagia, and nausea/vomiting. Use of opioid medications within three days before colonoscopy was extracted from the medication administration record. These variables were chosen as biologically plausible modifiers of bowel preparation or had previously been assessed in the literature.^1-6 The name and volume, classified as 4 L (GoLytely^®) and < 4 liters (MoviPrep^®) of bowel preparation, time of day when colonoscopy was performed, solid diet the day prior to colonoscopy, type of sedation used (conscious sedation or general anesthesia), and total colonoscopy time (defined as the time from scope insertion to removal) was recorded. Hospitalization-related variables, including the number of hospitalizations in the year before the current hospitalization, the year in which the colonoscopy was performed, and the number of days from admission to colonoscopy, were also obtained from the EHR.

Outcome Measures

An internally validated natural language algorithm, using Structured Queried Language was used to search through colonoscopy reports to identify adequacy of bowel preparation. ProVation^® software allows the gastroenterologist to use some terms to describe bowel preparation in a drop-down menu format. In addition to the Aronchik scale (which allows the gastroenterologist to rate bowel preparation on a five-point scale: “excellent,” “good,” “fair,” “poor,” and “inadequate”) it also allows the provider to use terms such as “adequate” or “adequate to detect polyps >5 mm” as well as “unsatisfactory.”⁷ Mirroring prior literature, bowel preparation quality was classified into “adequate” and “inadequate”; “good” and “excellent” on the Aronchik scale were categorized as adequate as was the term “adequate” in any form; “fair,” “poor,” or “inadequate” on the Aronchik scale were classified as inadequate as was the term “unsatisfactory.” We evaluated the hospital length of stay (LOS) as a secondary outcome measure.

Statistical Analysis

After describing the frequency of IBP, the quality of bowel preparation (adequate vs inadequate) was compared based on the predictors described above. Categorical variables were reported as frequencies with percentages and continuous variables were reported as medians with 25^th-75^th percentile values. The significance of the difference between the proportion or median values of those who had inadequate versus adequate bowel preparation was assessed. Two-sided chi-square analysis was used to assess the significance of differences between categorical variables and the Wilcoxon Rank-Sum test was used to assess the significance of differences between continuous variables.

Multivariate logistic regression analysis was performed to assess factors associated with hospital predictors and outcomes, after adjusting for all the aforementioned factors and clustering the effect based on the endoscopist. To evaluate the potential impact of modifiable factors on IBP, we performed counterfactual analysis, in which the observed distribution was compared to a hypothetical population in which all the modifiable risk factors were optimal.

Overall, 8,819 patients were included in our study population. They had a median age of 64 [53-76] years; 50.5% were female and 51% had an IBP. Patient characteristics and rates of IBP are presented in Table 1.

In unadjusted analyses, with regards to modifiable factors, opiate use within three days of colonoscopy was associated with a higher rate of IBP (55.4% vs 47.3%, P <.001), as was a lower volume (<4L) bowel preparation (55.3% vs 50.4%, P = .003). IBP was less frequent when colonoscopy was performed before noon vs afternoon (50.3% vs 57.4%, P < .001), and when patients were documented to receive a clear liquid diet or nil per os vs a solid diet the day prior to colonoscopy (50.3% vs 57.4%, P < .001). Overall bowel preparation quality improved over time (Figure 1). Median LOS was five [3-11] days. Patients who had IBP on their initial colonoscopy had a LOS one day longer than patients without IBP (six days vs five days, P < .001).

Multivariate Analysis

Table 2 shows the results of the multivariate analysis. The following modifiable factors were associated with IBP: opiate used within three days of the procedure (OR 1.31; 95% CI 1.8, 1.45), having the colonoscopy performed after12:00 pm (OR, 1.25; 95% CI, 1.10, 1.41), and consuming a solid diet the day prior to the colonoscopy (OR, 1.37; 95% CI, 1.18, 1.59). However, the volume of bowel preparation was not associated with IBP. The selected nonmodifiable factors that were found to be associated with IBP included age (increment of five years; OR, 1.04; 95% CI, 1.02, 1.05), male gender (OR, 1.33; 95% CI, 1.23, 1.44), Medicare insurance (OR, 1.17; 95% CI, 1.07, 1.28), Medicaid insurance (OR, 1.34; 95% CI, 1.07, 1.28), gastroparesis (OR, 1.62; 95% CI, 1.16, 2.27), nausea/vomiting (OR 1.21; 95% CI, 1.09, 1.34), and dysphagia (OR, 1.16; 95% CI, 1.01, 1.34).

Potential Impact of Modifiable Variables

We conducted a counterfactual analysis based on a multivariate model to assess the impact of each modifiable risk factor on the IBP rate (Figure 1). In the included study population, 44.9% received an opiate, 39.3% had a colonoscopy after 12:00 pm, and 9.1% received solid food the day prior to the procedure. Holding all other factors constant, if all patients were not prescribed opiates within three days of the procedure a 2.9% reduction in IBP would be expected. Similarly, if all patients underwent colonoscopy before noon, a 2.1% reduction in IBP rate would be expected. A 0.7% reduction would be expected if all patients were maintained on a liquid diet or nil per os. Combined, instituting all these changes (no opiates or solid diet before colonoscopy and performing all colonoscopies before noon) would produce a 5.6% reduction in IBP rate.

In this large, multihospital cohort, IBP was documented in half (51%) of 8,819 inpatient colonoscopies performed. Nonmodifiable patient characteristics independently associated with IBP were age, male gender, white race, Medicare and Medicaid insurance, nausea/vomiting, dysphagia, and gastroparesis. Modifiable factors included not consuming opiates within three days of colonoscopy, avoidance of a solid diet the day prior to colonoscopy and performing the colonoscopy before noon. The volume of bowel preparation consumed was not associated with IBP. In a counterfactual analysis, we found that if all three modifiable factors were optimized, the predicted rate of IBP would drop to 45%.

Many studies, including our analysis, have shown significant differences between the frequency of IBP in inpatient versus outpatient bowel preparations.^8-11 Therefore, it is crucial to study IBP in these settings separately. Three single-institution studies, including a total of 898 patients, have identified risk factors for inpatient IBP. Individual studies ranged in size from 130 to 524 patients with rates of IBP ranging from 22%-57%.^1-3 They found IBP to be associated with increasing age, lower income, ASA Grade >3, diabetes, coronary artery disease (CAD), nausea or vomiting, BMI >25, and chronic constipation. Modifiable factors included opiates, afternoon procedures, and runway times >6 hours.

We also found IBP to be associated with increasing age and male gender. However, we found no association with diabetes, chronic constipation, CAD or BMI. As we were able to adjust for a wider variety of variables, it is possible that we were able to account for residual confounding better than previous studies. For example, we found that having nausea/vomiting, dysphagia, and gastroparesis was associated with IBP. Gastroparesis with associated nausea and vomiting may be the mechanism by which diabetes increases the risk for IBP. Further studies are needed to assess if interventions or alternative bowel cleansing in these patients can result in improved IBP. Finally, in contrast to studies with smaller cohorts which found that lower volume bowel preps improved IBP in the right colon,^4,12 we found no association between IBP based and volume of bowel preparation consumed. Our impact analysis suggests that avoidance of opiates for at least three days before colonoscopy, avoidance of solid diet on the day before colonoscopy and performing all colonoscopies before noon would reduce the rate of IBP by 5.6%. While at first glance this does not appear to be a significant change, from a public health perspective with thousands of inpatient colonoscopies performed every year, it is crucial. We found that IBP was associated with an increased inpatient LOS of approximately one day. Assuming an average cost of one hospital day to be $2,000,¹³ this 5.6% improvement among our almost 9,000 patients, would translate into eliminating 494 unnecessary hospital days, or approximately $1 million in savings. More importantly, this savings comes without risk to patients and would result in an improvement in quality.

The factors mentioned above may not always be amenable to modification. For example, for patients with active gastrointestinal bleeding, postponing colonoscopy by one day for the sake of maintaining a patient on a clear diet may not be feasible. Similarly, performing colonoscopies in the morning is highly dependent on endoscopy suite availability and hospital logistics. Denying opiates to patients experiencing severe pain is not ethical. In many scenarios, however, these variables could be modified, and institutional efforts to support these practices could yield considerable savings. Future prospective studies are needed to verify the real impact of these changes.

Further discussion is needed to contextualize the finding that colonoscopies scheduled in the afternoon are associated with improved bowel preparation quality. Previous research—albeit in the outpatient setting—has demonstrated 11.8 hours as the maximum upper time limit for the time elapsed between the end of bowel preparation to colonoscopy.¹⁴ Another study found an inverse relationship between the quality of bowel preparation and the time after completion of the bowel preparation.¹⁵ This makes sense from a physiological perspective as delaying the time between completion of bowel preparation, and the procedure allows chyme from the small intestine to reaccumulate in the colon. Anecdotally, at our institution as well as at many others, the bowel preparations are ordered to start in the evening to allow the consumption of complete bowel preparation by midnight. As a result of this practice, only patients who have their colonoscopies scheduled before noon fall within the optimal period of 11.8 hours. In the outpatient setting, the use of split preparations has led to the obliteration of the difference in the quality of bowel preparation between morning and afternoon colonoscopies.¹⁶ Prospective trials are needed to evaluate the use of split preparations to improve the quality of afternoon inpatient colonoscopies.

In this large retrospective study evaluating bowel preparation quality in inpatients undergoing colonoscopy, we found that more than half of the patients have IBP and that IBP was associated with an extra day of hospitalization. Our study identifies those patients at highest risk and identifies modifiable risk factors for IBP. Specifically, we found that abstinence from opiates or solid diet before the colonoscopy, along with performing colonoscopies before noon were associated with improved outcomes. Prospective studies are needed to confirm the effects of these interventions on bowel preparation quality.

Disclosures

Carol A Burke, MD has received research funding from Ferring Pharmaceuticals. Other authors have no conflicts of interest to disclose.

Sepsis is both the most expensive condition treated and the most common cause of death in hospitals in the United States.^1-3 Most sepsis patients (as many as 80% to 90%) meet sepsis criteria on hospital arrival, but mortality and costs are higher when meeting criteria after admission.^3-6 Mechanisms of this increased mortality for these distinct populations are not well explored. Patients who present septic in the emergency department (ED) and patients who present as inpatients likely present very different challenges for recognition, treatment, and monitoring.⁷ Yet, how these groups differ by demographic and clinical characteristics, the etiology and severity of infection, and patterns of resuscitation care are not well described. Literature on sepsis epidemiology on hospital wards is particularly limited.⁸

This knowledge gap is important. If hospital-presenting sepsis (HPS) contributes disproportionately to disease burdCHFens, it reflects a high-yield population deserving the focus of quality improvement (QI) initiatives. If specific causes of disparities were identified—eg, poor initial resuscitation— they could be specifically targeted for correction. Given that current treatment guidelines are uniform for the two populations,^9,10 characterizing phenotypic differences could also have implications for both diagnostic and therapeutic recommendations, particularly if the groups display substantially differing clinical presentations. Our prior work has not probed these effects specifically, but suggested ED versus inpatient setting at the time of initial sepsis presentation might be an effect modifier for the association between several elements of fluid resuscitation and patient outcomes.^11,12

We, therefore, conducted a retrospective analysis to ask four sequential questions: (1) Do patients with HPS, compared with EDPS, contribute adverse outcome out of proportion to case prevalence? (2) At the time of initial presentation, how do HPS patients differ from EDPS patients with respect to demographics, comorbidities, infectious etiologies, clinical presentations, and severity of illness (3) If holding observed baseline factors constant, does the physical location of sepsis presentation inherently increase the risk for treatment delays and mortality? (4) To what extent can differences in the likelihood for timely initial treatment between the ED and inpatient settings explain differences in mortality and patient outcomes?

We hypothesized a priori that HPS would reflect chronically sicker patients whom both received less timely resuscitation and who contributed disproportionately frequent bad outcomes. We expected disparities in timely resuscitation care would explain a large proportion of this difference.

We performed a retrospective analysis of the Northwell Sepsis Database, a prospectively captured, multisite, real world, consecutive-sample cohort of all “severe sepsis” and septic shock patients treated at nine tertiary and community hospitals in New York from October 1, 2014, to March 31, 2016. We analyzed all patients from a previously published cohort.¹¹

Database Design and Structure

The Northwell Sepsis Database has previously been described in detail.^11,13,14 Briefly, all patients met clinical sepsis criteria: (1) infection AND (2) ≥2 (SIRS) criteria AND (3) ≥1 acute organ dysfunction criterion. Organ dysfunction criteria were hypotension, acute kidney injury (AKI), coagulopathy, altered gas exchange, elevated bilirubin (≥2.0 mg/dL), or altered mental status (AMS; clarified in Supplemental Table 1). All organ dysfunction was not otherwise explained by patients’ medical histories; eg, patients on warfarin anticoagulation were not documented to have coagulopathy based on international normalized ratio > 1.5. The time of the sepsis episode (and database inclusion) was the time of the first vital sign measurement or laboratory result where a patient simultaneously met all three inclusion criteria: infection, SIRS, and organ dysfunction. The database excludes patients who were <18 years, declined bundle interventions, had advance directives precluding interventions, or were admitted directly to palliative care or hospice. Abstractors assumed comorbidities were absent if not documented within the medical record and that physiologic abnormalities were absent if not measured by the treatment team. There were no missing data for the variables analyzed. We report analysis in adherence with the STROBE statement guidelines for observational research.

Exposure

The primary exposure was whether patients had EDPS versus HPS. We defined EDPS patients as meeting all objective clinical inclusion criteria while physically in the ED. We defined HPS as first meeting sepsis inclusion criteria outside the ED, regardless of the reason for admission, and regardless of whether patients were admitted through the ED or directly to the hospital. All ED patients were admitted to the hospital.

Outcomes

Process outcomes were full 3-hour bundle compliance, time to antibiotic administration, blood cultures before antibiotics, time to fluid initiation, the volume of administered fluid resuscitation, lactate result time, and whether repeat lactate was obtained (Supplemental Table 2). Treatment times were times of administration (rather than order time). The primary patient outcome was hospital mortality. Secondary patient outcomes were mechanical ventilation, ICU admission, ICU days, hospital length of stay (LOS). We discounted HPS patients’ LOS to include only days after meeting the inclusion criteria. Patients were excluded from the analysis of the ICU admission outcome if they were already in the ICU prior to meeting sepsis criteria.

Statistical Analysis

We report continuous variables as means (standard deviation) or medians (interquartile range), and categorical variables as frequencies (proportions), as appropriate. Summative statistics with 95% confidence intervals (CI) describe overall group contributions. We used generalized linear models to determine patient factors associated with EDPS versus HPS, entering random effects for individual study sites to control for intercenter variability.

Next, to generate a propensity-matched cohort, we computed propensity scores adjusted from a priori selected variables: age, sex, tertiary versus community hospital, congestive heart failure (CHF), renal failure, COPD, diabetes, liver failure, immunocompromise, primary source of infection, nosocomial source, temperature, initial lactate, presenting hypotension, altered gas exchange, AMS, AKI, and coagulopathy. We then matched subjects 1:1 without optimization or replacement, imposing a caliper width of 0.01; ie, we required matched pairs to have a <1.0% difference in propensity scores. The macro used to match subjects is publically available.¹⁵

We then compared resuscitation and patient outcomes in the matched cohort using generalized linear models, ie, doubly-robust estimation (DRE).¹⁶ When assessing patient outcomes corrected for resuscitation, we used mixed DRE/multivariable regression. We did this for two reasons: first, DRE has the advantage of only requiring only one approach (propensity vs covariate adjustments) to be correctly specified.¹⁶ Second, computing propensity scores adjusted for resuscitation would be inappropriate given that resuscitation occurs after the exposure allocation (HPS vs EDPS). However, these factors could still impact the outcome and in fact, we hypothesized they were potential mediators of the exposure effect. To interrogate this mediating relationship, we recapitulated the DRE modeling but added covariates for resuscitation factors. Resuscitation-adjusted models controlled for timeliness of antibiotics, fluids, and lactate results; blood cultures before antibiotics; repeat lactate obtained, and fluid volume in the first six hours. Since ICU days and LOS are subject to competing risks bias (LOS could be shorter if patients died earlier), we used proportional hazards models where “the event” was defined as a live discharge to censor for mortality and we report output as inverse hazard ratios. We also tested interaction coefficients for discrete bundle elements and HPS to determine if specific bundle elements were effect modifiers for the association between the presenting location and mortality risk. Finally, we estimated attributable risk differences by comparing adjusted odds ratios of adverse outcome with and without adjustment for resuscitation variables, as described by Sahai et al.¹⁷

As sensitivity analyses, we recomputed propensity scores and generated a new matched cohort that excluded HPS patients who met criteria for sepsis while already in the ICU for another reason (ie, excluding ICU-presenting sepsis). We then recapitulated all analyses as above for this cohort. We performed analyses using SAS version 9.4 (SAS Institute, Cary, North Carolina).

Prevalence and Outcome Contributions

Of the 11,182 sepsis patients in the database, we classified 2,509 (22%) as HPS (Figure 1). HPS contributed 785 (35%) of 2,241 sepsis-related mortalities, 1,241 (38%) mechanical ventilations, and 1,762 (34%) ICU admissions. Of 39,263 total ICU days and 127,178 hospital days, HPS contributed 18,104 (46.1%) and 44,412 (34.9%) days, respectively.

Patient Characteristics

Most HPS presented early in the hospital course, with 1,352 (53.9%) cases meeting study criteria within three days of admission. Median time from admission to meeting study criteria for HPS was two days (interquartile range: one to seven days). We report selected baseline patient characteristics in Table 1 and adjusted associations of baseline variables with HPS versus EDPS in Table 2. The full cohort characterization is available in Supplemental Table 3. Notably, HPS patients more often had CHF (aOR [adjusted odds ratio}: 1.31, CI: 1.18-1.47) or renal failure (aOR: 1.62, CI: 1.38-1.91), gastrointestinal source of infection (aOR: 1.84, CI: 1.48-2.29), hypothermia (aOR: 1.56, CI: 1.28-1.90) hypotension (aOR: 1.85, CI: 1.65-2.08), or altered gas exchange (aOR: 2.46, CI: 1.43-4.24). In contrast, HPS patients less frequently were admitted from skilled nursing facilities (aOR: 0.44, CI: 0.32-0.60), or had COPD (aOR: 0.53, CI: 0.36-0.76), fever (aOR: 0.70, CI: 0.52-0.91), tachypnea (aOR: 0.76, CI: 0.58-0.98), or AKI (aOR: 082, CI: 0.68-0.97). Other baseline variables were similar, including respiratory source, tachycardia, white cell abnormalities, AMS, and coagulopathies. These associations were preserved in the sensitivity analysis excluding ICU-presenting sepsis.

Propensity Matching

Propensity score matching yielded 1,942 matched pairs (n = 3,884, 77% of HPS patients, 22% of EDPS patients). Table 1 and Supplemental Table 3 show patient characteristics after propensity matching. Supplemental Table 4 shows the propensity model. The frequency densities are shown for the cohort as a function of propensity score in Supplemental Figure 1. After matching, frequencies between groups differed by <5% for all categorical variables assessed. In the sensitivity analysis, propensity matching (model in Supplemental Table 5) resulted in 1,233 matched pairs (n = 2,466, 49% of HPS patients, 14% of EDPS patients), with group differences comparable to the primary analysis.

Process Outcomes

We present propensity-matched differences in initial resuscitation in Figure 2A for all HPS patients, as well as non-ICU-presenting HPS, versus EDPS. HPS patients were roughly half as likely to receive fully 3-hour bundle compliant care (17.0% vs 30.3%, aOR: 0.47, CI: 0.40-0.57), to have blood cultures drawn within three hours prior to antibiotics (44.9% vs 67.2%, aOR: 0.40, CI: 0.35-0.46), or to receive fluid resuscitation initiated within two hours (11.1% vs 26.1%, aOR: 0.35, CI: 0.29-0.42). Antibiotic receipt within one hour was comparable (45.3% vs 48.1%, aOR: 0.89, CI: 0.79-1.01). However, differences emerged for antibiotics within three hours (66.2% vs 83.8%, aOR: 0.38, CI: 0.32-0.44) and persisted at six hours (77.0% vs 92.5%, aOR: 0.27, CI: 0.22-33). Excluding ICU-presenting sepsis from propensity matching exaggerated disparities in antibiotic receipt at one hour (43.4% vs 49.1%, aOR: 0.80, CI: 0.68-0.93), three hours (64.2% vs 86.1%, aOR: 0.29, CI: 0.24-0.35), and six hours (75.7% vs 93.0%, aOR: 0.23, CI: 0.18-0.30). HPS patients more frequently had repeat lactate obtained within 24 hours (62.4% vs 54.3%, aOR: 1.40, CI: 1.23-1.59).

Patient Outcomes

HPS patients had higher mortality (31.2% vs19.3%), mechanical ventilation (51.5% vs27.4%), and ICU admission (60.6% vs 46.5%) (Table 1 and Supplemental Table 6). Figure 2b shows propensity-matched and covariate-adjusted differences in patient outcomes before and after adjusting for initial resuscitation. aORs corresponded to approximate relative risk differences¹⁸ of 1.38 (CI: 1.28-1.48), 1.68 (CI: 1.57-1.79), and 1.72 (CI: 1.61-1.84) for mortality, mechanical ventilation, and ICU admission, respectively. HPS was associated with 83% longer mortality-censored ICU stays (five vs nine days, HR^–1: 1.83, CI: 1.65-2.03), and 108% longer hospital stay (eight vs 17 days, HR^–1: 2.08, CI: 1.93-2.24). After adjustment for resuscitation, all effect sizes decreased but persisted. The initial crystalloid volume was a significant negative effect modifier for mortality (Supplemental Table 7). That is, the magnitude of the association between HPS and greater mortality decreased by a factor of 0.89 per 10 mL/kg given (CI: 0.82-0.97). We did not observe significant interaction from other interventions, or overall bundle compliance, meaning these interventions’ association with mortality did not significantly differ between HPS versus EDPS.

The implied attributable risk difference from discrepancies in initial resuscitation was 23.3% for mortality, 35.2% for mechanical ventilation, and 7.6% for ICU admission (Figure 2B). Resuscitation explained 26.5% of longer ICU LOS and 16.7% of longer hospital LOS associated with HPS.

Figure 2C shows sensitivity analysis excluding ICU-presenting sepsis from propensity matching (ie, limiting HPS to hospital ward presentations). Again, HPS was associated with all adverse outcomes, though effect sizes were smaller than in the primary cohort for all outcomes except hospital LOS. In this cohort, resuscitation factors now explained 16.5% of HPS’ association with mortality, and 14.5% of the association with longer ICU LOS. However, they explained a greater proportion (13.0%) of ICU admissions. Attributable risk differences were comparable to the primary cohort for mechanical ventilation (37.6%) and hospital LOS (15.3%).

In this analysis of 11,182 sepsis and septic shock patients, HPS contributed 22% of prevalence but >35% of total sepsis mortalities, ICU utilization, and hospital days. HPS patients had higher comorbidity burdens and had clinical presentations less obviously attributable to infection with more severe organ dysfunction. EDPS received antibiotics within three hours about 1.62 times more often than do HPS patients. EDPS patients also receive fluids initiated within two hours about 1.82 times more often than HPS patients do. HPS had nearly 1.5-fold greater mortality and LOS, and nearly two-fold greater mechanical ventilation and ICU utilization. Resuscitation disparities could partially explain these associations. These patterns persisted when comparing only wards presenting HPS with EDPS.

Our analysis revealed several notable findings. First, these data confirm that HPS represents a potentially high-impact target population that contributes adverse outcomes disproportionately frequently with respect to case prevalence.

Our findings, unsurprisingly, revealed HPS and EDPS reflect dramatically different patient populations. We found that the two groups significantly differed by the majority of the baseline factors we compared. It may be worth asking if and how these substantial differences in illness etiology, chronic health, and acute physiology impact what we consider an optimal approach to management. Significant interaction effects of fluid volume on the association between HPS and mortality suggest differential treatment effects may exist between patients. Indeed, patients who newly arrive from the community and those who are several days into admission likely have different volume status. However, no interactions were noted with other bundle elements, such as timeliness of antibiotics or timeliness of initial fluids.

Another potentially concerning observation was that HPS patients were admitted much less frequently from skilled nursing facilities, as it could imply that this poorer-fairing population had a comparatively higher baseline functional status. The fact that 25% of EDPS cases were admitted from these facilities also underscores the need to engage skilled nursing facility providers in future sepsis initiatives.

We found marked disparities in resuscitation. Timely delivery of interventions, such as antibiotics and initial fluid resuscitation, occurred less than half as often for HPS, especially on hospital wards. While evidence supporting the efficacy of specific 3-hour bundle elements remains unsettled,¹⁹ a wealth of literature demonstrates a correlation between bundle uptake and decreased sepsis mortality, especially for early antibiotic administration.^13,20-26 Some analysis suggests that differing initial resuscitation practices explain different mortality rates in the early goal-directed therapy trials.²⁷ The comparatively poor performance for non-ICU HPS indicates further QI efforts are better focused on inpatient wards, rather than on EDs or ICUs where resuscitation is already delivered with substantially greater fidelity.

While resuscitation differences partially explained outcome discrepancies between groups, they did not account for as much variation as expected. Though resuscitation accounted for >35% of attributable mechanical ventilation risk, it explained only 16.5% of mortality differences for non-ICU HPS vs EDPS. We speculate that several factors may contribute.

First, HPS patients are already hospitalized for another acute insult and may be too physiologically brittle to derive equal benefit from initial resuscitation. Some literature suggests protocolized sepsis resuscitation may paradoxically be more effective in milder/earlier disease.²⁸

Second, clinical information indicating septic organ dysfunction may become available too late in HPS—a possible data limitation where inpatient providers are counterintuitively more likely to miss early signs of patients’ deterioration and a subsequent therapeutic window. Several studies found that fluid resuscitation is associated with improved sepsis outcomes only when it is administered very early.^11,29-31 In inpatient wards, decreased monitoring³² and human factors (eg, hospital workflow, provider-to-patient ratios, electronic documentation burdens)^33,34 may hinder early diagnosis. In contrast, ED environments are explicitly designed to identify acutely ill patients and deliver intervention rapidly. If HPS patients were sicker when they were identified, this would also explain their more severe organ dysfunctions. Our data seems to support this possibility. HPS patients had tachypnea less frequently but more often had impaired gas exchange. This finding may suggest that early tachypnea was either less often detected or documented, or that it had progressed further by the time of detection.

Third, inpatients with sepsis may more often present with greater diagnostic complexity. We observed that HPS patients were more often euthermic and less often tachypneic. Beyond suggesting a greater diagnostic challenge, this also raises questions as to whether differences reflect patient physiology (response to infection) or iatrogenic factors (eg, prior antipyretics). Higher comorbidity and acute physiological burdens also limit the degree to which new organ dysfunction can be clearly attributed to infection. We note differences in the proportion of patients who received antibiotics increased over time, suggesting that HPS patients who received delayed antibiotics did so much later than their EDPS counterparts. This lag could also arise from diagnostic difficulty.

All three possibilities highlight a potential lead time effect, where the same measured three-hour period on the wards, between meeting sepsis criteria and starting treatment, actually reflects a longer period between (as yet unmeasurable) pathobiologic “time zero” and treatment versus the ED. The time of sepsis detection, as distinct from the time of sepsis onset, therefore proves difficult to evaluate and impossible to account for statistically.

Regardless, our findings suggest additional difficulty in both the recognition and resuscitation of inpatient sepsis. Inpatients, especially with infections, may need closer monitoring. How to cost effectively implement this monitoring is a challenge that deserves attention.

A more rational systems approach to HPS likely combines efforts to improve initial resuscitation with other initiatives aimed at both improving monitoring and preventing infection.

To be clear, we do not imply that timely initial resuscitation does not matter on the wards. Rather, resuscitation-focused QI alone does not appear to be sufficient to overcome differences in outcomes for HPS. The 23.3% attributable mortality risk we observed still implies that resuscitation differences could explain nearly one in four excess HPS mortalities. We previously showed that timely resuscitation is strongly associated with better outcomes.^11,13,30 As discussed above, the unclear degree to which better resuscitation is a marker for more obvious presentations is a persistent limitation of prior investigations and the present study.

Taken together, the ultimate question that this study raises but cannot answer is whether the timely recognition of sepsis, rather than any specific treatment, is what truly improves outcomes.

In addition to those above, this study has several limitations. Our study did not differentiate HPS with respect to patients admitted for noninfectious reasons and who subsequently became septic versus nonseptic patients admitted for an infection who subsequently became septic from that infection. Nor could we discriminate between missed ED diagnoses and true delayed presentations. We note distinguishing these entities clinically can be equally challenging. Additionally, this was a propensity-matched retrospective analysis of an existing sepsis cohort, and the many limitations of both retrospective study and propensity matching apply.^35,36 We note that randomizing patients to develop sepsis in the community versus hospital is not feasible and that two of our aims intended to describe overall patterns rather than causal effects. We could not ascertain robust measures of severity of illness (eg, SOFA) because a real world setting precludes required data points—eg, urine output is unreliably recorded. We also note incomplete overlap between inclusion criteria and either Sepsis-2 or -3 definitions,^1,37 because we designed and populated our database prior to publication of Sepsis-3. Further, we could not account for surgical source control, the appropriateness of antimicrobial therapy, mechanical ventilation before sepsis onset, or most treatments given after initial resuscitation.

In conclusion, hospital-presenting sepsis accounted for adverse patient outcomes disproportionately to prevalence. HPS patients had more complex presentations, received timely antibiotics half as often ED-presenting sepsis, and had nearly twice the mortality odds. Resuscitation disparities explained roughly 25% of this difference.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This investigation was funded in part by a grant from the Center for Medicare and Medicaid Innovation to the High Value Healthcare Collaborative, of which the study sites’ umbrella health system was a part. This grant helped fund the underlying QI program and database in this study.

Sepsis is both the most expensive condition treated and the most common cause of death in hospitals in the United States.^1-3 Most sepsis patients (as many as 80% to 90%) meet sepsis criteria on hospital arrival, but mortality and costs are higher when meeting criteria after admission.^3-6 Mechanisms of this increased mortality for these distinct populations are not well explored. Patients who present septic in the emergency department (ED) and patients who present as inpatients likely present very different challenges for recognition, treatment, and monitoring.⁷ Yet, how these groups differ by demographic and clinical characteristics, the etiology and severity of infection, and patterns of resuscitation care are not well described. Literature on sepsis epidemiology on hospital wards is particularly limited.⁸

This knowledge gap is important. If hospital-presenting sepsis (HPS) contributes disproportionately to disease burdCHFens, it reflects a high-yield population deserving the focus of quality improvement (QI) initiatives. If specific causes of disparities were identified—eg, poor initial resuscitation— they could be specifically targeted for correction. Given that current treatment guidelines are uniform for the two populations,^9,10 characterizing phenotypic differences could also have implications for both diagnostic and therapeutic recommendations, particularly if the groups display substantially differing clinical presentations. Our prior work has not probed these effects specifically, but suggested ED versus inpatient setting at the time of initial sepsis presentation might be an effect modifier for the association between several elements of fluid resuscitation and patient outcomes.^11,12

We, therefore, conducted a retrospective analysis to ask four sequential questions: (1) Do patients with HPS, compared with EDPS, contribute adverse outcome out of proportion to case prevalence? (2) At the time of initial presentation, how do HPS patients differ from EDPS patients with respect to demographics, comorbidities, infectious etiologies, clinical presentations, and severity of illness (3) If holding observed baseline factors constant, does the physical location of sepsis presentation inherently increase the risk for treatment delays and mortality? (4) To what extent can differences in the likelihood for timely initial treatment between the ED and inpatient settings explain differences in mortality and patient outcomes?

We hypothesized a priori that HPS would reflect chronically sicker patients whom both received less timely resuscitation and who contributed disproportionately frequent bad outcomes. We expected disparities in timely resuscitation care would explain a large proportion of this difference.

We performed a retrospective analysis of the Northwell Sepsis Database, a prospectively captured, multisite, real world, consecutive-sample cohort of all “severe sepsis” and septic shock patients treated at nine tertiary and community hospitals in New York from October 1, 2014, to March 31, 2016. We analyzed all patients from a previously published cohort.¹¹

Database Design and Structure

The Northwell Sepsis Database has previously been described in detail.^11,13,14 Briefly, all patients met clinical sepsis criteria: (1) infection AND (2) ≥2 (SIRS) criteria AND (3) ≥1 acute organ dysfunction criterion. Organ dysfunction criteria were hypotension, acute kidney injury (AKI), coagulopathy, altered gas exchange, elevated bilirubin (≥2.0 mg/dL), or altered mental status (AMS; clarified in Supplemental Table 1). All organ dysfunction was not otherwise explained by patients’ medical histories; eg, patients on warfarin anticoagulation were not documented to have coagulopathy based on international normalized ratio > 1.5. The time of the sepsis episode (and database inclusion) was the time of the first vital sign measurement or laboratory result where a patient simultaneously met all three inclusion criteria: infection, SIRS, and organ dysfunction. The database excludes patients who were <18 years, declined bundle interventions, had advance directives precluding interventions, or were admitted directly to palliative care or hospice. Abstractors assumed comorbidities were absent if not documented within the medical record and that physiologic abnormalities were absent if not measured by the treatment team. There were no missing data for the variables analyzed. We report analysis in adherence with the STROBE statement guidelines for observational research.

Exposure

The primary exposure was whether patients had EDPS versus HPS. We defined EDPS patients as meeting all objective clinical inclusion criteria while physically in the ED. We defined HPS as first meeting sepsis inclusion criteria outside the ED, regardless of the reason for admission, and regardless of whether patients were admitted through the ED or directly to the hospital. All ED patients were admitted to the hospital.

Outcomes

Process outcomes were full 3-hour bundle compliance, time to antibiotic administration, blood cultures before antibiotics, time to fluid initiation, the volume of administered fluid resuscitation, lactate result time, and whether repeat lactate was obtained (Supplemental Table 2). Treatment times were times of administration (rather than order time). The primary patient outcome was hospital mortality. Secondary patient outcomes were mechanical ventilation, ICU admission, ICU days, hospital length of stay (LOS). We discounted HPS patients’ LOS to include only days after meeting the inclusion criteria. Patients were excluded from the analysis of the ICU admission outcome if they were already in the ICU prior to meeting sepsis criteria.

Statistical Analysis

We report continuous variables as means (standard deviation) or medians (interquartile range), and categorical variables as frequencies (proportions), as appropriate. Summative statistics with 95% confidence intervals (CI) describe overall group contributions. We used generalized linear models to determine patient factors associated with EDPS versus HPS, entering random effects for individual study sites to control for intercenter variability.

Next, to generate a propensity-matched cohort, we computed propensity scores adjusted from a priori selected variables: age, sex, tertiary versus community hospital, congestive heart failure (CHF), renal failure, COPD, diabetes, liver failure, immunocompromise, primary source of infection, nosocomial source, temperature, initial lactate, presenting hypotension, altered gas exchange, AMS, AKI, and coagulopathy. We then matched subjects 1:1 without optimization or replacement, imposing a caliper width of 0.01; ie, we required matched pairs to have a <1.0% difference in propensity scores. The macro used to match subjects is publically available.¹⁵

We then compared resuscitation and patient outcomes in the matched cohort using generalized linear models, ie, doubly-robust estimation (DRE).¹⁶ When assessing patient outcomes corrected for resuscitation, we used mixed DRE/multivariable regression. We did this for two reasons: first, DRE has the advantage of only requiring only one approach (propensity vs covariate adjustments) to be correctly specified.¹⁶ Second, computing propensity scores adjusted for resuscitation would be inappropriate given that resuscitation occurs after the exposure allocation (HPS vs EDPS). However, these factors could still impact the outcome and in fact, we hypothesized they were potential mediators of the exposure effect. To interrogate this mediating relationship, we recapitulated the DRE modeling but added covariates for resuscitation factors. Resuscitation-adjusted models controlled for timeliness of antibiotics, fluids, and lactate results; blood cultures before antibiotics; repeat lactate obtained, and fluid volume in the first six hours. Since ICU days and LOS are subject to competing risks bias (LOS could be shorter if patients died earlier), we used proportional hazards models where “the event” was defined as a live discharge to censor for mortality and we report output as inverse hazard ratios. We also tested interaction coefficients for discrete bundle elements and HPS to determine if specific bundle elements were effect modifiers for the association between the presenting location and mortality risk. Finally, we estimated attributable risk differences by comparing adjusted odds ratios of adverse outcome with and without adjustment for resuscitation variables, as described by Sahai et al.¹⁷

As sensitivity analyses, we recomputed propensity scores and generated a new matched cohort that excluded HPS patients who met criteria for sepsis while already in the ICU for another reason (ie, excluding ICU-presenting sepsis). We then recapitulated all analyses as above for this cohort. We performed analyses using SAS version 9.4 (SAS Institute, Cary, North Carolina).

Prevalence and Outcome Contributions

Of the 11,182 sepsis patients in the database, we classified 2,509 (22%) as HPS (Figure 1). HPS contributed 785 (35%) of 2,241 sepsis-related mortalities, 1,241 (38%) mechanical ventilations, and 1,762 (34%) ICU admissions. Of 39,263 total ICU days and 127,178 hospital days, HPS contributed 18,104 (46.1%) and 44,412 (34.9%) days, respectively.

Patient Characteristics

Most HPS presented early in the hospital course, with 1,352 (53.9%) cases meeting study criteria within three days of admission. Median time from admission to meeting study criteria for HPS was two days (interquartile range: one to seven days). We report selected baseline patient characteristics in Table 1 and adjusted associations of baseline variables with HPS versus EDPS in Table 2. The full cohort characterization is available in Supplemental Table 3. Notably, HPS patients more often had CHF (aOR [adjusted odds ratio}: 1.31, CI: 1.18-1.47) or renal failure (aOR: 1.62, CI: 1.38-1.91), gastrointestinal source of infection (aOR: 1.84, CI: 1.48-2.29), hypothermia (aOR: 1.56, CI: 1.28-1.90) hypotension (aOR: 1.85, CI: 1.65-2.08), or altered gas exchange (aOR: 2.46, CI: 1.43-4.24). In contrast, HPS patients less frequently were admitted from skilled nursing facilities (aOR: 0.44, CI: 0.32-0.60), or had COPD (aOR: 0.53, CI: 0.36-0.76), fever (aOR: 0.70, CI: 0.52-0.91), tachypnea (aOR: 0.76, CI: 0.58-0.98), or AKI (aOR: 082, CI: 0.68-0.97). Other baseline variables were similar, including respiratory source, tachycardia, white cell abnormalities, AMS, and coagulopathies. These associations were preserved in the sensitivity analysis excluding ICU-presenting sepsis.

Propensity Matching

Propensity score matching yielded 1,942 matched pairs (n = 3,884, 77% of HPS patients, 22% of EDPS patients). Table 1 and Supplemental Table 3 show patient characteristics after propensity matching. Supplemental Table 4 shows the propensity model. The frequency densities are shown for the cohort as a function of propensity score in Supplemental Figure 1. After matching, frequencies between groups differed by <5% for all categorical variables assessed. In the sensitivity analysis, propensity matching (model in Supplemental Table 5) resulted in 1,233 matched pairs (n = 2,466, 49% of HPS patients, 14% of EDPS patients), with group differences comparable to the primary analysis.

Process Outcomes

We present propensity-matched differences in initial resuscitation in Figure 2A for all HPS patients, as well as non-ICU-presenting HPS, versus EDPS. HPS patients were roughly half as likely to receive fully 3-hour bundle compliant care (17.0% vs 30.3%, aOR: 0.47, CI: 0.40-0.57), to have blood cultures drawn within three hours prior to antibiotics (44.9% vs 67.2%, aOR: 0.40, CI: 0.35-0.46), or to receive fluid resuscitation initiated within two hours (11.1% vs 26.1%, aOR: 0.35, CI: 0.29-0.42). Antibiotic receipt within one hour was comparable (45.3% vs 48.1%, aOR: 0.89, CI: 0.79-1.01). However, differences emerged for antibiotics within three hours (66.2% vs 83.8%, aOR: 0.38, CI: 0.32-0.44) and persisted at six hours (77.0% vs 92.5%, aOR: 0.27, CI: 0.22-33). Excluding ICU-presenting sepsis from propensity matching exaggerated disparities in antibiotic receipt at one hour (43.4% vs 49.1%, aOR: 0.80, CI: 0.68-0.93), three hours (64.2% vs 86.1%, aOR: 0.29, CI: 0.24-0.35), and six hours (75.7% vs 93.0%, aOR: 0.23, CI: 0.18-0.30). HPS patients more frequently had repeat lactate obtained within 24 hours (62.4% vs 54.3%, aOR: 1.40, CI: 1.23-1.59).

Patient Outcomes

HPS patients had higher mortality (31.2% vs19.3%), mechanical ventilation (51.5% vs27.4%), and ICU admission (60.6% vs 46.5%) (Table 1 and Supplemental Table 6). Figure 2b shows propensity-matched and covariate-adjusted differences in patient outcomes before and after adjusting for initial resuscitation. aORs corresponded to approximate relative risk differences¹⁸ of 1.38 (CI: 1.28-1.48), 1.68 (CI: 1.57-1.79), and 1.72 (CI: 1.61-1.84) for mortality, mechanical ventilation, and ICU admission, respectively. HPS was associated with 83% longer mortality-censored ICU stays (five vs nine days, HR^–1: 1.83, CI: 1.65-2.03), and 108% longer hospital stay (eight vs 17 days, HR^–1: 2.08, CI: 1.93-2.24). After adjustment for resuscitation, all effect sizes decreased but persisted. The initial crystalloid volume was a significant negative effect modifier for mortality (Supplemental Table 7). That is, the magnitude of the association between HPS and greater mortality decreased by a factor of 0.89 per 10 mL/kg given (CI: 0.82-0.97). We did not observe significant interaction from other interventions, or overall bundle compliance, meaning these interventions’ association with mortality did not significantly differ between HPS versus EDPS.

The implied attributable risk difference from discrepancies in initial resuscitation was 23.3% for mortality, 35.2% for mechanical ventilation, and 7.6% for ICU admission (Figure 2B). Resuscitation explained 26.5% of longer ICU LOS and 16.7% of longer hospital LOS associated with HPS.

Figure 2C shows sensitivity analysis excluding ICU-presenting sepsis from propensity matching (ie, limiting HPS to hospital ward presentations). Again, HPS was associated with all adverse outcomes, though effect sizes were smaller than in the primary cohort for all outcomes except hospital LOS. In this cohort, resuscitation factors now explained 16.5% of HPS’ association with mortality, and 14.5% of the association with longer ICU LOS. However, they explained a greater proportion (13.0%) of ICU admissions. Attributable risk differences were comparable to the primary cohort for mechanical ventilation (37.6%) and hospital LOS (15.3%).

In this analysis of 11,182 sepsis and septic shock patients, HPS contributed 22% of prevalence but >35% of total sepsis mortalities, ICU utilization, and hospital days. HPS patients had higher comorbidity burdens and had clinical presentations less obviously attributable to infection with more severe organ dysfunction. EDPS received antibiotics within three hours about 1.62 times more often than do HPS patients. EDPS patients also receive fluids initiated within two hours about 1.82 times more often than HPS patients do. HPS had nearly 1.5-fold greater mortality and LOS, and nearly two-fold greater mechanical ventilation and ICU utilization. Resuscitation disparities could partially explain these associations. These patterns persisted when comparing only wards presenting HPS with EDPS.

Our analysis revealed several notable findings. First, these data confirm that HPS represents a potentially high-impact target population that contributes adverse outcomes disproportionately frequently with respect to case prevalence.

Our findings, unsurprisingly, revealed HPS and EDPS reflect dramatically different patient populations. We found that the two groups significantly differed by the majority of the baseline factors we compared. It may be worth asking if and how these substantial differences in illness etiology, chronic health, and acute physiology impact what we consider an optimal approach to management. Significant interaction effects of fluid volume on the association between HPS and mortality suggest differential treatment effects may exist between patients. Indeed, patients who newly arrive from the community and those who are several days into admission likely have different volume status. However, no interactions were noted with other bundle elements, such as timeliness of antibiotics or timeliness of initial fluids.

Another potentially concerning observation was that HPS patients were admitted much less frequently from skilled nursing facilities, as it could imply that this poorer-fairing population had a comparatively higher baseline functional status. The fact that 25% of EDPS cases were admitted from these facilities also underscores the need to engage skilled nursing facility providers in future sepsis initiatives.

We found marked disparities in resuscitation. Timely delivery of interventions, such as antibiotics and initial fluid resuscitation, occurred less than half as often for HPS, especially on hospital wards. While evidence supporting the efficacy of specific 3-hour bundle elements remains unsettled,¹⁹ a wealth of literature demonstrates a correlation between bundle uptake and decreased sepsis mortality, especially for early antibiotic administration.^13,20-26 Some analysis suggests that differing initial resuscitation practices explain different mortality rates in the early goal-directed therapy trials.²⁷ The comparatively poor performance for non-ICU HPS indicates further QI efforts are better focused on inpatient wards, rather than on EDs or ICUs where resuscitation is already delivered with substantially greater fidelity.

While resuscitation differences partially explained outcome discrepancies between groups, they did not account for as much variation as expected. Though resuscitation accounted for >35% of attributable mechanical ventilation risk, it explained only 16.5% of mortality differences for non-ICU HPS vs EDPS. We speculate that several factors may contribute.

First, HPS patients are already hospitalized for another acute insult and may be too physiologically brittle to derive equal benefit from initial resuscitation. Some literature suggests protocolized sepsis resuscitation may paradoxically be more effective in milder/earlier disease.²⁸

Second, clinical information indicating septic organ dysfunction may become available too late in HPS—a possible data limitation where inpatient providers are counterintuitively more likely to miss early signs of patients’ deterioration and a subsequent therapeutic window. Several studies found that fluid resuscitation is associated with improved sepsis outcomes only when it is administered very early.^11,29-31 In inpatient wards, decreased monitoring³² and human factors (eg, hospital workflow, provider-to-patient ratios, electronic documentation burdens)^33,34 may hinder early diagnosis. In contrast, ED environments are explicitly designed to identify acutely ill patients and deliver intervention rapidly. If HPS patients were sicker when they were identified, this would also explain their more severe organ dysfunctions. Our data seems to support this possibility. HPS patients had tachypnea less frequently but more often had impaired gas exchange. This finding may suggest that early tachypnea was either less often detected or documented, or that it had progressed further by the time of detection.

Third, inpatients with sepsis may more often present with greater diagnostic complexity. We observed that HPS patients were more often euthermic and less often tachypneic. Beyond suggesting a greater diagnostic challenge, this also raises questions as to whether differences reflect patient physiology (response to infection) or iatrogenic factors (eg, prior antipyretics). Higher comorbidity and acute physiological burdens also limit the degree to which new organ dysfunction can be clearly attributed to infection. We note differences in the proportion of patients who received antibiotics increased over time, suggesting that HPS patients who received delayed antibiotics did so much later than their EDPS counterparts. This lag could also arise from diagnostic difficulty.

All three possibilities highlight a potential lead time effect, where the same measured three-hour period on the wards, between meeting sepsis criteria and starting treatment, actually reflects a longer period between (as yet unmeasurable) pathobiologic “time zero” and treatment versus the ED. The time of sepsis detection, as distinct from the time of sepsis onset, therefore proves difficult to evaluate and impossible to account for statistically.

Regardless, our findings suggest additional difficulty in both the recognition and resuscitation of inpatient sepsis. Inpatients, especially with infections, may need closer monitoring. How to cost effectively implement this monitoring is a challenge that deserves attention.

A more rational systems approach to HPS likely combines efforts to improve initial resuscitation with other initiatives aimed at both improving monitoring and preventing infection.

To be clear, we do not imply that timely initial resuscitation does not matter on the wards. Rather, resuscitation-focused QI alone does not appear to be sufficient to overcome differences in outcomes for HPS. The 23.3% attributable mortality risk we observed still implies that resuscitation differences could explain nearly one in four excess HPS mortalities. We previously showed that timely resuscitation is strongly associated with better outcomes.^11,13,30 As discussed above, the unclear degree to which better resuscitation is a marker for more obvious presentations is a persistent limitation of prior investigations and the present study.

Taken together, the ultimate question that this study raises but cannot answer is whether the timely recognition of sepsis, rather than any specific treatment, is what truly improves outcomes.

In addition to those above, this study has several limitations. Our study did not differentiate HPS with respect to patients admitted for noninfectious reasons and who subsequently became septic versus nonseptic patients admitted for an infection who subsequently became septic from that infection. Nor could we discriminate between missed ED diagnoses and true delayed presentations. We note distinguishing these entities clinically can be equally challenging. Additionally, this was a propensity-matched retrospective analysis of an existing sepsis cohort, and the many limitations of both retrospective study and propensity matching apply.^35,36 We note that randomizing patients to develop sepsis in the community versus hospital is not feasible and that two of our aims intended to describe overall patterns rather than causal effects. We could not ascertain robust measures of severity of illness (eg, SOFA) because a real world setting precludes required data points—eg, urine output is unreliably recorded. We also note incomplete overlap between inclusion criteria and either Sepsis-2 or -3 definitions,^1,37 because we designed and populated our database prior to publication of Sepsis-3. Further, we could not account for surgical source control, the appropriateness of antimicrobial therapy, mechanical ventilation before sepsis onset, or most treatments given after initial resuscitation.

In conclusion, hospital-presenting sepsis accounted for adverse patient outcomes disproportionately to prevalence. HPS patients had more complex presentations, received timely antibiotics half as often ED-presenting sepsis, and had nearly twice the mortality odds. Resuscitation disparities explained roughly 25% of this difference.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This investigation was funded in part by a grant from the Center for Medicare and Medicaid Innovation to the High Value Healthcare Collaborative, of which the study sites’ umbrella health system was a part. This grant helped fund the underlying QI program and database in this study.

Sepsis is both the most expensive condition treated and the most common cause of death in hospitals in the United States.^1-3 Most sepsis patients (as many as 80% to 90%) meet sepsis criteria on hospital arrival, but mortality and costs are higher when meeting criteria after admission.^3-6 Mechanisms of this increased mortality for these distinct populations are not well explored. Patients who present septic in the emergency department (ED) and patients who present as inpatients likely present very different challenges for recognition, treatment, and monitoring.⁷ Yet, how these groups differ by demographic and clinical characteristics, the etiology and severity of infection, and patterns of resuscitation care are not well described. Literature on sepsis epidemiology on hospital wards is particularly limited.⁸

This knowledge gap is important. If hospital-presenting sepsis (HPS) contributes disproportionately to disease burdCHFens, it reflects a high-yield population deserving the focus of quality improvement (QI) initiatives. If specific causes of disparities were identified—eg, poor initial resuscitation— they could be specifically targeted for correction. Given that current treatment guidelines are uniform for the two populations,^9,10 characterizing phenotypic differences could also have implications for both diagnostic and therapeutic recommendations, particularly if the groups display substantially differing clinical presentations. Our prior work has not probed these effects specifically, but suggested ED versus inpatient setting at the time of initial sepsis presentation might be an effect modifier for the association between several elements of fluid resuscitation and patient outcomes.^11,12

We, therefore, conducted a retrospective analysis to ask four sequential questions: (1) Do patients with HPS, compared with EDPS, contribute adverse outcome out of proportion to case prevalence? (2) At the time of initial presentation, how do HPS patients differ from EDPS patients with respect to demographics, comorbidities, infectious etiologies, clinical presentations, and severity of illness (3) If holding observed baseline factors constant, does the physical location of sepsis presentation inherently increase the risk for treatment delays and mortality? (4) To what extent can differences in the likelihood for timely initial treatment between the ED and inpatient settings explain differences in mortality and patient outcomes?

We hypothesized a priori that HPS would reflect chronically sicker patients whom both received less timely resuscitation and who contributed disproportionately frequent bad outcomes. We expected disparities in timely resuscitation care would explain a large proportion of this difference.

We performed a retrospective analysis of the Northwell Sepsis Database, a prospectively captured, multisite, real world, consecutive-sample cohort of all “severe sepsis” and septic shock patients treated at nine tertiary and community hospitals in New York from October 1, 2014, to March 31, 2016. We analyzed all patients from a previously published cohort.¹¹

Database Design and Structure

The Northwell Sepsis Database has previously been described in detail.^11,13,14 Briefly, all patients met clinical sepsis criteria: (1) infection AND (2) ≥2 (SIRS) criteria AND (3) ≥1 acute organ dysfunction criterion. Organ dysfunction criteria were hypotension, acute kidney injury (AKI), coagulopathy, altered gas exchange, elevated bilirubin (≥2.0 mg/dL), or altered mental status (AMS; clarified in Supplemental Table 1). All organ dysfunction was not otherwise explained by patients’ medical histories; eg, patients on warfarin anticoagulation were not documented to have coagulopathy based on international normalized ratio > 1.5. The time of the sepsis episode (and database inclusion) was the time of the first vital sign measurement or laboratory result where a patient simultaneously met all three inclusion criteria: infection, SIRS, and organ dysfunction. The database excludes patients who were <18 years, declined bundle interventions, had advance directives precluding interventions, or were admitted directly to palliative care or hospice. Abstractors assumed comorbidities were absent if not documented within the medical record and that physiologic abnormalities were absent if not measured by the treatment team. There were no missing data for the variables analyzed. We report analysis in adherence with the STROBE statement guidelines for observational research.

Exposure

The primary exposure was whether patients had EDPS versus HPS. We defined EDPS patients as meeting all objective clinical inclusion criteria while physically in the ED. We defined HPS as first meeting sepsis inclusion criteria outside the ED, regardless of the reason for admission, and regardless of whether patients were admitted through the ED or directly to the hospital. All ED patients were admitted to the hospital.

Outcomes

Process outcomes were full 3-hour bundle compliance, time to antibiotic administration, blood cultures before antibiotics, time to fluid initiation, the volume of administered fluid resuscitation, lactate result time, and whether repeat lactate was obtained (Supplemental Table 2). Treatment times were times of administration (rather than order time). The primary patient outcome was hospital mortality. Secondary patient outcomes were mechanical ventilation, ICU admission, ICU days, hospital length of stay (LOS). We discounted HPS patients’ LOS to include only days after meeting the inclusion criteria. Patients were excluded from the analysis of the ICU admission outcome if they were already in the ICU prior to meeting sepsis criteria.

Statistical Analysis

We report continuous variables as means (standard deviation) or medians (interquartile range), and categorical variables as frequencies (proportions), as appropriate. Summative statistics with 95% confidence intervals (CI) describe overall group contributions. We used generalized linear models to determine patient factors associated with EDPS versus HPS, entering random effects for individual study sites to control for intercenter variability.

Next, to generate a propensity-matched cohort, we computed propensity scores adjusted from a priori selected variables: age, sex, tertiary versus community hospital, congestive heart failure (CHF), renal failure, COPD, diabetes, liver failure, immunocompromise, primary source of infection, nosocomial source, temperature, initial lactate, presenting hypotension, altered gas exchange, AMS, AKI, and coagulopathy. We then matched subjects 1:1 without optimization or replacement, imposing a caliper width of 0.01; ie, we required matched pairs to have a <1.0% difference in propensity scores. The macro used to match subjects is publically available.¹⁵

We then compared resuscitation and patient outcomes in the matched cohort using generalized linear models, ie, doubly-robust estimation (DRE).¹⁶ When assessing patient outcomes corrected for resuscitation, we used mixed DRE/multivariable regression. We did this for two reasons: first, DRE has the advantage of only requiring only one approach (propensity vs covariate adjustments) to be correctly specified.¹⁶ Second, computing propensity scores adjusted for resuscitation would be inappropriate given that resuscitation occurs after the exposure allocation (HPS vs EDPS). However, these factors could still impact the outcome and in fact, we hypothesized they were potential mediators of the exposure effect. To interrogate this mediating relationship, we recapitulated the DRE modeling but added covariates for resuscitation factors. Resuscitation-adjusted models controlled for timeliness of antibiotics, fluids, and lactate results; blood cultures before antibiotics; repeat lactate obtained, and fluid volume in the first six hours. Since ICU days and LOS are subject to competing risks bias (LOS could be shorter if patients died earlier), we used proportional hazards models where “the event” was defined as a live discharge to censor for mortality and we report output as inverse hazard ratios. We also tested interaction coefficients for discrete bundle elements and HPS to determine if specific bundle elements were effect modifiers for the association between the presenting location and mortality risk. Finally, we estimated attributable risk differences by comparing adjusted odds ratios of adverse outcome with and without adjustment for resuscitation variables, as described by Sahai et al.¹⁷

As sensitivity analyses, we recomputed propensity scores and generated a new matched cohort that excluded HPS patients who met criteria for sepsis while already in the ICU for another reason (ie, excluding ICU-presenting sepsis). We then recapitulated all analyses as above for this cohort. We performed analyses using SAS version 9.4 (SAS Institute, Cary, North Carolina).

Prevalence and Outcome Contributions

Of the 11,182 sepsis patients in the database, we classified 2,509 (22%) as HPS (Figure 1). HPS contributed 785 (35%) of 2,241 sepsis-related mortalities, 1,241 (38%) mechanical ventilations, and 1,762 (34%) ICU admissions. Of 39,263 total ICU days and 127,178 hospital days, HPS contributed 18,104 (46.1%) and 44,412 (34.9%) days, respectively.

Patient Characteristics

Most HPS presented early in the hospital course, with 1,352 (53.9%) cases meeting study criteria within three days of admission. Median time from admission to meeting study criteria for HPS was two days (interquartile range: one to seven days). We report selected baseline patient characteristics in Table 1 and adjusted associations of baseline variables with HPS versus EDPS in Table 2. The full cohort characterization is available in Supplemental Table 3. Notably, HPS patients more often had CHF (aOR [adjusted odds ratio}: 1.31, CI: 1.18-1.47) or renal failure (aOR: 1.62, CI: 1.38-1.91), gastrointestinal source of infection (aOR: 1.84, CI: 1.48-2.29), hypothermia (aOR: 1.56, CI: 1.28-1.90) hypotension (aOR: 1.85, CI: 1.65-2.08), or altered gas exchange (aOR: 2.46, CI: 1.43-4.24). In contrast, HPS patients less frequently were admitted from skilled nursing facilities (aOR: 0.44, CI: 0.32-0.60), or had COPD (aOR: 0.53, CI: 0.36-0.76), fever (aOR: 0.70, CI: 0.52-0.91), tachypnea (aOR: 0.76, CI: 0.58-0.98), or AKI (aOR: 082, CI: 0.68-0.97). Other baseline variables were similar, including respiratory source, tachycardia, white cell abnormalities, AMS, and coagulopathies. These associations were preserved in the sensitivity analysis excluding ICU-presenting sepsis.

Propensity Matching

Propensity score matching yielded 1,942 matched pairs (n = 3,884, 77% of HPS patients, 22% of EDPS patients). Table 1 and Supplemental Table 3 show patient characteristics after propensity matching. Supplemental Table 4 shows the propensity model. The frequency densities are shown for the cohort as a function of propensity score in Supplemental Figure 1. After matching, frequencies between groups differed by <5% for all categorical variables assessed. In the sensitivity analysis, propensity matching (model in Supplemental Table 5) resulted in 1,233 matched pairs (n = 2,466, 49% of HPS patients, 14% of EDPS patients), with group differences comparable to the primary analysis.

Process Outcomes

We present propensity-matched differences in initial resuscitation in Figure 2A for all HPS patients, as well as non-ICU-presenting HPS, versus EDPS. HPS patients were roughly half as likely to receive fully 3-hour bundle compliant care (17.0% vs 30.3%, aOR: 0.47, CI: 0.40-0.57), to have blood cultures drawn within three hours prior to antibiotics (44.9% vs 67.2%, aOR: 0.40, CI: 0.35-0.46), or to receive fluid resuscitation initiated within two hours (11.1% vs 26.1%, aOR: 0.35, CI: 0.29-0.42). Antibiotic receipt within one hour was comparable (45.3% vs 48.1%, aOR: 0.89, CI: 0.79-1.01). However, differences emerged for antibiotics within three hours (66.2% vs 83.8%, aOR: 0.38, CI: 0.32-0.44) and persisted at six hours (77.0% vs 92.5%, aOR: 0.27, CI: 0.22-33). Excluding ICU-presenting sepsis from propensity matching exaggerated disparities in antibiotic receipt at one hour (43.4% vs 49.1%, aOR: 0.80, CI: 0.68-0.93), three hours (64.2% vs 86.1%, aOR: 0.29, CI: 0.24-0.35), and six hours (75.7% vs 93.0%, aOR: 0.23, CI: 0.18-0.30). HPS patients more frequently had repeat lactate obtained within 24 hours (62.4% vs 54.3%, aOR: 1.40, CI: 1.23-1.59).

Patient Outcomes

HPS patients had higher mortality (31.2% vs19.3%), mechanical ventilation (51.5% vs27.4%), and ICU admission (60.6% vs 46.5%) (Table 1 and Supplemental Table 6). Figure 2b shows propensity-matched and covariate-adjusted differences in patient outcomes before and after adjusting for initial resuscitation. aORs corresponded to approximate relative risk differences¹⁸ of 1.38 (CI: 1.28-1.48), 1.68 (CI: 1.57-1.79), and 1.72 (CI: 1.61-1.84) for mortality, mechanical ventilation, and ICU admission, respectively. HPS was associated with 83% longer mortality-censored ICU stays (five vs nine days, HR^–1: 1.83, CI: 1.65-2.03), and 108% longer hospital stay (eight vs 17 days, HR^–1: 2.08, CI: 1.93-2.24). After adjustment for resuscitation, all effect sizes decreased but persisted. The initial crystalloid volume was a significant negative effect modifier for mortality (Supplemental Table 7). That is, the magnitude of the association between HPS and greater mortality decreased by a factor of 0.89 per 10 mL/kg given (CI: 0.82-0.97). We did not observe significant interaction from other interventions, or overall bundle compliance, meaning these interventions’ association with mortality did not significantly differ between HPS versus EDPS.

The implied attributable risk difference from discrepancies in initial resuscitation was 23.3% for mortality, 35.2% for mechanical ventilation, and 7.6% for ICU admission (Figure 2B). Resuscitation explained 26.5% of longer ICU LOS and 16.7% of longer hospital LOS associated with HPS.

Figure 2C shows sensitivity analysis excluding ICU-presenting sepsis from propensity matching (ie, limiting HPS to hospital ward presentations). Again, HPS was associated with all adverse outcomes, though effect sizes were smaller than in the primary cohort for all outcomes except hospital LOS. In this cohort, resuscitation factors now explained 16.5% of HPS’ association with mortality, and 14.5% of the association with longer ICU LOS. However, they explained a greater proportion (13.0%) of ICU admissions. Attributable risk differences were comparable to the primary cohort for mechanical ventilation (37.6%) and hospital LOS (15.3%).

In this analysis of 11,182 sepsis and septic shock patients, HPS contributed 22% of prevalence but >35% of total sepsis mortalities, ICU utilization, and hospital days. HPS patients had higher comorbidity burdens and had clinical presentations less obviously attributable to infection with more severe organ dysfunction. EDPS received antibiotics within three hours about 1.62 times more often than do HPS patients. EDPS patients also receive fluids initiated within two hours about 1.82 times more often than HPS patients do. HPS had nearly 1.5-fold greater mortality and LOS, and nearly two-fold greater mechanical ventilation and ICU utilization. Resuscitation disparities could partially explain these associations. These patterns persisted when comparing only wards presenting HPS with EDPS.

Our analysis revealed several notable findings. First, these data confirm that HPS represents a potentially high-impact target population that contributes adverse outcomes disproportionately frequently with respect to case prevalence.

Our findings, unsurprisingly, revealed HPS and EDPS reflect dramatically different patient populations. We found that the two groups significantly differed by the majority of the baseline factors we compared. It may be worth asking if and how these substantial differences in illness etiology, chronic health, and acute physiology impact what we consider an optimal approach to management. Significant interaction effects of fluid volume on the association between HPS and mortality suggest differential treatment effects may exist between patients. Indeed, patients who newly arrive from the community and those who are several days into admission likely have different volume status. However, no interactions were noted with other bundle elements, such as timeliness of antibiotics or timeliness of initial fluids.

Another potentially concerning observation was that HPS patients were admitted much less frequently from skilled nursing facilities, as it could imply that this poorer-fairing population had a comparatively higher baseline functional status. The fact that 25% of EDPS cases were admitted from these facilities also underscores the need to engage skilled nursing facility providers in future sepsis initiatives.

We found marked disparities in resuscitation. Timely delivery of interventions, such as antibiotics and initial fluid resuscitation, occurred less than half as often for HPS, especially on hospital wards. While evidence supporting the efficacy of specific 3-hour bundle elements remains unsettled,¹⁹ a wealth of literature demonstrates a correlation between bundle uptake and decreased sepsis mortality, especially for early antibiotic administration.^13,20-26 Some analysis suggests that differing initial resuscitation practices explain different mortality rates in the early goal-directed therapy trials.²⁷ The comparatively poor performance for non-ICU HPS indicates further QI efforts are better focused on inpatient wards, rather than on EDs or ICUs where resuscitation is already delivered with substantially greater fidelity.

While resuscitation differences partially explained outcome discrepancies between groups, they did not account for as much variation as expected. Though resuscitation accounted for >35% of attributable mechanical ventilation risk, it explained only 16.5% of mortality differences for non-ICU HPS vs EDPS. We speculate that several factors may contribute.

First, HPS patients are already hospitalized for another acute insult and may be too physiologically brittle to derive equal benefit from initial resuscitation. Some literature suggests protocolized sepsis resuscitation may paradoxically be more effective in milder/earlier disease.²⁸

Second, clinical information indicating septic organ dysfunction may become available too late in HPS—a possible data limitation where inpatient providers are counterintuitively more likely to miss early signs of patients’ deterioration and a subsequent therapeutic window. Several studies found that fluid resuscitation is associated with improved sepsis outcomes only when it is administered very early.^11,29-31 In inpatient wards, decreased monitoring³² and human factors (eg, hospital workflow, provider-to-patient ratios, electronic documentation burdens)^33,34 may hinder early diagnosis. In contrast, ED environments are explicitly designed to identify acutely ill patients and deliver intervention rapidly. If HPS patients were sicker when they were identified, this would also explain their more severe organ dysfunctions. Our data seems to support this possibility. HPS patients had tachypnea less frequently but more often had impaired gas exchange. This finding may suggest that early tachypnea was either less often detected or documented, or that it had progressed further by the time of detection.

Third, inpatients with sepsis may more often present with greater diagnostic complexity. We observed that HPS patients were more often euthermic and less often tachypneic. Beyond suggesting a greater diagnostic challenge, this also raises questions as to whether differences reflect patient physiology (response to infection) or iatrogenic factors (eg, prior antipyretics). Higher comorbidity and acute physiological burdens also limit the degree to which new organ dysfunction can be clearly attributed to infection. We note differences in the proportion of patients who received antibiotics increased over time, suggesting that HPS patients who received delayed antibiotics did so much later than their EDPS counterparts. This lag could also arise from diagnostic difficulty.

All three possibilities highlight a potential lead time effect, where the same measured three-hour period on the wards, between meeting sepsis criteria and starting treatment, actually reflects a longer period between (as yet unmeasurable) pathobiologic “time zero” and treatment versus the ED. The time of sepsis detection, as distinct from the time of sepsis onset, therefore proves difficult to evaluate and impossible to account for statistically.

Regardless, our findings suggest additional difficulty in both the recognition and resuscitation of inpatient sepsis. Inpatients, especially with infections, may need closer monitoring. How to cost effectively implement this monitoring is a challenge that deserves attention.

A more rational systems approach to HPS likely combines efforts to improve initial resuscitation with other initiatives aimed at both improving monitoring and preventing infection.

To be clear, we do not imply that timely initial resuscitation does not matter on the wards. Rather, resuscitation-focused QI alone does not appear to be sufficient to overcome differences in outcomes for HPS. The 23.3% attributable mortality risk we observed still implies that resuscitation differences could explain nearly one in four excess HPS mortalities. We previously showed that timely resuscitation is strongly associated with better outcomes.^11,13,30 As discussed above, the unclear degree to which better resuscitation is a marker for more obvious presentations is a persistent limitation of prior investigations and the present study.

Taken together, the ultimate question that this study raises but cannot answer is whether the timely recognition of sepsis, rather than any specific treatment, is what truly improves outcomes.

In addition to those above, this study has several limitations. Our study did not differentiate HPS with respect to patients admitted for noninfectious reasons and who subsequently became septic versus nonseptic patients admitted for an infection who subsequently became septic from that infection. Nor could we discriminate between missed ED diagnoses and true delayed presentations. We note distinguishing these entities clinically can be equally challenging. Additionally, this was a propensity-matched retrospective analysis of an existing sepsis cohort, and the many limitations of both retrospective study and propensity matching apply.^35,36 We note that randomizing patients to develop sepsis in the community versus hospital is not feasible and that two of our aims intended to describe overall patterns rather than causal effects. We could not ascertain robust measures of severity of illness (eg, SOFA) because a real world setting precludes required data points—eg, urine output is unreliably recorded. We also note incomplete overlap between inclusion criteria and either Sepsis-2 or -3 definitions,^1,37 because we designed and populated our database prior to publication of Sepsis-3. Further, we could not account for surgical source control, the appropriateness of antimicrobial therapy, mechanical ventilation before sepsis onset, or most treatments given after initial resuscitation.

In conclusion, hospital-presenting sepsis accounted for adverse patient outcomes disproportionately to prevalence. HPS patients had more complex presentations, received timely antibiotics half as often ED-presenting sepsis, and had nearly twice the mortality odds. Resuscitation disparities explained roughly 25% of this difference.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This investigation was funded in part by a grant from the Center for Medicare and Medicaid Innovation to the High Value Healthcare Collaborative, of which the study sites’ umbrella health system was a part. This grant helped fund the underlying QI program and database in this study.

Anticoagulation is commonly used in the management of atrial fibrillation to reduce the risk of ischemic stroke. Warfarin and other anticoagulants increase the risk of hemorrhagic complications, including upper gastrointestinal bleeding (UGIB). Following UGIB, management of anticoagulation is highly variable. Many patients permanently discontinue anticoagulation, while others continue without interruption.^1-4 Among patients who resume warfarin, different cohorts have measured median times to resumption ranging from four days to 50 days.^1-3 Outcomes data are sparse, and clinical guidelines offer little direction.⁵

Following UGIB, the balance between the risks and benefits of anticoagulation changes over time. Rebleeding risk is highest immediately after the event and declines quickly; therefore, rapid resumption of anticoagulation causes patient harm.³ Meanwhile, the risk of stroke remains constant, and delay in resumption of anticoagulation is associated with increased risk of stroke and death.¹ At some point in time following the initial UGIB, the expected harm from bleeding would equal the expected harm from stroke. This time point would represent the optimal time to restart anticoagulation.

Trial data are unlikely to identify the optimal time for restarting anticoagulation. A randomized trial comparing discrete reinitiation times (eg, two weeks vs six weeks) may easily miss the optimal timing. Moreover, because the daily probability of thromboembolic events is low, large numbers of patients would be required to power such a study. In addition, a number of oral anticoagulants are now approved for prevention of thromboembolic stroke in atrial fibrillation, and each drug may have different optimal timing.

In contrast to randomized trials that would be impracticable for addressing this clinical issue, microsimulation modeling can provide granular information regarding the optimal time to restart anticoagulation. Herein, we set out to estimate the expected benefit of reinitiation of warfarin, the most commonly used oral anticoagulant,⁶ or apixaban, the direct oral anticoagulant with the most favorable risk profile,⁷ as a function of days after UGIB.

We previously described a microsimulation model of anticoagulation among patients with nonvalvular atrial fibrillation (NVAF; hereafter, we refer to this model as the Personalized Anticoagulation Decision-Making Assistance model, or PADMA).^8,9 For this study, we extended this model to incorporate the probability of rebleeding following UGIB and include apixaban as an alternative to warfarin. This model begins with a synthetic population following UGIB, the members of which are at varying risk for thromboembolism, recurrent UGIB, and other hemorrhages. For each patient, the model simulates a number of possible events (eg, thromboembolic stroke, intracranial hemorrhage, rebleeding, and other major extracranial hemorrhages) on each day of an acute period of 90 days after hemostasis. Patients who survive until the end of the acute period enter a simulation with annual, rather than daily, cycles. Our model then estimates total quality-adjusted life-years (QALYs) for each patient, discounted to the present. We report the average discounted QALYs produced by the model for the same population if all individuals in our input population were to resume either warfarin or apixaban on a specific day. Input parameters and ranges are summarized in Table 1, a simplified schematic of our model is shown in the Supplemental Appendix, and additional details regarding model structure and assumptions can be found in earlier work.^8,9 We simulated from a health system perspective over a lifelong time horizon. All analyses were performed in version 14 of Stata (StataCorp, LLC, College Station, Texas).

Synthetic Population

To generate a population reflective of the comorbidities and age distribution of the US population with NVAF, we merged relevant variables from the National Health and Nutrition Examination Survey (NHANES; 2011-2012), using multiple imputation to correct for missing variables.¹⁰ We then bootstrapped to national population estimates by age and sex to arrive at a hypothetical population of the United States.¹¹ Because NHANES does not include atrial fibrillation, we applied sex- and age-specific prevalence rates from the AnTicoagulation and Risk Factors In Atrial Fibrillation study.¹² We then calculated commonly used risk scores (CHA₂DS₂-Vasc and HAS-BLED) for each patient and limited the population to patients with a CHA₂DS₂-Vasc score of one or greater.^13,14 The population resuming apixaban was further limited to patients whose creatinine clearance was 25 mL/min or greater in keeping with the entry criteria in the phase 3 clinical trial on which the medication’s approval was based.¹⁵

To estimate patient-specific probability of rebleeding, we generated a Rockall score for each patient.¹⁶ Although the discrimination of the Rockall score is limited for individual patients, as with all other tools used to predict rebleeding following UGIB, the Rockall score has demonstrated reasonable calibration across a threefold risk gradient.^17-19 International consensus guidelines recommend the Rockall score as one of two risk prediction tools for clinical use in the management of patients with UGIB.²⁰ In addition, because the Rockall score includes some demographic components (five of a possible 11 points), our estimates of rebleeding risk are covariant with other patient-specific risks. We assumed that the endoscopic components of the Rockall score were present in our cohort at the same frequency as in the original derivation and are independent of known patient risk factors.¹⁶ For example, 441 out of 4,025 patients in the original Rockall derivation cohort presented with a systolic blood pressure less than 100 mm Hg. We assumed that an independent and random 10.96% of the cohort would present with shock, which confers two points in the Rockall score.

The population was replicated 60 times, with identical copies of the population resuming anticoagulation on each of days 1-60 (where day zero represents hemostasis). Intermediate data regarding our simulated population can be found in the Supplemental Appendix and in prior work.

Event Type, Severity, and Mortality

Each patient in our simulation could sustain several discrete and independent events: ischemic stroke, intracranial hemorrhage, recurrent UGIB, or extracranial major hemorrhage other than recurrent UGIB. As in prior analyses using the PADMA model, we did not consider minor hemorrhagic events.⁸

The probability of each event was conditional on the corresponding risk scoring system. Patient-specific probability of ischemic stroke was conditional on CHA₂DS₂-Vasc score.^21,22 Patient-specific probability of intracranial hemorrhage was conditional on HAS-BLED score, with the proportions of intracranial hemorrhage of each considered subtype (intracerebral, subarachnoid, or subdural) bootstrapped from previously-published data.^21-24 Patient-specific probability of rebleeding was conditional on Rockall score from the combined Rockall and Vreeburg validation cohorts.¹⁷ Patient-specific probability of extracranial major hemorrhage was conditional on HAS-BLED score.²¹ To avoid double-counting of UGIB, we subtracted the baseline risk of UGIB from the overall rate of extracranial major hemorrhages using previously-published data regarding relative frequency and a bootstrapping approach.²⁵

Probability of Rebleeding Over Time

To estimate the decrease in rebleeding risk over time, we searched the Medline database for systematic reviews of recurrent bleeding following UGIB using the strategy detailed in the Supplemental Appendix. Using the interval rates of rebleeding we identified, we calculated implied daily rates of rebleeding at the midpoint of each interval. For example, 39.5% of rebleeding events occurred within three days of hemostasis, implying a daily rate of approximately 13.2% on day two (32 of 81 events over a three-day period). We repeated this process to estimate daily rates at the midpoint of each reported time interval and fitted an exponential decay function.²⁶ Our exponential fitted these datapoints quite well, but we lacked sufficient data to test other survival functions (eg, Gompertz, lognormal, etc.). Our fitted exponential can be expressed as:

P _rebleeding = b ₀ *exp(b ₁ *day)

where b₀ = 0.1843 (SE: 0.0136) and b₁ = –0.1563 (SE: 0.0188). For example, a mean of 3.9% of rebleeding episodes will occur on day 10 (0.1843 *exp(–0.1563 *10)).

Relative Risks of Events with Anticoagulation

For patients resuming warfarin, the probabilities of each event were adjusted based on patient-specific daily INR. All INRs were assumed to be 1.0 until the day of warfarin reinitiation, after which interpolated trajectories of postinitiation INR measurements were sampled for each patient from an earlier study of clinical warfarin initiation.²⁷ Relative risks of ischemic stroke and hemorrhagic events were calculated based on each day’s INR.

For patients taking apixaban, we assumed that the medication would reach full therapeutic effect one day after reinitiation. Based on available evidence, we applied the relative risks of each event with apixaban compared with warfarin.²⁵

Future Disability and Mortality

Each event in our simulation resulted in hospitalization. Length of stay was sampled for each diagnosis.²⁸ The disutility of hospitalization was estimated based on length of stay.⁸ Inpatient mortality and future disability were predicted for each event as previously described.⁸ We assumed that recurrent episodes of UGIB conferred morbidity and mortality identical to extracranial major hemorrhages more broadly.^29,30

Disutilities

We used a multiplicative model for disutility with baseline utilities conditional on age and sex.³¹ Each day after resumption of anticoagulation carried a disutility of 0.012 for warfarin or 0.002 for apixaban, which we assumed to be equivalent to aspirin in disutility.³² Long-term disutility and life expectancy were conditional on modified Rankin Score (mRS).^33,34 We discounted all QALYs to day zero using standard exponential discounting and a discount rate centered at 3%. We then computed the average discounted QALYs among the cohort of patients that resumed anticoagulation on each day following the index UGIB.

Sensitivity Analyses and Metamodel

To assess sensitivity to continuously varying input parameters, such as discount rate, the proportion of extracranial major hemorrhages that are upper GI bleeds, and inpatient mortality from extracranial major hemorrhage, we constructed a metamodel (a regression model of our microsimulation results).³⁵ We tested for interactions among input parameters and dropped parameters that were not statistically significant predictors of discounted QALYs from our metamodel. We then tested for interactions between each parameter and day resuming anticoagulation to determine which factors may impact the optimal day of reinitiation. Finally, we used predicted marginal effects from our metamodel to assess the change in optimal day across the ranges of each input parameter when other parameters were held at their medians.

Resuming warfarin on day zero produced the fewest QALYs. With delay in reinitiation of anticoagulation, expected QALYs increased, peaked, and then declined for all scenarios. In our base-case simulation of warfarin, peak utility was achieved by resumption 41 days after the index UGIB. Resumption between days 32 and 51 produced greater than 99.9% of peak utility. In our base-case simulation of apixaban, peak utility was achieved by resumption 32 days after the index UGIB. Resumption between days 21 and 47 produced greater than 99.9% of peak utility. Results for warfarin and apixaban are shown in Figures 1 and 2, respectively.

The optimal day of warfarin reinitiation was most sensitive to CHA₂DS₂-Vasc scores and varied by around 11 days between a CHA₂DS₂-Vasc score of one and a CHA₂DS₂-Vasc score of six (the 5^th and 95^th percentiles, respectively) when all other parameters are held at their medians. Results were comparatively insensitive to rebleeding risk. Varying Rockall score from two to seven (the 5^th and 95^th percentiles, respectively) added three days to optimal warfarin resumption. Varying other parameters from the 5^th to the 95^th percentile (including HAS-BLED score, sex, age, and discount rate) changed expected QALYs but did not change the optimal day of reinitiation of warfarin. Optimal day of reinitiation for warfarin stratified by CHA₂DS₂-Vasc score is shown in Table 2.

Anticoagulation is frequently prescribed for patients with NVAF, and hemorrhagic complications are common. Although anticoagulants are withheld following hemorrhages, scant evidence to inform the optimal timing of reinitiation is available. In this microsimulation analysis, we found that the optimal time to reinitiate anticoagulation following UGIB is around 41 days for warfarin and around 32 days for apixaban. We have further demonstrated that the optimal timing of reinitiation can vary by nearly two weeks, depending on a patient’s underlying risk of stroke, and that early reinitiation is more sensitive to rebleeding risk than late reinitiation.

Prior work has shown that early reinitiation of anticoagulation leads to higher rates of recurrent hemorrhage while failure to reinitiate anticoagulation is associated with higher rates of stroke and mortality.^1-4,36 Our results add to the literature in a number of important ways. First, our model not only confirms that anticoagulation should be restarted but also suggests when this action should be taken. The competing risks of bleeding and stroke have left clinicians with little guidance; we have quantified the clinical reasoning required for the decision to resume anticoagulation. Second, by including the disutility of hospitalization and long-term disability, our model more accurately represents the complex tradeoffs between recurrent hemorrhage and (potentially disabling) stroke than would a comparison of event rates. Third, our model is conditional upon patient risk factors, allowing clinicians to personalize the timing of anticoagulation resumption. Theory would suggest that patients at higher risk of ischemic stroke benefit from earlier resumption of anticoagulation, while patients at higher risk of hemorrhage benefit from delayed reinitiation. We have quantified the extent to which patient-specific risks should change timing. Fourth, we offer a means of improving expected health outcomes that requires little more than appropriate scheduling. Current practice regarding resuming anticoagulation is widely variable. Many patients never resume warfarin, and those that do resume do so after highly varied periods of time.^1-5,36 We offer a means of standardizing clinical practice and improving expected patient outcomes.

Disclosures

The authors have nothing to disclose.

Funding

The authors received no specific funding for this work.

Anticoagulation is commonly used in the management of atrial fibrillation to reduce the risk of ischemic stroke. Warfarin and other anticoagulants increase the risk of hemorrhagic complications, including upper gastrointestinal bleeding (UGIB). Following UGIB, management of anticoagulation is highly variable. Many patients permanently discontinue anticoagulation, while others continue without interruption.^1-4 Among patients who resume warfarin, different cohorts have measured median times to resumption ranging from four days to 50 days.^1-3 Outcomes data are sparse, and clinical guidelines offer little direction.⁵

Following UGIB, the balance between the risks and benefits of anticoagulation changes over time. Rebleeding risk is highest immediately after the event and declines quickly; therefore, rapid resumption of anticoagulation causes patient harm.³ Meanwhile, the risk of stroke remains constant, and delay in resumption of anticoagulation is associated with increased risk of stroke and death.¹ At some point in time following the initial UGIB, the expected harm from bleeding would equal the expected harm from stroke. This time point would represent the optimal time to restart anticoagulation.

Trial data are unlikely to identify the optimal time for restarting anticoagulation. A randomized trial comparing discrete reinitiation times (eg, two weeks vs six weeks) may easily miss the optimal timing. Moreover, because the daily probability of thromboembolic events is low, large numbers of patients would be required to power such a study. In addition, a number of oral anticoagulants are now approved for prevention of thromboembolic stroke in atrial fibrillation, and each drug may have different optimal timing.

In contrast to randomized trials that would be impracticable for addressing this clinical issue, microsimulation modeling can provide granular information regarding the optimal time to restart anticoagulation. Herein, we set out to estimate the expected benefit of reinitiation of warfarin, the most commonly used oral anticoagulant,⁶ or apixaban, the direct oral anticoagulant with the most favorable risk profile,⁷ as a function of days after UGIB.

We previously described a microsimulation model of anticoagulation among patients with nonvalvular atrial fibrillation (NVAF; hereafter, we refer to this model as the Personalized Anticoagulation Decision-Making Assistance model, or PADMA).^8,9 For this study, we extended this model to incorporate the probability of rebleeding following UGIB and include apixaban as an alternative to warfarin. This model begins with a synthetic population following UGIB, the members of which are at varying risk for thromboembolism, recurrent UGIB, and other hemorrhages. For each patient, the model simulates a number of possible events (eg, thromboembolic stroke, intracranial hemorrhage, rebleeding, and other major extracranial hemorrhages) on each day of an acute period of 90 days after hemostasis. Patients who survive until the end of the acute period enter a simulation with annual, rather than daily, cycles. Our model then estimates total quality-adjusted life-years (QALYs) for each patient, discounted to the present. We report the average discounted QALYs produced by the model for the same population if all individuals in our input population were to resume either warfarin or apixaban on a specific day. Input parameters and ranges are summarized in Table 1, a simplified schematic of our model is shown in the Supplemental Appendix, and additional details regarding model structure and assumptions can be found in earlier work.^8,9 We simulated from a health system perspective over a lifelong time horizon. All analyses were performed in version 14 of Stata (StataCorp, LLC, College Station, Texas).

Synthetic Population

To generate a population reflective of the comorbidities and age distribution of the US population with NVAF, we merged relevant variables from the National Health and Nutrition Examination Survey (NHANES; 2011-2012), using multiple imputation to correct for missing variables.¹⁰ We then bootstrapped to national population estimates by age and sex to arrive at a hypothetical population of the United States.¹¹ Because NHANES does not include atrial fibrillation, we applied sex- and age-specific prevalence rates from the AnTicoagulation and Risk Factors In Atrial Fibrillation study.¹² We then calculated commonly used risk scores (CHA₂DS₂-Vasc and HAS-BLED) for each patient and limited the population to patients with a CHA₂DS₂-Vasc score of one or greater.^13,14 The population resuming apixaban was further limited to patients whose creatinine clearance was 25 mL/min or greater in keeping with the entry criteria in the phase 3 clinical trial on which the medication’s approval was based.¹⁵

To estimate patient-specific probability of rebleeding, we generated a Rockall score for each patient.¹⁶ Although the discrimination of the Rockall score is limited for individual patients, as with all other tools used to predict rebleeding following UGIB, the Rockall score has demonstrated reasonable calibration across a threefold risk gradient.^17-19 International consensus guidelines recommend the Rockall score as one of two risk prediction tools for clinical use in the management of patients with UGIB.²⁰ In addition, because the Rockall score includes some demographic components (five of a possible 11 points), our estimates of rebleeding risk are covariant with other patient-specific risks. We assumed that the endoscopic components of the Rockall score were present in our cohort at the same frequency as in the original derivation and are independent of known patient risk factors.¹⁶ For example, 441 out of 4,025 patients in the original Rockall derivation cohort presented with a systolic blood pressure less than 100 mm Hg. We assumed that an independent and random 10.96% of the cohort would present with shock, which confers two points in the Rockall score.

The population was replicated 60 times, with identical copies of the population resuming anticoagulation on each of days 1-60 (where day zero represents hemostasis). Intermediate data regarding our simulated population can be found in the Supplemental Appendix and in prior work.

Event Type, Severity, and Mortality

Each patient in our simulation could sustain several discrete and independent events: ischemic stroke, intracranial hemorrhage, recurrent UGIB, or extracranial major hemorrhage other than recurrent UGIB. As in prior analyses using the PADMA model, we did not consider minor hemorrhagic events.⁸

The probability of each event was conditional on the corresponding risk scoring system. Patient-specific probability of ischemic stroke was conditional on CHA₂DS₂-Vasc score.^21,22 Patient-specific probability of intracranial hemorrhage was conditional on HAS-BLED score, with the proportions of intracranial hemorrhage of each considered subtype (intracerebral, subarachnoid, or subdural) bootstrapped from previously-published data.^21-24 Patient-specific probability of rebleeding was conditional on Rockall score from the combined Rockall and Vreeburg validation cohorts.¹⁷ Patient-specific probability of extracranial major hemorrhage was conditional on HAS-BLED score.²¹ To avoid double-counting of UGIB, we subtracted the baseline risk of UGIB from the overall rate of extracranial major hemorrhages using previously-published data regarding relative frequency and a bootstrapping approach.²⁵

Probability of Rebleeding Over Time

To estimate the decrease in rebleeding risk over time, we searched the Medline database for systematic reviews of recurrent bleeding following UGIB using the strategy detailed in the Supplemental Appendix. Using the interval rates of rebleeding we identified, we calculated implied daily rates of rebleeding at the midpoint of each interval. For example, 39.5% of rebleeding events occurred within three days of hemostasis, implying a daily rate of approximately 13.2% on day two (32 of 81 events over a three-day period). We repeated this process to estimate daily rates at the midpoint of each reported time interval and fitted an exponential decay function.²⁶ Our exponential fitted these datapoints quite well, but we lacked sufficient data to test other survival functions (eg, Gompertz, lognormal, etc.). Our fitted exponential can be expressed as:

P _rebleeding = b ₀ *exp(b ₁ *day)

where b₀ = 0.1843 (SE: 0.0136) and b₁ = –0.1563 (SE: 0.0188). For example, a mean of 3.9% of rebleeding episodes will occur on day 10 (0.1843 *exp(–0.1563 *10)).

Relative Risks of Events with Anticoagulation

For patients resuming warfarin, the probabilities of each event were adjusted based on patient-specific daily INR. All INRs were assumed to be 1.0 until the day of warfarin reinitiation, after which interpolated trajectories of postinitiation INR measurements were sampled for each patient from an earlier study of clinical warfarin initiation.²⁷ Relative risks of ischemic stroke and hemorrhagic events were calculated based on each day’s INR.

For patients taking apixaban, we assumed that the medication would reach full therapeutic effect one day after reinitiation. Based on available evidence, we applied the relative risks of each event with apixaban compared with warfarin.²⁵

Future Disability and Mortality

Each event in our simulation resulted in hospitalization. Length of stay was sampled for each diagnosis.²⁸ The disutility of hospitalization was estimated based on length of stay.⁸ Inpatient mortality and future disability were predicted for each event as previously described.⁸ We assumed that recurrent episodes of UGIB conferred morbidity and mortality identical to extracranial major hemorrhages more broadly.^29,30

Disutilities

We used a multiplicative model for disutility with baseline utilities conditional on age and sex.³¹ Each day after resumption of anticoagulation carried a disutility of 0.012 for warfarin or 0.002 for apixaban, which we assumed to be equivalent to aspirin in disutility.³² Long-term disutility and life expectancy were conditional on modified Rankin Score (mRS).^33,34 We discounted all QALYs to day zero using standard exponential discounting and a discount rate centered at 3%. We then computed the average discounted QALYs among the cohort of patients that resumed anticoagulation on each day following the index UGIB.

Sensitivity Analyses and Metamodel

To assess sensitivity to continuously varying input parameters, such as discount rate, the proportion of extracranial major hemorrhages that are upper GI bleeds, and inpatient mortality from extracranial major hemorrhage, we constructed a metamodel (a regression model of our microsimulation results).³⁵ We tested for interactions among input parameters and dropped parameters that were not statistically significant predictors of discounted QALYs from our metamodel. We then tested for interactions between each parameter and day resuming anticoagulation to determine which factors may impact the optimal day of reinitiation. Finally, we used predicted marginal effects from our metamodel to assess the change in optimal day across the ranges of each input parameter when other parameters were held at their medians.

Resuming warfarin on day zero produced the fewest QALYs. With delay in reinitiation of anticoagulation, expected QALYs increased, peaked, and then declined for all scenarios. In our base-case simulation of warfarin, peak utility was achieved by resumption 41 days after the index UGIB. Resumption between days 32 and 51 produced greater than 99.9% of peak utility. In our base-case simulation of apixaban, peak utility was achieved by resumption 32 days after the index UGIB. Resumption between days 21 and 47 produced greater than 99.9% of peak utility. Results for warfarin and apixaban are shown in Figures 1 and 2, respectively.

The optimal day of warfarin reinitiation was most sensitive to CHA₂DS₂-Vasc scores and varied by around 11 days between a CHA₂DS₂-Vasc score of one and a CHA₂DS₂-Vasc score of six (the 5^th and 95^th percentiles, respectively) when all other parameters are held at their medians. Results were comparatively insensitive to rebleeding risk. Varying Rockall score from two to seven (the 5^th and 95^th percentiles, respectively) added three days to optimal warfarin resumption. Varying other parameters from the 5^th to the 95^th percentile (including HAS-BLED score, sex, age, and discount rate) changed expected QALYs but did not change the optimal day of reinitiation of warfarin. Optimal day of reinitiation for warfarin stratified by CHA₂DS₂-Vasc score is shown in Table 2.

Anticoagulation is frequently prescribed for patients with NVAF, and hemorrhagic complications are common. Although anticoagulants are withheld following hemorrhages, scant evidence to inform the optimal timing of reinitiation is available. In this microsimulation analysis, we found that the optimal time to reinitiate anticoagulation following UGIB is around 41 days for warfarin and around 32 days for apixaban. We have further demonstrated that the optimal timing of reinitiation can vary by nearly two weeks, depending on a patient’s underlying risk of stroke, and that early reinitiation is more sensitive to rebleeding risk than late reinitiation.

Prior work has shown that early reinitiation of anticoagulation leads to higher rates of recurrent hemorrhage while failure to reinitiate anticoagulation is associated with higher rates of stroke and mortality.^1-4,36 Our results add to the literature in a number of important ways. First, our model not only confirms that anticoagulation should be restarted but also suggests when this action should be taken. The competing risks of bleeding and stroke have left clinicians with little guidance; we have quantified the clinical reasoning required for the decision to resume anticoagulation. Second, by including the disutility of hospitalization and long-term disability, our model more accurately represents the complex tradeoffs between recurrent hemorrhage and (potentially disabling) stroke than would a comparison of event rates. Third, our model is conditional upon patient risk factors, allowing clinicians to personalize the timing of anticoagulation resumption. Theory would suggest that patients at higher risk of ischemic stroke benefit from earlier resumption of anticoagulation, while patients at higher risk of hemorrhage benefit from delayed reinitiation. We have quantified the extent to which patient-specific risks should change timing. Fourth, we offer a means of improving expected health outcomes that requires little more than appropriate scheduling. Current practice regarding resuming anticoagulation is widely variable. Many patients never resume warfarin, and those that do resume do so after highly varied periods of time.^1-5,36 We offer a means of standardizing clinical practice and improving expected patient outcomes.

Disclosures

The authors have nothing to disclose.

Funding

The authors received no specific funding for this work.

Anticoagulation is commonly used in the management of atrial fibrillation to reduce the risk of ischemic stroke. Warfarin and other anticoagulants increase the risk of hemorrhagic complications, including upper gastrointestinal bleeding (UGIB). Following UGIB, management of anticoagulation is highly variable. Many patients permanently discontinue anticoagulation, while others continue without interruption.^1-4 Among patients who resume warfarin, different cohorts have measured median times to resumption ranging from four days to 50 days.^1-3 Outcomes data are sparse, and clinical guidelines offer little direction.⁵

Following UGIB, the balance between the risks and benefits of anticoagulation changes over time. Rebleeding risk is highest immediately after the event and declines quickly; therefore, rapid resumption of anticoagulation causes patient harm.³ Meanwhile, the risk of stroke remains constant, and delay in resumption of anticoagulation is associated with increased risk of stroke and death.¹ At some point in time following the initial UGIB, the expected harm from bleeding would equal the expected harm from stroke. This time point would represent the optimal time to restart anticoagulation.

Trial data are unlikely to identify the optimal time for restarting anticoagulation. A randomized trial comparing discrete reinitiation times (eg, two weeks vs six weeks) may easily miss the optimal timing. Moreover, because the daily probability of thromboembolic events is low, large numbers of patients would be required to power such a study. In addition, a number of oral anticoagulants are now approved for prevention of thromboembolic stroke in atrial fibrillation, and each drug may have different optimal timing.

In contrast to randomized trials that would be impracticable for addressing this clinical issue, microsimulation modeling can provide granular information regarding the optimal time to restart anticoagulation. Herein, we set out to estimate the expected benefit of reinitiation of warfarin, the most commonly used oral anticoagulant,⁶ or apixaban, the direct oral anticoagulant with the most favorable risk profile,⁷ as a function of days after UGIB.

We previously described a microsimulation model of anticoagulation among patients with nonvalvular atrial fibrillation (NVAF; hereafter, we refer to this model as the Personalized Anticoagulation Decision-Making Assistance model, or PADMA).^8,9 For this study, we extended this model to incorporate the probability of rebleeding following UGIB and include apixaban as an alternative to warfarin. This model begins with a synthetic population following UGIB, the members of which are at varying risk for thromboembolism, recurrent UGIB, and other hemorrhages. For each patient, the model simulates a number of possible events (eg, thromboembolic stroke, intracranial hemorrhage, rebleeding, and other major extracranial hemorrhages) on each day of an acute period of 90 days after hemostasis. Patients who survive until the end of the acute period enter a simulation with annual, rather than daily, cycles. Our model then estimates total quality-adjusted life-years (QALYs) for each patient, discounted to the present. We report the average discounted QALYs produced by the model for the same population if all individuals in our input population were to resume either warfarin or apixaban on a specific day. Input parameters and ranges are summarized in Table 1, a simplified schematic of our model is shown in the Supplemental Appendix, and additional details regarding model structure and assumptions can be found in earlier work.^8,9 We simulated from a health system perspective over a lifelong time horizon. All analyses were performed in version 14 of Stata (StataCorp, LLC, College Station, Texas).

Synthetic Population

To generate a population reflective of the comorbidities and age distribution of the US population with NVAF, we merged relevant variables from the National Health and Nutrition Examination Survey (NHANES; 2011-2012), using multiple imputation to correct for missing variables.¹⁰ We then bootstrapped to national population estimates by age and sex to arrive at a hypothetical population of the United States.¹¹ Because NHANES does not include atrial fibrillation, we applied sex- and age-specific prevalence rates from the AnTicoagulation and Risk Factors In Atrial Fibrillation study.¹² We then calculated commonly used risk scores (CHA₂DS₂-Vasc and HAS-BLED) for each patient and limited the population to patients with a CHA₂DS₂-Vasc score of one or greater.^13,14 The population resuming apixaban was further limited to patients whose creatinine clearance was 25 mL/min or greater in keeping with the entry criteria in the phase 3 clinical trial on which the medication’s approval was based.¹⁵

To estimate patient-specific probability of rebleeding, we generated a Rockall score for each patient.¹⁶ Although the discrimination of the Rockall score is limited for individual patients, as with all other tools used to predict rebleeding following UGIB, the Rockall score has demonstrated reasonable calibration across a threefold risk gradient.^17-19 International consensus guidelines recommend the Rockall score as one of two risk prediction tools for clinical use in the management of patients with UGIB.²⁰ In addition, because the Rockall score includes some demographic components (five of a possible 11 points), our estimates of rebleeding risk are covariant with other patient-specific risks. We assumed that the endoscopic components of the Rockall score were present in our cohort at the same frequency as in the original derivation and are independent of known patient risk factors.¹⁶ For example, 441 out of 4,025 patients in the original Rockall derivation cohort presented with a systolic blood pressure less than 100 mm Hg. We assumed that an independent and random 10.96% of the cohort would present with shock, which confers two points in the Rockall score.

The population was replicated 60 times, with identical copies of the population resuming anticoagulation on each of days 1-60 (where day zero represents hemostasis). Intermediate data regarding our simulated population can be found in the Supplemental Appendix and in prior work.

Event Type, Severity, and Mortality

Each patient in our simulation could sustain several discrete and independent events: ischemic stroke, intracranial hemorrhage, recurrent UGIB, or extracranial major hemorrhage other than recurrent UGIB. As in prior analyses using the PADMA model, we did not consider minor hemorrhagic events.⁸

The probability of each event was conditional on the corresponding risk scoring system. Patient-specific probability of ischemic stroke was conditional on CHA₂DS₂-Vasc score.^21,22 Patient-specific probability of intracranial hemorrhage was conditional on HAS-BLED score, with the proportions of intracranial hemorrhage of each considered subtype (intracerebral, subarachnoid, or subdural) bootstrapped from previously-published data.^21-24 Patient-specific probability of rebleeding was conditional on Rockall score from the combined Rockall and Vreeburg validation cohorts.¹⁷ Patient-specific probability of extracranial major hemorrhage was conditional on HAS-BLED score.²¹ To avoid double-counting of UGIB, we subtracted the baseline risk of UGIB from the overall rate of extracranial major hemorrhages using previously-published data regarding relative frequency and a bootstrapping approach.²⁵

Probability of Rebleeding Over Time

To estimate the decrease in rebleeding risk over time, we searched the Medline database for systematic reviews of recurrent bleeding following UGIB using the strategy detailed in the Supplemental Appendix. Using the interval rates of rebleeding we identified, we calculated implied daily rates of rebleeding at the midpoint of each interval. For example, 39.5% of rebleeding events occurred within three days of hemostasis, implying a daily rate of approximately 13.2% on day two (32 of 81 events over a three-day period). We repeated this process to estimate daily rates at the midpoint of each reported time interval and fitted an exponential decay function.²⁶ Our exponential fitted these datapoints quite well, but we lacked sufficient data to test other survival functions (eg, Gompertz, lognormal, etc.). Our fitted exponential can be expressed as:

P _rebleeding = b ₀ *exp(b ₁ *day)

where b₀ = 0.1843 (SE: 0.0136) and b₁ = –0.1563 (SE: 0.0188). For example, a mean of 3.9% of rebleeding episodes will occur on day 10 (0.1843 *exp(–0.1563 *10)).

Relative Risks of Events with Anticoagulation

For patients resuming warfarin, the probabilities of each event were adjusted based on patient-specific daily INR. All INRs were assumed to be 1.0 until the day of warfarin reinitiation, after which interpolated trajectories of postinitiation INR measurements were sampled for each patient from an earlier study of clinical warfarin initiation.²⁷ Relative risks of ischemic stroke and hemorrhagic events were calculated based on each day’s INR.

For patients taking apixaban, we assumed that the medication would reach full therapeutic effect one day after reinitiation. Based on available evidence, we applied the relative risks of each event with apixaban compared with warfarin.²⁵

Future Disability and Mortality

Each event in our simulation resulted in hospitalization. Length of stay was sampled for each diagnosis.²⁸ The disutility of hospitalization was estimated based on length of stay.⁸ Inpatient mortality and future disability were predicted for each event as previously described.⁸ We assumed that recurrent episodes of UGIB conferred morbidity and mortality identical to extracranial major hemorrhages more broadly.^29,30

Disutilities

We used a multiplicative model for disutility with baseline utilities conditional on age and sex.³¹ Each day after resumption of anticoagulation carried a disutility of 0.012 for warfarin or 0.002 for apixaban, which we assumed to be equivalent to aspirin in disutility.³² Long-term disutility and life expectancy were conditional on modified Rankin Score (mRS).^33,34 We discounted all QALYs to day zero using standard exponential discounting and a discount rate centered at 3%. We then computed the average discounted QALYs among the cohort of patients that resumed anticoagulation on each day following the index UGIB.

Sensitivity Analyses and Metamodel

To assess sensitivity to continuously varying input parameters, such as discount rate, the proportion of extracranial major hemorrhages that are upper GI bleeds, and inpatient mortality from extracranial major hemorrhage, we constructed a metamodel (a regression model of our microsimulation results).³⁵ We tested for interactions among input parameters and dropped parameters that were not statistically significant predictors of discounted QALYs from our metamodel. We then tested for interactions between each parameter and day resuming anticoagulation to determine which factors may impact the optimal day of reinitiation. Finally, we used predicted marginal effects from our metamodel to assess the change in optimal day across the ranges of each input parameter when other parameters were held at their medians.

Resuming warfarin on day zero produced the fewest QALYs. With delay in reinitiation of anticoagulation, expected QALYs increased, peaked, and then declined for all scenarios. In our base-case simulation of warfarin, peak utility was achieved by resumption 41 days after the index UGIB. Resumption between days 32 and 51 produced greater than 99.9% of peak utility. In our base-case simulation of apixaban, peak utility was achieved by resumption 32 days after the index UGIB. Resumption between days 21 and 47 produced greater than 99.9% of peak utility. Results for warfarin and apixaban are shown in Figures 1 and 2, respectively.

The optimal day of warfarin reinitiation was most sensitive to CHA₂DS₂-Vasc scores and varied by around 11 days between a CHA₂DS₂-Vasc score of one and a CHA₂DS₂-Vasc score of six (the 5^th and 95^th percentiles, respectively) when all other parameters are held at their medians. Results were comparatively insensitive to rebleeding risk. Varying Rockall score from two to seven (the 5^th and 95^th percentiles, respectively) added three days to optimal warfarin resumption. Varying other parameters from the 5^th to the 95^th percentile (including HAS-BLED score, sex, age, and discount rate) changed expected QALYs but did not change the optimal day of reinitiation of warfarin. Optimal day of reinitiation for warfarin stratified by CHA₂DS₂-Vasc score is shown in Table 2.

Anticoagulation is frequently prescribed for patients with NVAF, and hemorrhagic complications are common. Although anticoagulants are withheld following hemorrhages, scant evidence to inform the optimal timing of reinitiation is available. In this microsimulation analysis, we found that the optimal time to reinitiate anticoagulation following UGIB is around 41 days for warfarin and around 32 days for apixaban. We have further demonstrated that the optimal timing of reinitiation can vary by nearly two weeks, depending on a patient’s underlying risk of stroke, and that early reinitiation is more sensitive to rebleeding risk than late reinitiation.

Prior work has shown that early reinitiation of anticoagulation leads to higher rates of recurrent hemorrhage while failure to reinitiate anticoagulation is associated with higher rates of stroke and mortality.^1-4,36 Our results add to the literature in a number of important ways. First, our model not only confirms that anticoagulation should be restarted but also suggests when this action should be taken. The competing risks of bleeding and stroke have left clinicians with little guidance; we have quantified the clinical reasoning required for the decision to resume anticoagulation. Second, by including the disutility of hospitalization and long-term disability, our model more accurately represents the complex tradeoffs between recurrent hemorrhage and (potentially disabling) stroke than would a comparison of event rates. Third, our model is conditional upon patient risk factors, allowing clinicians to personalize the timing of anticoagulation resumption. Theory would suggest that patients at higher risk of ischemic stroke benefit from earlier resumption of anticoagulation, while patients at higher risk of hemorrhage benefit from delayed reinitiation. We have quantified the extent to which patient-specific risks should change timing. Fourth, we offer a means of improving expected health outcomes that requires little more than appropriate scheduling. Current practice regarding resuming anticoagulation is widely variable. Many patients never resume warfarin, and those that do resume do so after highly varied periods of time.^1-5,36 We offer a means of standardizing clinical practice and improving expected patient outcomes.

Disclosures

The authors have nothing to disclose.

Funding

The authors received no specific funding for this work.

Historically, academic medicine faculty were predominantly physician-scientists.¹ During the past decade, the number of clinician-educators and nontenured clinicians has grown.² Many academically oriented clinical faculty at our institution would like to participate in and learn how to conduct quality scholarship. While institutional requirements vary, scholarly work is often required for promotion,³ and faculty may also desire to support the scholarly work of residents. Moreover, a core program component of the Accreditation Council of Graduate Medical Education standards requires faculty to “maintain an environment of inquiry and scholarship with an active research component.”⁴ Yet clinical faculty often find academic projects to be challenging. Similar to residents, clinical academic faculty frequently lack formal training in health services research or quality improvement science, have insufficient mentorship, and typically have limited uncommitted time and resources.⁵

One approach to this problem has been to pair junior clinicians with traditional physician scientists as mentors.^6,7 This type of mentorship for clinical faculty is increasingly difficult to access because of growing pressure on physician-scientist faculty to conduct their own research, seek extramural funding, meet clinical expectations, and mentor fellows and faculty in their own disciplines.⁸ Moreover, senior research faculty may not be prepared or have the time to teach junior faculty how to deal with common stumbling blocks (eg, institutional review board [IRB] applications, statistically testable hypothesis development, and statistical analysis).^8,9 Seminars or works-in-progress sessions are another strategy to bolster scholarly work, but the experience at our institution is that such sessions are often not relevant at the time of delivery and can be intimidating to clinical faculty who lack extensive knowledge about research methods and prior research experience.

Another approach to supporting the research efforts of academic clinicians is to fund a consulting statistician. However, without sufficient content expertise, statisticians may be frustrated in their efforts to assist clinicians who struggle to formulate a testable question or to work directly with data collected. Statisticians may be inexperienced in writing IRB applications or implementing protocols in a clinical or educational setting. Furthermore, statistical consultations are often limited in scope¹⁰ and, in our setting, rarely produce a durable improvement in the research skills of the faculty member or the enduring partnership required to complete a longer-term project. Because of these shortcomings, we have found that purely statistical support resources are often underutilized and ineffective.

Other models to facilitate scholarship have been employed, but few focus on facilitating scholarship of clinical faculty. One strategy involved supporting hospitalist’s academic productivity by reducing hospitalists’ full-time equivalent (FTE) and providing mentorship.¹¹ For many, this approach is likely cost-prohibitive. Others have focused primarily on resident and fellow scholarships.^5,6

In this report, we describe an educational innovation to educate and support the scholarly work of academic hospitalists and internists by using an academic research coach. We recruited a health researcher with extensive experience in research methods and strong interpersonal skills with the ability to explain and teach research concepts in an accessible manner. We sought an individual who would provide high-yield single consultations, join project teams to provide ongoing mentorship from conception to completion, and consequently, bolster scholarly productivity and learning among nonresearch clinicians in our Division. We anticipated that providing support for multiple aspects of a project would be more likely to help faculty overcome barriers to research and disseminate their project results as scholarly output.

The coach initiative was implemented in the Division of General Internal Medicine at the University of Washington. The Division has over 200 members (60 hospitalists), including clinical instructors and acting instructors, who have not yet been appointed to the regular faculty (clinician-educators and physician scientists), and full-time clinical faculty. Division members staff clinical services at four area hospitals and 10 affiliated internal medicine and specialty clinics. Eligible clients were all Division members, although the focus of the initial program targeted hospitalists at our three primary teaching hospitals. Fellows, residents, students, and faculty from within and outside the Division were welcome to participate in a project involving coaching as long as a Division faculty member was engaged in the project.

Program Description

The overall goal of the coach initiative was to support the scholarly work of primarily clinical Division members. Given our focus was on clinical faculty with little training on research methodology, we did not expect the coach to secure grant funding for the position. Instead, we aimed to increase the quality and quantity of scholarship through publications, abstracts, and small grants. We defined scholarly work broadly: clinical research, quality improvement, medical education research, and other forms of scientific inquiry or synthesis. The coach was established as a 0.50 FTE position with a 12-month annually renewable appointment. The role was deemed that of a coach instead of a mentor because the coach was available to all Division members and involved task-oriented consultations with check-ins to facilitate projects, rather than a deeper more developmental relationship that typically exists with mentoring. The Division leadership identified support for scholarly activity as a high priority and mentorship as an unmet need based on faculty feedback. Clinical revenue supported the position.

Necessary qualifications, determined prior to hiring, included a PhD in health services or related field (eg, epidemiology) or a master’s degree with five years of experience in project management, clinical research, and study design. The position also called for expertise in articulating research questions, selecting study designs, navigating the IRB approval process, collecting/managing data, analyzing statistics, and mentoring and teaching clinical faculty in their scholarly endeavors. A track record in generating academic output (manuscripts and abstracts at regional/national meetings) was required. We circulated a description of the position to Division faculty and to leadership in our School of Public Health.

Based on these criteria, an inaugural coach was hired (author C.M.M.). The coach had a PhD in epidemiology, 10 years of research experience, 16 publications, and had recently finished a National Institutes of Health (NIH) career development award. At the time of hiring, she was a Clinical Assistant Professor in the School of Dentistry, which provided additional FTE. She had no extramural funding but was applying for NIH-level grants and had received several small grants.

To ensure uptake of the coach’s services, we realized that it was necessary to delineate the scope of services available, clarify availability of the coach, and define expectations regarding authorship. We used an iterative process that took into consideration the coach’s expertise, services most needed by the Division’s clinicians, and discussions with Division leadership and faculty at faculty meetings across hospitals and clinics. A range of services and authorship expectations were defined. Consensus was reached that the coach should be invited to coauthor projects where design, analysis, and/or substantial intellectual content was provided and for which authorship criteria were met.¹² Collegial reviews by the coach of already developed manuscripts and time-limited, low-intensity consultations that did not involve substantial intellectual contributions did not warrant authorship.¹² On this basis, we created and distributed a flyer to publicize these guidelines and invite Division members to contact the coach (Figure 1).

The coach attended Division, section, and clinical group meetings to publicize the initiative. The coach also individually met with faculty throughout the Division, explained her role, described services available, and answered questions. The marketing effort was continuous and calibrated with more or less exposure depending on existing projects and the coach’s availability. In addition, the coach coordinated with the director of the Division’s faculty development program to cohost works-in-progress seminars, identify coach clients to present at these meetings, and provide brief presentations on a basic research skill at meetings. Faculty built rapport with the coach through these activities and became more comfortable reaching out for assistance. Because of the large size of the Division, it was decided to roll out the initiative in a stepwise fashion, starting with hospitalists before expanding to the rest of the Division.

Most faculty contacted the coach by e-mail to request a consultation, at which time the coach requested that they complete a preconsultation handout (Figure 2). Initial coaching appointments lasted one hour and were in-person. Coaching entailed an in-depth analysis of the project plan and advice on how to move the project forward. The coach provided tailored scholarly project advice and expertise in research methods. After initial consultations, she would review grant proposals, IRB applications, manuscripts, case report forms, abstracts, and other products. Her efforts typically focused on improving the methods and scientific and technical writing. Assistance with statistical analysis was provided on a case-by-case basis to maintain broad availability. To address statistically complex questions, the coach had five hours of monthly access to a PhD biostatistician via an on-campus consulting service. Follow-up appointments were encouraged and provided as needed by e-mail, phone, or in-person. The coach conducted regular reach outs to facilitate projects. However, execution of the research was generally the responsibility of the faculty member.

Program Evaluation

To characterize the reach and scope of the program, the coach tracked the number of faculty supported, types of services provided, status of initiated projects, numbers of grants generated, and the dissemination of scholarly products including papers and abstracts. We used these metrics to create summary reports to identify successes and areas for improvement. Monthly meetings between the coach and Division leadership were used to fine-tune the approach.

We surveyed coach clients anonymously to assess their satisfaction with the coach initiative. Using Likert scale questions where 1 = completely disagree and 5 = completely agree, we asked (1) if they would recommend the coach to colleagues, (2) if their work was higher quality because of the coach, (3) if they were overall satisfied with the coach, (4) whether the Division should continue to support the coach, and (5) if the coach’s lack of clinical training negatively affected their experience. This work was considered a quality improvement initiative for which IRB approval was not required.

Over 18 months, the coach supported a 49 Division members including 30 hospitalists and 63 projects. Projects included a wide range of scholarship: medical education research, qualitative research, clinical quality improvement projects, observational studies, and a randomized clinical trial. Many clients (n = 16) used the coach for more than one project. The scope of work included limited support projects (identifying research resource and brainstorming project feasibility) lasting one to two sessions (n = 25), projects with a limited scope (collegial reviews of manuscripts and assistance with IRB submissions) but requiring more than two consultations (n = 24), and ongoing in-depth support projects (contributions on design, data collection, analysis, and manuscript writing) that required three consultations or more (n = 14). The majority of Division members (75%) supported did not have master’s level training in a health services-related area, six had NIH or other national-level funding, and two had small grants funded by local sources prior to providing support. The number of Division faculty on a given project ranged from one to four.

The coach directly supported 13 manuscripts with coach authorship, seven manuscripts without authorship, 11 abstracts, and four grant submissions (Appendix). The coach was a coauthor on all the abstracts and a coinvestigator on the grant applications. Of the 13 publications the coach coauthored, 11 publications have been accepted to peer-reviewed journals and two are currently in the submission process. The types of articles published included one medical evaluation report, one qualitative study, one randomized clinical trial, three quality assessment/improvement reports, and five epidemiologic studies. The types of abstracts included one qualitative report, one systematic review, one randomized clinical trial, two quality improvement projects, two epidemiologic studies, and four medical education projects. Three of four small grants submitted to local and national funders were funded.

The coach’s influence extended beyond the Division. Forty-eight university faculty, fellows, or students not affiliated with general internal medicine benefited from coach coaching: 26 were authors on papers and/or abstracts coauthored by the coach, 17 on manuscripts the coach reviewed without authorship, and five participated in consultations.

The coach found the experience rewarding. She enjoyed working on the methodologic aspects of projects and benefited from being included as coauthor on papers.

Twenty-nine of the 43 faculty (67%) still at the institution responded to the program assessment survey. Faculty strongly agreed that they would recommend the coach to colleagues (average ± standard deviation [SD]: 4.7 ± 0.5), that it improved the quality of their work (4.5 ± 0.9), that they were overall satisfied with the coaching (4.6 ± 0.7), and that the Division should continue to support the coach (4.9 ± 0.4). Faculty did not agree that the lack of clinical training of the coach was a barrier (2.0 ± 1.3).

The coach program was highly utilized, well regarded, and delivered substantial, tangible, and academic output. We anticipate the coach initiative will continue to be a valuable resource for our Division and could prove to be a valuable model for other institutions seeking to bolster the scholarly work of clinical academicians.

Several lessons emerged through the course of this project. First, we realized it is essential to select a coach who is both knowledgeable and approachable. We found that after meeting the coach, many faculty sought her help who otherwise would not have. An explicit, ongoing marketing strategy with regular contact with faculty at meetings was a key to receiving consult requests.

Second, the lack of a clinical background did not seem to hinder the coach’s ability to coach clinicians. The coach acknowledged her lack of clinical experience and relied on clients to explain the clinical context of projects. We also learned that the coach’s substantial experience with the logistics of research was invaluable. For example, the coach had substantial experience with the IRB process and her pre-reviews of IRB applications made for a short and relatively seamless experience navigating the IRB process. The coach also facilitated collaborations and leveraged existing resources at our institution. For example, for a qualitative research project, the coach helped identify a health services faculty member with this specific expertise, which led to a successful collaboration and publication. Although a more junior coach with less established qualifications may be helpful with research methods and with the research process, our endeavor suggests that having a more highly trained and experienced researcher was extremely valuable. Finally, we learned that for a Division of our size, the 0.50 FTE allotted to the coach is a minimum requirement. The coach spent approximately four hours a week on marketing, attending faculty meetings and conducting brief didactics, two hours per week on administration, and 14 hours per week on consultations. Faculty generally received support soon after their requests, but there were occasional wait times, which may have delayed some projects.

Academic leaders at our institution have noted the success of our coach initiative and have created a demand for coach services. We are exploring funding models that would allow for the expansion of coach services to other departments and divisions. We are in the initial stages of creating an Academic Scholarship Support Core under the supervision of the coach. Within this Core, we envision that various research support services will be triaged to staff with appropriate expertise; for example, a regulatory coordinator would review IRB applications while a master’s level statistician would conduct statistical analyses.

We have also transitioned to a new coach and have continued to experience success with the program. Our initial coach (author C.M.M.) obtained an NIH R01, a foundation grant, and took over a summer program that trains dental faculty in clinical research methods leaving insufficient time for coaching. Our new coach also has a PhD in epidemiology with NIH R01 funding but has more available FTE. Both of our coaches are graduates of our School of Public Health and institutions with such schools may have good access to the expertise needed. Nonclinical PhDs are often almost entirely reliant on grants, and some nongrant support is often attractive to these researchers. Additionally, PhDs who are junior or mid-career faculty that have the needed training are relatively affordable, particularly when the resource is made available to large number of faculty. For example, our first coach cost $48,000 a year for 50% FTE.

A limitation to our assessment of the coach initiative was the lack of pre- and postintervention metrics of scholarly productivity. We cannot definitively say that the Division’s scholarly output has increased because of the coach. Nevertheless, we are confident that the coach’s coaching has enhanced the scholarly work of individual clinicians and provided value to the Division as a whole. The coach program has been a success in our Division. Other institutions facing the challenge of supporting the research efforts of academic clinicians may consider this model as a worthy investment.

Disclosures

The authors have nothing to disclose.

Historically, academic medicine faculty were predominantly physician-scientists.¹ During the past decade, the number of clinician-educators and nontenured clinicians has grown.² Many academically oriented clinical faculty at our institution would like to participate in and learn how to conduct quality scholarship. While institutional requirements vary, scholarly work is often required for promotion,³ and faculty may also desire to support the scholarly work of residents. Moreover, a core program component of the Accreditation Council of Graduate Medical Education standards requires faculty to “maintain an environment of inquiry and scholarship with an active research component.”⁴ Yet clinical faculty often find academic projects to be challenging. Similar to residents, clinical academic faculty frequently lack formal training in health services research or quality improvement science, have insufficient mentorship, and typically have limited uncommitted time and resources.⁵

One approach to this problem has been to pair junior clinicians with traditional physician scientists as mentors.^6,7 This type of mentorship for clinical faculty is increasingly difficult to access because of growing pressure on physician-scientist faculty to conduct their own research, seek extramural funding, meet clinical expectations, and mentor fellows and faculty in their own disciplines.⁸ Moreover, senior research faculty may not be prepared or have the time to teach junior faculty how to deal with common stumbling blocks (eg, institutional review board [IRB] applications, statistically testable hypothesis development, and statistical analysis).^8,9 Seminars or works-in-progress sessions are another strategy to bolster scholarly work, but the experience at our institution is that such sessions are often not relevant at the time of delivery and can be intimidating to clinical faculty who lack extensive knowledge about research methods and prior research experience.

Another approach to supporting the research efforts of academic clinicians is to fund a consulting statistician. However, without sufficient content expertise, statisticians may be frustrated in their efforts to assist clinicians who struggle to formulate a testable question or to work directly with data collected. Statisticians may be inexperienced in writing IRB applications or implementing protocols in a clinical or educational setting. Furthermore, statistical consultations are often limited in scope¹⁰ and, in our setting, rarely produce a durable improvement in the research skills of the faculty member or the enduring partnership required to complete a longer-term project. Because of these shortcomings, we have found that purely statistical support resources are often underutilized and ineffective.

Other models to facilitate scholarship have been employed, but few focus on facilitating scholarship of clinical faculty. One strategy involved supporting hospitalist’s academic productivity by reducing hospitalists’ full-time equivalent (FTE) and providing mentorship.¹¹ For many, this approach is likely cost-prohibitive. Others have focused primarily on resident and fellow scholarships.^5,6

In this report, we describe an educational innovation to educate and support the scholarly work of academic hospitalists and internists by using an academic research coach. We recruited a health researcher with extensive experience in research methods and strong interpersonal skills with the ability to explain and teach research concepts in an accessible manner. We sought an individual who would provide high-yield single consultations, join project teams to provide ongoing mentorship from conception to completion, and consequently, bolster scholarly productivity and learning among nonresearch clinicians in our Division. We anticipated that providing support for multiple aspects of a project would be more likely to help faculty overcome barriers to research and disseminate their project results as scholarly output.

The coach initiative was implemented in the Division of General Internal Medicine at the University of Washington. The Division has over 200 members (60 hospitalists), including clinical instructors and acting instructors, who have not yet been appointed to the regular faculty (clinician-educators and physician scientists), and full-time clinical faculty. Division members staff clinical services at four area hospitals and 10 affiliated internal medicine and specialty clinics. Eligible clients were all Division members, although the focus of the initial program targeted hospitalists at our three primary teaching hospitals. Fellows, residents, students, and faculty from within and outside the Division were welcome to participate in a project involving coaching as long as a Division faculty member was engaged in the project.

Program Description

The overall goal of the coach initiative was to support the scholarly work of primarily clinical Division members. Given our focus was on clinical faculty with little training on research methodology, we did not expect the coach to secure grant funding for the position. Instead, we aimed to increase the quality and quantity of scholarship through publications, abstracts, and small grants. We defined scholarly work broadly: clinical research, quality improvement, medical education research, and other forms of scientific inquiry or synthesis. The coach was established as a 0.50 FTE position with a 12-month annually renewable appointment. The role was deemed that of a coach instead of a mentor because the coach was available to all Division members and involved task-oriented consultations with check-ins to facilitate projects, rather than a deeper more developmental relationship that typically exists with mentoring. The Division leadership identified support for scholarly activity as a high priority and mentorship as an unmet need based on faculty feedback. Clinical revenue supported the position.

Necessary qualifications, determined prior to hiring, included a PhD in health services or related field (eg, epidemiology) or a master’s degree with five years of experience in project management, clinical research, and study design. The position also called for expertise in articulating research questions, selecting study designs, navigating the IRB approval process, collecting/managing data, analyzing statistics, and mentoring and teaching clinical faculty in their scholarly endeavors. A track record in generating academic output (manuscripts and abstracts at regional/national meetings) was required. We circulated a description of the position to Division faculty and to leadership in our School of Public Health.

Based on these criteria, an inaugural coach was hired (author C.M.M.). The coach had a PhD in epidemiology, 10 years of research experience, 16 publications, and had recently finished a National Institutes of Health (NIH) career development award. At the time of hiring, she was a Clinical Assistant Professor in the School of Dentistry, which provided additional FTE. She had no extramural funding but was applying for NIH-level grants and had received several small grants.

To ensure uptake of the coach’s services, we realized that it was necessary to delineate the scope of services available, clarify availability of the coach, and define expectations regarding authorship. We used an iterative process that took into consideration the coach’s expertise, services most needed by the Division’s clinicians, and discussions with Division leadership and faculty at faculty meetings across hospitals and clinics. A range of services and authorship expectations were defined. Consensus was reached that the coach should be invited to coauthor projects where design, analysis, and/or substantial intellectual content was provided and for which authorship criteria were met.¹² Collegial reviews by the coach of already developed manuscripts and time-limited, low-intensity consultations that did not involve substantial intellectual contributions did not warrant authorship.¹² On this basis, we created and distributed a flyer to publicize these guidelines and invite Division members to contact the coach (Figure 1).

The coach attended Division, section, and clinical group meetings to publicize the initiative. The coach also individually met with faculty throughout the Division, explained her role, described services available, and answered questions. The marketing effort was continuous and calibrated with more or less exposure depending on existing projects and the coach’s availability. In addition, the coach coordinated with the director of the Division’s faculty development program to cohost works-in-progress seminars, identify coach clients to present at these meetings, and provide brief presentations on a basic research skill at meetings. Faculty built rapport with the coach through these activities and became more comfortable reaching out for assistance. Because of the large size of the Division, it was decided to roll out the initiative in a stepwise fashion, starting with hospitalists before expanding to the rest of the Division.

Most faculty contacted the coach by e-mail to request a consultation, at which time the coach requested that they complete a preconsultation handout (Figure 2). Initial coaching appointments lasted one hour and were in-person. Coaching entailed an in-depth analysis of the project plan and advice on how to move the project forward. The coach provided tailored scholarly project advice and expertise in research methods. After initial consultations, she would review grant proposals, IRB applications, manuscripts, case report forms, abstracts, and other products. Her efforts typically focused on improving the methods and scientific and technical writing. Assistance with statistical analysis was provided on a case-by-case basis to maintain broad availability. To address statistically complex questions, the coach had five hours of monthly access to a PhD biostatistician via an on-campus consulting service. Follow-up appointments were encouraged and provided as needed by e-mail, phone, or in-person. The coach conducted regular reach outs to facilitate projects. However, execution of the research was generally the responsibility of the faculty member.

Program Evaluation

To characterize the reach and scope of the program, the coach tracked the number of faculty supported, types of services provided, status of initiated projects, numbers of grants generated, and the dissemination of scholarly products including papers and abstracts. We used these metrics to create summary reports to identify successes and areas for improvement. Monthly meetings between the coach and Division leadership were used to fine-tune the approach.

We surveyed coach clients anonymously to assess their satisfaction with the coach initiative. Using Likert scale questions where 1 = completely disagree and 5 = completely agree, we asked (1) if they would recommend the coach to colleagues, (2) if their work was higher quality because of the coach, (3) if they were overall satisfied with the coach, (4) whether the Division should continue to support the coach, and (5) if the coach’s lack of clinical training negatively affected their experience. This work was considered a quality improvement initiative for which IRB approval was not required.

Over 18 months, the coach supported a 49 Division members including 30 hospitalists and 63 projects. Projects included a wide range of scholarship: medical education research, qualitative research, clinical quality improvement projects, observational studies, and a randomized clinical trial. Many clients (n = 16) used the coach for more than one project. The scope of work included limited support projects (identifying research resource and brainstorming project feasibility) lasting one to two sessions (n = 25), projects with a limited scope (collegial reviews of manuscripts and assistance with IRB submissions) but requiring more than two consultations (n = 24), and ongoing in-depth support projects (contributions on design, data collection, analysis, and manuscript writing) that required three consultations or more (n = 14). The majority of Division members (75%) supported did not have master’s level training in a health services-related area, six had NIH or other national-level funding, and two had small grants funded by local sources prior to providing support. The number of Division faculty on a given project ranged from one to four.

The coach directly supported 13 manuscripts with coach authorship, seven manuscripts without authorship, 11 abstracts, and four grant submissions (Appendix). The coach was a coauthor on all the abstracts and a coinvestigator on the grant applications. Of the 13 publications the coach coauthored, 11 publications have been accepted to peer-reviewed journals and two are currently in the submission process. The types of articles published included one medical evaluation report, one qualitative study, one randomized clinical trial, three quality assessment/improvement reports, and five epidemiologic studies. The types of abstracts included one qualitative report, one systematic review, one randomized clinical trial, two quality improvement projects, two epidemiologic studies, and four medical education projects. Three of four small grants submitted to local and national funders were funded.

The coach’s influence extended beyond the Division. Forty-eight university faculty, fellows, or students not affiliated with general internal medicine benefited from coach coaching: 26 were authors on papers and/or abstracts coauthored by the coach, 17 on manuscripts the coach reviewed without authorship, and five participated in consultations.

The coach found the experience rewarding. She enjoyed working on the methodologic aspects of projects and benefited from being included as coauthor on papers.

Twenty-nine of the 43 faculty (67%) still at the institution responded to the program assessment survey. Faculty strongly agreed that they would recommend the coach to colleagues (average ± standard deviation [SD]: 4.7 ± 0.5), that it improved the quality of their work (4.5 ± 0.9), that they were overall satisfied with the coaching (4.6 ± 0.7), and that the Division should continue to support the coach (4.9 ± 0.4). Faculty did not agree that the lack of clinical training of the coach was a barrier (2.0 ± 1.3).

The coach program was highly utilized, well regarded, and delivered substantial, tangible, and academic output. We anticipate the coach initiative will continue to be a valuable resource for our Division and could prove to be a valuable model for other institutions seeking to bolster the scholarly work of clinical academicians.

Several lessons emerged through the course of this project. First, we realized it is essential to select a coach who is both knowledgeable and approachable. We found that after meeting the coach, many faculty sought her help who otherwise would not have. An explicit, ongoing marketing strategy with regular contact with faculty at meetings was a key to receiving consult requests.

Second, the lack of a clinical background did not seem to hinder the coach’s ability to coach clinicians. The coach acknowledged her lack of clinical experience and relied on clients to explain the clinical context of projects. We also learned that the coach’s substantial experience with the logistics of research was invaluable. For example, the coach had substantial experience with the IRB process and her pre-reviews of IRB applications made for a short and relatively seamless experience navigating the IRB process. The coach also facilitated collaborations and leveraged existing resources at our institution. For example, for a qualitative research project, the coach helped identify a health services faculty member with this specific expertise, which led to a successful collaboration and publication. Although a more junior coach with less established qualifications may be helpful with research methods and with the research process, our endeavor suggests that having a more highly trained and experienced researcher was extremely valuable. Finally, we learned that for a Division of our size, the 0.50 FTE allotted to the coach is a minimum requirement. The coach spent approximately four hours a week on marketing, attending faculty meetings and conducting brief didactics, two hours per week on administration, and 14 hours per week on consultations. Faculty generally received support soon after their requests, but there were occasional wait times, which may have delayed some projects.

Academic leaders at our institution have noted the success of our coach initiative and have created a demand for coach services. We are exploring funding models that would allow for the expansion of coach services to other departments and divisions. We are in the initial stages of creating an Academic Scholarship Support Core under the supervision of the coach. Within this Core, we envision that various research support services will be triaged to staff with appropriate expertise; for example, a regulatory coordinator would review IRB applications while a master’s level statistician would conduct statistical analyses.

We have also transitioned to a new coach and have continued to experience success with the program. Our initial coach (author C.M.M.) obtained an NIH R01, a foundation grant, and took over a summer program that trains dental faculty in clinical research methods leaving insufficient time for coaching. Our new coach also has a PhD in epidemiology with NIH R01 funding but has more available FTE. Both of our coaches are graduates of our School of Public Health and institutions with such schools may have good access to the expertise needed. Nonclinical PhDs are often almost entirely reliant on grants, and some nongrant support is often attractive to these researchers. Additionally, PhDs who are junior or mid-career faculty that have the needed training are relatively affordable, particularly when the resource is made available to large number of faculty. For example, our first coach cost $48,000 a year for 50% FTE.

A limitation to our assessment of the coach initiative was the lack of pre- and postintervention metrics of scholarly productivity. We cannot definitively say that the Division’s scholarly output has increased because of the coach. Nevertheless, we are confident that the coach’s coaching has enhanced the scholarly work of individual clinicians and provided value to the Division as a whole. The coach program has been a success in our Division. Other institutions facing the challenge of supporting the research efforts of academic clinicians may consider this model as a worthy investment.

Disclosures

The authors have nothing to disclose.

Historically, academic medicine faculty were predominantly physician-scientists.¹ During the past decade, the number of clinician-educators and nontenured clinicians has grown.² Many academically oriented clinical faculty at our institution would like to participate in and learn how to conduct quality scholarship. While institutional requirements vary, scholarly work is often required for promotion,³ and faculty may also desire to support the scholarly work of residents. Moreover, a core program component of the Accreditation Council of Graduate Medical Education standards requires faculty to “maintain an environment of inquiry and scholarship with an active research component.”⁴ Yet clinical faculty often find academic projects to be challenging. Similar to residents, clinical academic faculty frequently lack formal training in health services research or quality improvement science, have insufficient mentorship, and typically have limited uncommitted time and resources.⁵

One approach to this problem has been to pair junior clinicians with traditional physician scientists as mentors.^6,7 This type of mentorship for clinical faculty is increasingly difficult to access because of growing pressure on physician-scientist faculty to conduct their own research, seek extramural funding, meet clinical expectations, and mentor fellows and faculty in their own disciplines.⁸ Moreover, senior research faculty may not be prepared or have the time to teach junior faculty how to deal with common stumbling blocks (eg, institutional review board [IRB] applications, statistically testable hypothesis development, and statistical analysis).^8,9 Seminars or works-in-progress sessions are another strategy to bolster scholarly work, but the experience at our institution is that such sessions are often not relevant at the time of delivery and can be intimidating to clinical faculty who lack extensive knowledge about research methods and prior research experience.

Another approach to supporting the research efforts of academic clinicians is to fund a consulting statistician. However, without sufficient content expertise, statisticians may be frustrated in their efforts to assist clinicians who struggle to formulate a testable question or to work directly with data collected. Statisticians may be inexperienced in writing IRB applications or implementing protocols in a clinical or educational setting. Furthermore, statistical consultations are often limited in scope¹⁰ and, in our setting, rarely produce a durable improvement in the research skills of the faculty member or the enduring partnership required to complete a longer-term project. Because of these shortcomings, we have found that purely statistical support resources are often underutilized and ineffective.

Other models to facilitate scholarship have been employed, but few focus on facilitating scholarship of clinical faculty. One strategy involved supporting hospitalist’s academic productivity by reducing hospitalists’ full-time equivalent (FTE) and providing mentorship.¹¹ For many, this approach is likely cost-prohibitive. Others have focused primarily on resident and fellow scholarships.^5,6

In this report, we describe an educational innovation to educate and support the scholarly work of academic hospitalists and internists by using an academic research coach. We recruited a health researcher with extensive experience in research methods and strong interpersonal skills with the ability to explain and teach research concepts in an accessible manner. We sought an individual who would provide high-yield single consultations, join project teams to provide ongoing mentorship from conception to completion, and consequently, bolster scholarly productivity and learning among nonresearch clinicians in our Division. We anticipated that providing support for multiple aspects of a project would be more likely to help faculty overcome barriers to research and disseminate their project results as scholarly output.

The coach initiative was implemented in the Division of General Internal Medicine at the University of Washington. The Division has over 200 members (60 hospitalists), including clinical instructors and acting instructors, who have not yet been appointed to the regular faculty (clinician-educators and physician scientists), and full-time clinical faculty. Division members staff clinical services at four area hospitals and 10 affiliated internal medicine and specialty clinics. Eligible clients were all Division members, although the focus of the initial program targeted hospitalists at our three primary teaching hospitals. Fellows, residents, students, and faculty from within and outside the Division were welcome to participate in a project involving coaching as long as a Division faculty member was engaged in the project.

Program Description

The overall goal of the coach initiative was to support the scholarly work of primarily clinical Division members. Given our focus was on clinical faculty with little training on research methodology, we did not expect the coach to secure grant funding for the position. Instead, we aimed to increase the quality and quantity of scholarship through publications, abstracts, and small grants. We defined scholarly work broadly: clinical research, quality improvement, medical education research, and other forms of scientific inquiry or synthesis. The coach was established as a 0.50 FTE position with a 12-month annually renewable appointment. The role was deemed that of a coach instead of a mentor because the coach was available to all Division members and involved task-oriented consultations with check-ins to facilitate projects, rather than a deeper more developmental relationship that typically exists with mentoring. The Division leadership identified support for scholarly activity as a high priority and mentorship as an unmet need based on faculty feedback. Clinical revenue supported the position.

Necessary qualifications, determined prior to hiring, included a PhD in health services or related field (eg, epidemiology) or a master’s degree with five years of experience in project management, clinical research, and study design. The position also called for expertise in articulating research questions, selecting study designs, navigating the IRB approval process, collecting/managing data, analyzing statistics, and mentoring and teaching clinical faculty in their scholarly endeavors. A track record in generating academic output (manuscripts and abstracts at regional/national meetings) was required. We circulated a description of the position to Division faculty and to leadership in our School of Public Health.

Based on these criteria, an inaugural coach was hired (author C.M.M.). The coach had a PhD in epidemiology, 10 years of research experience, 16 publications, and had recently finished a National Institutes of Health (NIH) career development award. At the time of hiring, she was a Clinical Assistant Professor in the School of Dentistry, which provided additional FTE. She had no extramural funding but was applying for NIH-level grants and had received several small grants.

To ensure uptake of the coach’s services, we realized that it was necessary to delineate the scope of services available, clarify availability of the coach, and define expectations regarding authorship. We used an iterative process that took into consideration the coach’s expertise, services most needed by the Division’s clinicians, and discussions with Division leadership and faculty at faculty meetings across hospitals and clinics. A range of services and authorship expectations were defined. Consensus was reached that the coach should be invited to coauthor projects where design, analysis, and/or substantial intellectual content was provided and for which authorship criteria were met.¹² Collegial reviews by the coach of already developed manuscripts and time-limited, low-intensity consultations that did not involve substantial intellectual contributions did not warrant authorship.¹² On this basis, we created and distributed a flyer to publicize these guidelines and invite Division members to contact the coach (Figure 1).

The coach attended Division, section, and clinical group meetings to publicize the initiative. The coach also individually met with faculty throughout the Division, explained her role, described services available, and answered questions. The marketing effort was continuous and calibrated with more or less exposure depending on existing projects and the coach’s availability. In addition, the coach coordinated with the director of the Division’s faculty development program to cohost works-in-progress seminars, identify coach clients to present at these meetings, and provide brief presentations on a basic research skill at meetings. Faculty built rapport with the coach through these activities and became more comfortable reaching out for assistance. Because of the large size of the Division, it was decided to roll out the initiative in a stepwise fashion, starting with hospitalists before expanding to the rest of the Division.

Most faculty contacted the coach by e-mail to request a consultation, at which time the coach requested that they complete a preconsultation handout (Figure 2). Initial coaching appointments lasted one hour and were in-person. Coaching entailed an in-depth analysis of the project plan and advice on how to move the project forward. The coach provided tailored scholarly project advice and expertise in research methods. After initial consultations, she would review grant proposals, IRB applications, manuscripts, case report forms, abstracts, and other products. Her efforts typically focused on improving the methods and scientific and technical writing. Assistance with statistical analysis was provided on a case-by-case basis to maintain broad availability. To address statistically complex questions, the coach had five hours of monthly access to a PhD biostatistician via an on-campus consulting service. Follow-up appointments were encouraged and provided as needed by e-mail, phone, or in-person. The coach conducted regular reach outs to facilitate projects. However, execution of the research was generally the responsibility of the faculty member.

Program Evaluation

To characterize the reach and scope of the program, the coach tracked the number of faculty supported, types of services provided, status of initiated projects, numbers of grants generated, and the dissemination of scholarly products including papers and abstracts. We used these metrics to create summary reports to identify successes and areas for improvement. Monthly meetings between the coach and Division leadership were used to fine-tune the approach.

We surveyed coach clients anonymously to assess their satisfaction with the coach initiative. Using Likert scale questions where 1 = completely disagree and 5 = completely agree, we asked (1) if they would recommend the coach to colleagues, (2) if their work was higher quality because of the coach, (3) if they were overall satisfied with the coach, (4) whether the Division should continue to support the coach, and (5) if the coach’s lack of clinical training negatively affected their experience. This work was considered a quality improvement initiative for which IRB approval was not required.

Over 18 months, the coach supported a 49 Division members including 30 hospitalists and 63 projects. Projects included a wide range of scholarship: medical education research, qualitative research, clinical quality improvement projects, observational studies, and a randomized clinical trial. Many clients (n = 16) used the coach for more than one project. The scope of work included limited support projects (identifying research resource and brainstorming project feasibility) lasting one to two sessions (n = 25), projects with a limited scope (collegial reviews of manuscripts and assistance with IRB submissions) but requiring more than two consultations (n = 24), and ongoing in-depth support projects (contributions on design, data collection, analysis, and manuscript writing) that required three consultations or more (n = 14). The majority of Division members (75%) supported did not have master’s level training in a health services-related area, six had NIH or other national-level funding, and two had small grants funded by local sources prior to providing support. The number of Division faculty on a given project ranged from one to four.

The coach directly supported 13 manuscripts with coach authorship, seven manuscripts without authorship, 11 abstracts, and four grant submissions (Appendix). The coach was a coauthor on all the abstracts and a coinvestigator on the grant applications. Of the 13 publications the coach coauthored, 11 publications have been accepted to peer-reviewed journals and two are currently in the submission process. The types of articles published included one medical evaluation report, one qualitative study, one randomized clinical trial, three quality assessment/improvement reports, and five epidemiologic studies. The types of abstracts included one qualitative report, one systematic review, one randomized clinical trial, two quality improvement projects, two epidemiologic studies, and four medical education projects. Three of four small grants submitted to local and national funders were funded.

The coach’s influence extended beyond the Division. Forty-eight university faculty, fellows, or students not affiliated with general internal medicine benefited from coach coaching: 26 were authors on papers and/or abstracts coauthored by the coach, 17 on manuscripts the coach reviewed without authorship, and five participated in consultations.

The coach found the experience rewarding. She enjoyed working on the methodologic aspects of projects and benefited from being included as coauthor on papers.

Twenty-nine of the 43 faculty (67%) still at the institution responded to the program assessment survey. Faculty strongly agreed that they would recommend the coach to colleagues (average ± standard deviation [SD]: 4.7 ± 0.5), that it improved the quality of their work (4.5 ± 0.9), that they were overall satisfied with the coaching (4.6 ± 0.7), and that the Division should continue to support the coach (4.9 ± 0.4). Faculty did not agree that the lack of clinical training of the coach was a barrier (2.0 ± 1.3).

The coach program was highly utilized, well regarded, and delivered substantial, tangible, and academic output. We anticipate the coach initiative will continue to be a valuable resource for our Division and could prove to be a valuable model for other institutions seeking to bolster the scholarly work of clinical academicians.

Several lessons emerged through the course of this project. First, we realized it is essential to select a coach who is both knowledgeable and approachable. We found that after meeting the coach, many faculty sought her help who otherwise would not have. An explicit, ongoing marketing strategy with regular contact with faculty at meetings was a key to receiving consult requests.

Second, the lack of a clinical background did not seem to hinder the coach’s ability to coach clinicians. The coach acknowledged her lack of clinical experience and relied on clients to explain the clinical context of projects. We also learned that the coach’s substantial experience with the logistics of research was invaluable. For example, the coach had substantial experience with the IRB process and her pre-reviews of IRB applications made for a short and relatively seamless experience navigating the IRB process. The coach also facilitated collaborations and leveraged existing resources at our institution. For example, for a qualitative research project, the coach helped identify a health services faculty member with this specific expertise, which led to a successful collaboration and publication. Although a more junior coach with less established qualifications may be helpful with research methods and with the research process, our endeavor suggests that having a more highly trained and experienced researcher was extremely valuable. Finally, we learned that for a Division of our size, the 0.50 FTE allotted to the coach is a minimum requirement. The coach spent approximately four hours a week on marketing, attending faculty meetings and conducting brief didactics, two hours per week on administration, and 14 hours per week on consultations. Faculty generally received support soon after their requests, but there were occasional wait times, which may have delayed some projects.

Academic leaders at our institution have noted the success of our coach initiative and have created a demand for coach services. We are exploring funding models that would allow for the expansion of coach services to other departments and divisions. We are in the initial stages of creating an Academic Scholarship Support Core under the supervision of the coach. Within this Core, we envision that various research support services will be triaged to staff with appropriate expertise; for example, a regulatory coordinator would review IRB applications while a master’s level statistician would conduct statistical analyses.

We have also transitioned to a new coach and have continued to experience success with the program. Our initial coach (author C.M.M.) obtained an NIH R01, a foundation grant, and took over a summer program that trains dental faculty in clinical research methods leaving insufficient time for coaching. Our new coach also has a PhD in epidemiology with NIH R01 funding but has more available FTE. Both of our coaches are graduates of our School of Public Health and institutions with such schools may have good access to the expertise needed. Nonclinical PhDs are often almost entirely reliant on grants, and some nongrant support is often attractive to these researchers. Additionally, PhDs who are junior or mid-career faculty that have the needed training are relatively affordable, particularly when the resource is made available to large number of faculty. For example, our first coach cost $48,000 a year for 50% FTE.

A limitation to our assessment of the coach initiative was the lack of pre- and postintervention metrics of scholarly productivity. We cannot definitively say that the Division’s scholarly output has increased because of the coach. Nevertheless, we are confident that the coach’s coaching has enhanced the scholarly work of individual clinicians and provided value to the Division as a whole. The coach program has been a success in our Division. Other institutions facing the challenge of supporting the research efforts of academic clinicians may consider this model as a worthy investment.

Disclosures

The authors have nothing to disclose.