Original Research

Sliding scale insulin (SSI) for inpatient glycemic control was first proposed by Elliott P Joslin in 1934 when he recommended titration of insulin based on urine glucose levels.¹ As bedside glucose meters became widely available, physicians transitioned to dosing SSI based on capillary blood glucose (BG) levels,^2,3 and SSI became widely used for the management of inpatient hyperglycemia.¹ However, during the past decade, there has been strong opposition to the use of SSI in hospitals. Many authors oppose its use, highlighting the retrospective rather than prospective nature of SSI therapy and concerns about inadequate glycemic control.^4-6 In 2004, the American College of Endocrinology first released a position statement discouraging the use of SSI alone and recommended basal-bolus insulin as the preferred method of glycemic control for inpatients with type 2 diabetes (T2D).⁷ The American Diabetes Association (ADA) inpatient guidelines in 2005⁸ and the Endocrine Society guidelines in 2012⁹ also opposed SSI monotherapy and reaffirmed that a basal-bolus insulin regimen should be used for most non–critically ill patients with diabetes. Those guidelines remain in place currently.

Several randomized controlled trials (RCTs) and meta-analyses have shown that basal-bolus insulin regimens provide superior glycemic control in non–critical inpatients when compared with SSI alone.^10-14 In addition, the RABBIT 2 (Randomized Study of Basal-Bolus Insulin Therapy in the Inpatient Management of Patients With Type 2 Diabetes) trial showed a significant reduction in perioperative complications¹⁰ among surgical patients when treated with basal-bolus insulin therapy. Despite these studies and strong recommendations against its use, SSI continues to be widely used in the United States. According to a 2007 survey of 44 US hospitals, 41% of noncritical patients with hyperglycemia were treated with SSI alone.¹⁵ In addition, SSI remains one of the most commonly prescribed insulin regimens in many countries around the world.^16-19 The persistence of SSI use raises questions as to why clinicians continue to use a therapy that has been strongly criticized. Some authors point to convenience and fear of hypoglycemia with a basal-bolus insulin regimen.^20,21 Alternatively, it is possible that SSI usage remains so pervasive because it is effective in a subset of patients. In fact, a 2018 Cochrane review concluded that existing evidence is not sufficiently robust to definitively recommend basal-bolus insulin over SSI for inpatient diabetes management of non–critically ill patients despite existing guidelines.²²

Owing to the ongoing controversy and widespread use of SSI, we designed an exploratory analysis to understand the rationale for such therapy by investigating whether a certain subpopulation of hospitalized patients with T2D may achieve target glycemic control with SSI alone. We hypothesized that noncritical patients with mild hyperglycemia and admission BG <180 mg/dL would do well with SSI alone and may not require intensive treatment with basal-bolus insulin regimens. To address this question, we used electronic health records with individual-level patient data to assess inpatient glycemic control of non–critically ill patients with T2D treated with SSI alone.

Participants

Data from 25,813 adult noncritical inpatients with T2D, with an index admission between June 1, 2010, and June 30, 2018, were obtained through the Emory Healthcare Clinical Data Warehouse infrastructure program. All patients were admitted to Emory Healthcare hospitals, including Emory University Hospital, Emory University Hospital Midtown, and Emory Saint Joseph’s Hospital, in Atlanta, Georgia. Data were extracted for each patient during the index hospitalization, including demographics, anthropometrics, and admission and inpatient laboratory values. Information was collected on daily point-of-care glucose values, hemoglobin A1c (HbA1c), hypoglycemic events, insulin doses, hospital complications, comorbidities, and hospital setting (medical vs surgical admission). International Classification of Diseases, 9th and 10th Revisions (ICD-9/10) codes were used to determine diagnosis of T2D, comorbidities, and complications.

From our initial dataset, we identified 16,366 patients who were treated with SSI during hospitalization. We excluded patients who were admitted to the intensive care unit (ICU) or placed on intravenous insulin, patients with missing admission BG values, and patients with a length of stay less than 1 day. To prevent inclusion of patients presenting in diabetic ketoacidosis or hyperosmolar hyperglycemic syndrome, we excluded patients with an admission BG >500 mg/dL. We then excluded 6,739 patients who received basal insulin within the first 2 days of hospitalization, as well as 943 patients who were treated with noninsulin (oral or injectable) antidiabetic agents. Our final dataset included 8,095 patients (Appendix Figure).

Patients in the SSI cohort included all patients who were treated with short-acting insulin only (regular insulin or rapid-acting [lispro, aspart, glulisine] insulin analogs) during the first 2 days of hospitalization. Patients who remained on only short-acting insulin during the entire hospitalization were defined as continuous SSI patients. Patients who subsequently received basal insulin after day 2 of hospitalization were defined as patients who transitioned to basal. Patients were stratified according to admission BG levels (first BG available on day of admission) and HbA1c (when available during index admission). We compared the baseline characteristics and clinical outcomes of patients who remained on SSI alone throughout the entirety of hospitalization with those of patients who required transition to basal insulin. The mean hospital BG was calculated by taking the average of all BG measurements during the hospital stay. We defined hypoglycemia as a BG <70 mg/dL and severe hypoglycemia as BG <40 mg/dL. Repeated hypoglycemia values were excluded if they occurred within a period of 2 hours.

Outcome Measures

The primary outcome was the percentage of patients with T2D achieving target glycemic control with SSI therapy, defined as mean hospital BG between 70 and 180 mg/dL without hypoglycemia <70 mg/dL during hospital stay. This threshold was determined based on 2019 ADA recommendations targeting hospital BG <180 mg/dL and avoidance of hypoglycemia.²³

Statistical Analysis

Patients were stratified according to continuous SSI versus transitioned to basal treatment. Patients who remained on continuous SSI were further categorized into four categories based on admission BG: <140 mg/dL, 140 to 180 mg/dL, 180 to 250 mg/dL, and ≥250 mg/dL. Clinical characteristics were compared using Wilcoxon rank-sum tests (if continuous) and chi-square tests or Fisher exact tests (if categorical). We then compared the clinical outcomes among continuous SSI patients with different admission BG levels (<140 mg/dL, 140-180 mg/dL, 180-250 mg/dL, and ≥250 mg/dL) and with different HbA1c levels (<7%, 7%-8%, 8%-9%, ≥9%). Within each scenario, logistic regression for the outcome of poor glycemic control, defined as mean hospital BG >180 mg/dL, was performed to evaluate the HbA1c levels and admission BG levels controlling for other factors (age, gender, body mass index [BMI], race, setting [medicine versus surgery] and Charlson Comorbidity Index score). A P value < .05 was regarded as statistically significant. All analyses were performed based on available cases and conducted in SAS version 9.4 (SAS Institute Inc.).

Among 25,813 adult patients with T2D, 8,095 patients (31.4%) were treated with SSI alone during the first 2 days of hospitalization. Of those patients treated with SSI, 6,903 (85%) remained on continuous SSI alone during the entire hospitalization, and 1,192 (15%) were transitioned to basal insulin. The clinical characteristics of these patients on continuous SSI and those who transitioned to basal insulin are shown in Table 1. Patients who transitioned to basal insulin had significantly higher mean (SD) admission BG (191.8 [88.2] mg/dL vs 156.4 [65.4] mg/dL, P < .001) and higher mean (SD) HbA1c (8.1% [2.0%] vs 7.01% [1.5%], P < .001), compared with those who remained on continuous SSI. Patients who transitioned to basal insulin were also younger and more likely to have chronic kidney disease (CKD), but less likely to have congestive heart failure, coronary artery disease, or chronic obstructive pulmonary disease (COPD). The Charlson Comorbidity Index score was significantly higher for patients who transitioned to basal (4.4 [2.5]) than for those who remained on continuous SSI (4.1 [2.5], P < .001). There were no significant differences among sex, BMI, or glomerular filtration rate (GFR) on admission. Of those transitioned to basal insulin, 53% achieved a mean hospitalization BG <180 mg/dL, compared with 82% of those on continuous SSI. The overall rate of hypoglycemia in the continuous SSI group was 8% compared with 18% in those transitioned to basal insulin.

Of the patients who remained on continuous SSI throughout the hospitalization, 3,319 patients (48%) had admission BG <140 mg/dL, 1,671 patients (24%) had admission BG 140 to 180 mg/dL, and 1,913 patients (28%) had admission BG >180 mg/dL. Only 9% of patients who remained on continuous SSI had admission BG ≥250 mg/dL. Patients with admission BG <140 mg/dL were older, had lower BMI and HbA1c, had higher rates of COPD and CKD, and were more likely to be admitted to a surgical service compared with patients with admission BG >140 mg/dL (P < .05 for all; Table 2).

Hospital glycemic control for patients on continuous SSI according to admission BG is displayed in Table 3. Among patients who remained on continuous SSI, 96% of patients with admission BG <140 mg/dL had a mean hospital BG <180 mg/dL; of them, 86% achieved target control without hypoglycemia. Similar rates of target control were achieved in patients with admission BG 140 to 180 mg/dL (83%), in contrast to patients with admission BG ≥250 mg/dL, of whom only 18% achieved target control (P < .001). These findings parallel those seen in patients transitioned to basal insulin. Of patients in the transition group admitted with BG <140 mg/dL and <180 mg/dL, 88.5% and 84.6% had mean hospital BG <180 mg/dL, respectively, while 69.1% and 68.9% had mean BG between 70 and 180 mg/dL without hypoglycemia. The overall frequency of hypoglycemia <70 mg/dL among patients on continuous SSI was 8% and was more common in patients with admission BG <140 mg/dL (10%) compared with patients with higher admission glucose levels (BG 140-180 mg/dL [4%], 180-250 mg/dL [4%], or ≥250 mg/dL [6%], P < .001). There was no difference in rates of severe hypoglycemia <40 mg/dL among groups.

HbA1c data were available for 2,560 of the patients on continuous SSI (Table 3). Mean hospital BG increased significantly with increasing HbA1c values. Patients admitted with HbA1c <7% had lower mean (SD) hospital BG (132.2 [28.2] mg/dL) and were more likely to achieve target glucose control during hospitalization (85%) compared with those with HbA1c 7% to 8% (mean BG, 148.7 [30.8] mg/dL; 80% target control), HbA1c 8% to 9% (mean BG, 169.1 [37.9] mg/dL; 61% target control), or HbA1c ≥9% (mean BG, 194.9 [53.4] mg/dL; 38% target control) (P < .001).

In a logistic regression analysis adjusted for age, gender, BMI, race, setting (medicine vs surgery), and Charlson Comorbidity Index score, the odds of poor glycemic control increased with higher admission BG (admission BG 140-180 mg/dL: odds ratio [OR], 1.8; 95% CI, 1.5-2.2; admission BG 180-250 mg/dL: OR, 3.7; 95% CI, 3.1-4.4; admission BG ≥250 mg/dL: OR, 7.2; 95% CI, 5.8-9.0; reference admission BG <140 mg/dL; Figure). Similarly, the logistic regression analysis showed greater odds of poor in-hospital glycemic control with increasing HbA1c (OR, 6.1; 95% CI, 4.3-8.8 for HbA1c >9% compared with HbA1c <7%).

This large retrospective cohort study examined the effectiveness of SSI for glycemic control in noncritical inpatients with T2D. Our results indicate that SSI is still widely used in our hospital system, with 31.4% of our initial cohort managed with SSI alone. We found that 86% of patients with BG <140 mg/dL and 83% of patients with BG 140 to 180 mg/dL achieved glycemic control without hypoglycemia when managed with SSI alone, compared with 53% of those admitted with BG 180 to 250 mg/dL and only 18% of those with admission BG ≥250 mg/dL. This high success rate of achieving optimal BG control with SSI alone is comparable to that seen with transition to basal insulin and may explain the prevalent use of SSI for the management of patients with T2D and mild to moderate hyperglycemia.

Published clinical guideline recommendations promoting the use of basal-bolus insulin treatment algorithms are based on the results of a few RCTs that compared the efficacy of SSI vs a basal-bolus insulin regimen. These studies reported significantly lower mean daily BG concentration with basal or basal-bolus insulin therapy compared with SSI.^10,11,24 However, it is interesting to note that the mean admission BG of patients treated with SSI in these RCTs ranged from 184 to 225 mg/dL. Patients in these trials were excluded if admission BG was <140 mg/dL.^10,11,24 This is in contrast to our study evaluating real-world data in non–critically ill settings in which we found that 48% of patients treated with SSI had admission BG <140 mg/dL, and nearly 75% had admission BG <180 mg/dL. This suggests that by nature of study design, most RCTs excluded the population of patients who do achieve good glycemic control with SSI and may have contributed to the perception that basal insulin is preferable in all populations.

Our analysis indicates that healthcare professionals should consider admission BG when selecting the type of insulin regimen to manage patients with T2D in the hospital. Our results suggest that SSI may be appropriate for many patients with admission BG <180 mg/dL and should be avoided as monotherapy in patients with admission BG ≥180 mg/dL, as the proportion of patients achieving target control decreased with increasing admission BG. More importantly, if a patient is not controlled with SSI alone, intensification of therapy with the addition of basal insulin is indicated to achieve glycemic control. In addition, we found that the admission HbA1c is an appropriate marker to consider as well, with hospital glycemic control deteriorating with increasing HbA1c values, paralleling the admission BG. The main limitation to widespread use of HbA1c for therapeutic decision-making is access to values at time of patient admission; in our population, only 37% of patients had an HbA1c value available during the index hospitalization.

Previous publications have reported that hypoglycemia carries significant safety concerns, especially among a hospitalized population.^25-27 As such, we included hypoglycemia as an important metric in our definition of target glycemic control rather than simply using mean hospital BG or number of hyperglycemic events to define treatment effectiveness. We did find a higher rate of hypoglycemia in patients with moderate admission BG treated with SSI compared with those with higher admission BG; however, few patients overall experienced clinically significant (<54 mg/dL) or severe (<40 mg/dL) hypoglycemia.

In our population, only 15% of patients started on SSI received additional basal insulin during hospitalization. This finding is similar to data reported in the Rabbit 2 trial, in which 14% of patients failed SSI alone, with a higher failure rate among those with higher BG on admission.¹⁰ Given the observational nature of this study, we cannot definitively state why certain patients in our population required additional basal insulin, but we can hypothesize that these patients admitted with BG ≥180 mg/dL had higher treatment failure rates and greater rates of hyperglycemia, therefore receiving intensified insulin therapy as clinically indicated at the discretion of the treating physician. Patients who transitioned from SSI to basal insulin had significantly higher admission BG and HbA1c compared with patients who remained on SSI alone. We noted that the rates of hypoglycemia were higher in the group that transitioned to basal (18% vs 8%) and similar to rates reported in previous RCTs.^11,24

This observational study takes advantage of a large, diverse study population and a combination of medicine and surgery patients in a real-world setting. We acknowledge several limitations in our study. Our primary data were observational in nature, and as such, some baseline patient characteristics were notably different between groups, suggesting selection bias for treatment allocation to SSI. We do not know which patients were managed by primary teams compared with specialized diabetes consult services, which may also influence treatment regimens. We did not have access to information about patients’ at-home diabetes medication regimens or duration of diabetes, both of which have been shown in prior publications to affect an individual’s overall hospital glycemic control. Data on HbA1c values were available for only approximately one-third of patients. In addition, our study did not include patients without a history of diabetes who developed stress-induced hyperglycemia, a population that may benefit from conservative therapy such as SSI.²⁸ A diagnosis of CKD was defined based on ICD 9/10 codes and not on admission estimated GFR. More specific data regarding stage of CKD or changes in renal function over the duration of hospitalization are not available, which could influence insulin prescribing practice. In addition, we defined the basal group as patients prescribed any form of basal insulin (NPH, glargine, detemir or degludec), and we do not have information on the use of prandial versus correction doses of rapid-acting insulin in the basal insulin–treated group.

In conclusion, our observational study indicates that the use of SSI results in appropriate target glycemic control for most noncritical medicine and surgery patients with admission BG <180 mg/dL. In agreement with previous RCTs, our study confirms that SSI as monotherapy is frequently inadequate in patients with significant hyperglycemia >180 mg/dL.^10,11,24,29 We propose that an individualized approach to inpatient glycemic management is imperative, and cautious use of SSI may be a viable option for certain patients with mild hyperglycemia and admission BG <180 mg/dL. Further observational and randomized studies are needed to confirm the efficacy of SSI therapy in T2D patients with mild hyperglycemia. By identifying which subset of patients can be safely managed with SSI alone, we can better understand which patients will require escalation of therapy with intensive glucose management.

Sliding scale insulin (SSI) for inpatient glycemic control was first proposed by Elliott P Joslin in 1934 when he recommended titration of insulin based on urine glucose levels.¹ As bedside glucose meters became widely available, physicians transitioned to dosing SSI based on capillary blood glucose (BG) levels,^2,3 and SSI became widely used for the management of inpatient hyperglycemia.¹ However, during the past decade, there has been strong opposition to the use of SSI in hospitals. Many authors oppose its use, highlighting the retrospective rather than prospective nature of SSI therapy and concerns about inadequate glycemic control.^4-6 In 2004, the American College of Endocrinology first released a position statement discouraging the use of SSI alone and recommended basal-bolus insulin as the preferred method of glycemic control for inpatients with type 2 diabetes (T2D).⁷ The American Diabetes Association (ADA) inpatient guidelines in 2005⁸ and the Endocrine Society guidelines in 2012⁹ also opposed SSI monotherapy and reaffirmed that a basal-bolus insulin regimen should be used for most non–critically ill patients with diabetes. Those guidelines remain in place currently.

Several randomized controlled trials (RCTs) and meta-analyses have shown that basal-bolus insulin regimens provide superior glycemic control in non–critical inpatients when compared with SSI alone.^10-14 In addition, the RABBIT 2 (Randomized Study of Basal-Bolus Insulin Therapy in the Inpatient Management of Patients With Type 2 Diabetes) trial showed a significant reduction in perioperative complications¹⁰ among surgical patients when treated with basal-bolus insulin therapy. Despite these studies and strong recommendations against its use, SSI continues to be widely used in the United States. According to a 2007 survey of 44 US hospitals, 41% of noncritical patients with hyperglycemia were treated with SSI alone.¹⁵ In addition, SSI remains one of the most commonly prescribed insulin regimens in many countries around the world.^16-19 The persistence of SSI use raises questions as to why clinicians continue to use a therapy that has been strongly criticized. Some authors point to convenience and fear of hypoglycemia with a basal-bolus insulin regimen.^20,21 Alternatively, it is possible that SSI usage remains so pervasive because it is effective in a subset of patients. In fact, a 2018 Cochrane review concluded that existing evidence is not sufficiently robust to definitively recommend basal-bolus insulin over SSI for inpatient diabetes management of non–critically ill patients despite existing guidelines.²²

Owing to the ongoing controversy and widespread use of SSI, we designed an exploratory analysis to understand the rationale for such therapy by investigating whether a certain subpopulation of hospitalized patients with T2D may achieve target glycemic control with SSI alone. We hypothesized that noncritical patients with mild hyperglycemia and admission BG <180 mg/dL would do well with SSI alone and may not require intensive treatment with basal-bolus insulin regimens. To address this question, we used electronic health records with individual-level patient data to assess inpatient glycemic control of non–critically ill patients with T2D treated with SSI alone.

Participants

Data from 25,813 adult noncritical inpatients with T2D, with an index admission between June 1, 2010, and June 30, 2018, were obtained through the Emory Healthcare Clinical Data Warehouse infrastructure program. All patients were admitted to Emory Healthcare hospitals, including Emory University Hospital, Emory University Hospital Midtown, and Emory Saint Joseph’s Hospital, in Atlanta, Georgia. Data were extracted for each patient during the index hospitalization, including demographics, anthropometrics, and admission and inpatient laboratory values. Information was collected on daily point-of-care glucose values, hemoglobin A1c (HbA1c), hypoglycemic events, insulin doses, hospital complications, comorbidities, and hospital setting (medical vs surgical admission). International Classification of Diseases, 9th and 10th Revisions (ICD-9/10) codes were used to determine diagnosis of T2D, comorbidities, and complications.

From our initial dataset, we identified 16,366 patients who were treated with SSI during hospitalization. We excluded patients who were admitted to the intensive care unit (ICU) or placed on intravenous insulin, patients with missing admission BG values, and patients with a length of stay less than 1 day. To prevent inclusion of patients presenting in diabetic ketoacidosis or hyperosmolar hyperglycemic syndrome, we excluded patients with an admission BG >500 mg/dL. We then excluded 6,739 patients who received basal insulin within the first 2 days of hospitalization, as well as 943 patients who were treated with noninsulin (oral or injectable) antidiabetic agents. Our final dataset included 8,095 patients (Appendix Figure).

Patients in the SSI cohort included all patients who were treated with short-acting insulin only (regular insulin or rapid-acting [lispro, aspart, glulisine] insulin analogs) during the first 2 days of hospitalization. Patients who remained on only short-acting insulin during the entire hospitalization were defined as continuous SSI patients. Patients who subsequently received basal insulin after day 2 of hospitalization were defined as patients who transitioned to basal. Patients were stratified according to admission BG levels (first BG available on day of admission) and HbA1c (when available during index admission). We compared the baseline characteristics and clinical outcomes of patients who remained on SSI alone throughout the entirety of hospitalization with those of patients who required transition to basal insulin. The mean hospital BG was calculated by taking the average of all BG measurements during the hospital stay. We defined hypoglycemia as a BG <70 mg/dL and severe hypoglycemia as BG <40 mg/dL. Repeated hypoglycemia values were excluded if they occurred within a period of 2 hours.

Outcome Measures

The primary outcome was the percentage of patients with T2D achieving target glycemic control with SSI therapy, defined as mean hospital BG between 70 and 180 mg/dL without hypoglycemia <70 mg/dL during hospital stay. This threshold was determined based on 2019 ADA recommendations targeting hospital BG <180 mg/dL and avoidance of hypoglycemia.²³

Statistical Analysis

Patients were stratified according to continuous SSI versus transitioned to basal treatment. Patients who remained on continuous SSI were further categorized into four categories based on admission BG: <140 mg/dL, 140 to 180 mg/dL, 180 to 250 mg/dL, and ≥250 mg/dL. Clinical characteristics were compared using Wilcoxon rank-sum tests (if continuous) and chi-square tests or Fisher exact tests (if categorical). We then compared the clinical outcomes among continuous SSI patients with different admission BG levels (<140 mg/dL, 140-180 mg/dL, 180-250 mg/dL, and ≥250 mg/dL) and with different HbA1c levels (<7%, 7%-8%, 8%-9%, ≥9%). Within each scenario, logistic regression for the outcome of poor glycemic control, defined as mean hospital BG >180 mg/dL, was performed to evaluate the HbA1c levels and admission BG levels controlling for other factors (age, gender, body mass index [BMI], race, setting [medicine versus surgery] and Charlson Comorbidity Index score). A P value < .05 was regarded as statistically significant. All analyses were performed based on available cases and conducted in SAS version 9.4 (SAS Institute Inc.).

Among 25,813 adult patients with T2D, 8,095 patients (31.4%) were treated with SSI alone during the first 2 days of hospitalization. Of those patients treated with SSI, 6,903 (85%) remained on continuous SSI alone during the entire hospitalization, and 1,192 (15%) were transitioned to basal insulin. The clinical characteristics of these patients on continuous SSI and those who transitioned to basal insulin are shown in Table 1. Patients who transitioned to basal insulin had significantly higher mean (SD) admission BG (191.8 [88.2] mg/dL vs 156.4 [65.4] mg/dL, P < .001) and higher mean (SD) HbA1c (8.1% [2.0%] vs 7.01% [1.5%], P < .001), compared with those who remained on continuous SSI. Patients who transitioned to basal insulin were also younger and more likely to have chronic kidney disease (CKD), but less likely to have congestive heart failure, coronary artery disease, or chronic obstructive pulmonary disease (COPD). The Charlson Comorbidity Index score was significantly higher for patients who transitioned to basal (4.4 [2.5]) than for those who remained on continuous SSI (4.1 [2.5], P < .001). There were no significant differences among sex, BMI, or glomerular filtration rate (GFR) on admission. Of those transitioned to basal insulin, 53% achieved a mean hospitalization BG <180 mg/dL, compared with 82% of those on continuous SSI. The overall rate of hypoglycemia in the continuous SSI group was 8% compared with 18% in those transitioned to basal insulin.

Of the patients who remained on continuous SSI throughout the hospitalization, 3,319 patients (48%) had admission BG <140 mg/dL, 1,671 patients (24%) had admission BG 140 to 180 mg/dL, and 1,913 patients (28%) had admission BG >180 mg/dL. Only 9% of patients who remained on continuous SSI had admission BG ≥250 mg/dL. Patients with admission BG <140 mg/dL were older, had lower BMI and HbA1c, had higher rates of COPD and CKD, and were more likely to be admitted to a surgical service compared with patients with admission BG >140 mg/dL (P < .05 for all; Table 2).

Hospital glycemic control for patients on continuous SSI according to admission BG is displayed in Table 3. Among patients who remained on continuous SSI, 96% of patients with admission BG <140 mg/dL had a mean hospital BG <180 mg/dL; of them, 86% achieved target control without hypoglycemia. Similar rates of target control were achieved in patients with admission BG 140 to 180 mg/dL (83%), in contrast to patients with admission BG ≥250 mg/dL, of whom only 18% achieved target control (P < .001). These findings parallel those seen in patients transitioned to basal insulin. Of patients in the transition group admitted with BG <140 mg/dL and <180 mg/dL, 88.5% and 84.6% had mean hospital BG <180 mg/dL, respectively, while 69.1% and 68.9% had mean BG between 70 and 180 mg/dL without hypoglycemia. The overall frequency of hypoglycemia <70 mg/dL among patients on continuous SSI was 8% and was more common in patients with admission BG <140 mg/dL (10%) compared with patients with higher admission glucose levels (BG 140-180 mg/dL [4%], 180-250 mg/dL [4%], or ≥250 mg/dL [6%], P < .001). There was no difference in rates of severe hypoglycemia <40 mg/dL among groups.

HbA1c data were available for 2,560 of the patients on continuous SSI (Table 3). Mean hospital BG increased significantly with increasing HbA1c values. Patients admitted with HbA1c <7% had lower mean (SD) hospital BG (132.2 [28.2] mg/dL) and were more likely to achieve target glucose control during hospitalization (85%) compared with those with HbA1c 7% to 8% (mean BG, 148.7 [30.8] mg/dL; 80% target control), HbA1c 8% to 9% (mean BG, 169.1 [37.9] mg/dL; 61% target control), or HbA1c ≥9% (mean BG, 194.9 [53.4] mg/dL; 38% target control) (P < .001).

In a logistic regression analysis adjusted for age, gender, BMI, race, setting (medicine vs surgery), and Charlson Comorbidity Index score, the odds of poor glycemic control increased with higher admission BG (admission BG 140-180 mg/dL: odds ratio [OR], 1.8; 95% CI, 1.5-2.2; admission BG 180-250 mg/dL: OR, 3.7; 95% CI, 3.1-4.4; admission BG ≥250 mg/dL: OR, 7.2; 95% CI, 5.8-9.0; reference admission BG <140 mg/dL; Figure). Similarly, the logistic regression analysis showed greater odds of poor in-hospital glycemic control with increasing HbA1c (OR, 6.1; 95% CI, 4.3-8.8 for HbA1c >9% compared with HbA1c <7%).

This large retrospective cohort study examined the effectiveness of SSI for glycemic control in noncritical inpatients with T2D. Our results indicate that SSI is still widely used in our hospital system, with 31.4% of our initial cohort managed with SSI alone. We found that 86% of patients with BG <140 mg/dL and 83% of patients with BG 140 to 180 mg/dL achieved glycemic control without hypoglycemia when managed with SSI alone, compared with 53% of those admitted with BG 180 to 250 mg/dL and only 18% of those with admission BG ≥250 mg/dL. This high success rate of achieving optimal BG control with SSI alone is comparable to that seen with transition to basal insulin and may explain the prevalent use of SSI for the management of patients with T2D and mild to moderate hyperglycemia.

Published clinical guideline recommendations promoting the use of basal-bolus insulin treatment algorithms are based on the results of a few RCTs that compared the efficacy of SSI vs a basal-bolus insulin regimen. These studies reported significantly lower mean daily BG concentration with basal or basal-bolus insulin therapy compared with SSI.^10,11,24 However, it is interesting to note that the mean admission BG of patients treated with SSI in these RCTs ranged from 184 to 225 mg/dL. Patients in these trials were excluded if admission BG was <140 mg/dL.^10,11,24 This is in contrast to our study evaluating real-world data in non–critically ill settings in which we found that 48% of patients treated with SSI had admission BG <140 mg/dL, and nearly 75% had admission BG <180 mg/dL. This suggests that by nature of study design, most RCTs excluded the population of patients who do achieve good glycemic control with SSI and may have contributed to the perception that basal insulin is preferable in all populations.

Our analysis indicates that healthcare professionals should consider admission BG when selecting the type of insulin regimen to manage patients with T2D in the hospital. Our results suggest that SSI may be appropriate for many patients with admission BG <180 mg/dL and should be avoided as monotherapy in patients with admission BG ≥180 mg/dL, as the proportion of patients achieving target control decreased with increasing admission BG. More importantly, if a patient is not controlled with SSI alone, intensification of therapy with the addition of basal insulin is indicated to achieve glycemic control. In addition, we found that the admission HbA1c is an appropriate marker to consider as well, with hospital glycemic control deteriorating with increasing HbA1c values, paralleling the admission BG. The main limitation to widespread use of HbA1c for therapeutic decision-making is access to values at time of patient admission; in our population, only 37% of patients had an HbA1c value available during the index hospitalization.

Previous publications have reported that hypoglycemia carries significant safety concerns, especially among a hospitalized population.^25-27 As such, we included hypoglycemia as an important metric in our definition of target glycemic control rather than simply using mean hospital BG or number of hyperglycemic events to define treatment effectiveness. We did find a higher rate of hypoglycemia in patients with moderate admission BG treated with SSI compared with those with higher admission BG; however, few patients overall experienced clinically significant (<54 mg/dL) or severe (<40 mg/dL) hypoglycemia.

In our population, only 15% of patients started on SSI received additional basal insulin during hospitalization. This finding is similar to data reported in the Rabbit 2 trial, in which 14% of patients failed SSI alone, with a higher failure rate among those with higher BG on admission.¹⁰ Given the observational nature of this study, we cannot definitively state why certain patients in our population required additional basal insulin, but we can hypothesize that these patients admitted with BG ≥180 mg/dL had higher treatment failure rates and greater rates of hyperglycemia, therefore receiving intensified insulin therapy as clinically indicated at the discretion of the treating physician. Patients who transitioned from SSI to basal insulin had significantly higher admission BG and HbA1c compared with patients who remained on SSI alone. We noted that the rates of hypoglycemia were higher in the group that transitioned to basal (18% vs 8%) and similar to rates reported in previous RCTs.^11,24

This observational study takes advantage of a large, diverse study population and a combination of medicine and surgery patients in a real-world setting. We acknowledge several limitations in our study. Our primary data were observational in nature, and as such, some baseline patient characteristics were notably different between groups, suggesting selection bias for treatment allocation to SSI. We do not know which patients were managed by primary teams compared with specialized diabetes consult services, which may also influence treatment regimens. We did not have access to information about patients’ at-home diabetes medication regimens or duration of diabetes, both of which have been shown in prior publications to affect an individual’s overall hospital glycemic control. Data on HbA1c values were available for only approximately one-third of patients. In addition, our study did not include patients without a history of diabetes who developed stress-induced hyperglycemia, a population that may benefit from conservative therapy such as SSI.²⁸ A diagnosis of CKD was defined based on ICD 9/10 codes and not on admission estimated GFR. More specific data regarding stage of CKD or changes in renal function over the duration of hospitalization are not available, which could influence insulin prescribing practice. In addition, we defined the basal group as patients prescribed any form of basal insulin (NPH, glargine, detemir or degludec), and we do not have information on the use of prandial versus correction doses of rapid-acting insulin in the basal insulin–treated group.

In conclusion, our observational study indicates that the use of SSI results in appropriate target glycemic control for most noncritical medicine and surgery patients with admission BG <180 mg/dL. In agreement with previous RCTs, our study confirms that SSI as monotherapy is frequently inadequate in patients with significant hyperglycemia >180 mg/dL.^10,11,24,29 We propose that an individualized approach to inpatient glycemic management is imperative, and cautious use of SSI may be a viable option for certain patients with mild hyperglycemia and admission BG <180 mg/dL. Further observational and randomized studies are needed to confirm the efficacy of SSI therapy in T2D patients with mild hyperglycemia. By identifying which subset of patients can be safely managed with SSI alone, we can better understand which patients will require escalation of therapy with intensive glucose management.

Sliding scale insulin (SSI) for inpatient glycemic control was first proposed by Elliott P Joslin in 1934 when he recommended titration of insulin based on urine glucose levels.¹ As bedside glucose meters became widely available, physicians transitioned to dosing SSI based on capillary blood glucose (BG) levels,^2,3 and SSI became widely used for the management of inpatient hyperglycemia.¹ However, during the past decade, there has been strong opposition to the use of SSI in hospitals. Many authors oppose its use, highlighting the retrospective rather than prospective nature of SSI therapy and concerns about inadequate glycemic control.^4-6 In 2004, the American College of Endocrinology first released a position statement discouraging the use of SSI alone and recommended basal-bolus insulin as the preferred method of glycemic control for inpatients with type 2 diabetes (T2D).⁷ The American Diabetes Association (ADA) inpatient guidelines in 2005⁸ and the Endocrine Society guidelines in 2012⁹ also opposed SSI monotherapy and reaffirmed that a basal-bolus insulin regimen should be used for most non–critically ill patients with diabetes. Those guidelines remain in place currently.

Several randomized controlled trials (RCTs) and meta-analyses have shown that basal-bolus insulin regimens provide superior glycemic control in non–critical inpatients when compared with SSI alone.^10-14 In addition, the RABBIT 2 (Randomized Study of Basal-Bolus Insulin Therapy in the Inpatient Management of Patients With Type 2 Diabetes) trial showed a significant reduction in perioperative complications¹⁰ among surgical patients when treated with basal-bolus insulin therapy. Despite these studies and strong recommendations against its use, SSI continues to be widely used in the United States. According to a 2007 survey of 44 US hospitals, 41% of noncritical patients with hyperglycemia were treated with SSI alone.¹⁵ In addition, SSI remains one of the most commonly prescribed insulin regimens in many countries around the world.^16-19 The persistence of SSI use raises questions as to why clinicians continue to use a therapy that has been strongly criticized. Some authors point to convenience and fear of hypoglycemia with a basal-bolus insulin regimen.^20,21 Alternatively, it is possible that SSI usage remains so pervasive because it is effective in a subset of patients. In fact, a 2018 Cochrane review concluded that existing evidence is not sufficiently robust to definitively recommend basal-bolus insulin over SSI for inpatient diabetes management of non–critically ill patients despite existing guidelines.²²

Owing to the ongoing controversy and widespread use of SSI, we designed an exploratory analysis to understand the rationale for such therapy by investigating whether a certain subpopulation of hospitalized patients with T2D may achieve target glycemic control with SSI alone. We hypothesized that noncritical patients with mild hyperglycemia and admission BG <180 mg/dL would do well with SSI alone and may not require intensive treatment with basal-bolus insulin regimens. To address this question, we used electronic health records with individual-level patient data to assess inpatient glycemic control of non–critically ill patients with T2D treated with SSI alone.

Participants

Data from 25,813 adult noncritical inpatients with T2D, with an index admission between June 1, 2010, and June 30, 2018, were obtained through the Emory Healthcare Clinical Data Warehouse infrastructure program. All patients were admitted to Emory Healthcare hospitals, including Emory University Hospital, Emory University Hospital Midtown, and Emory Saint Joseph’s Hospital, in Atlanta, Georgia. Data were extracted for each patient during the index hospitalization, including demographics, anthropometrics, and admission and inpatient laboratory values. Information was collected on daily point-of-care glucose values, hemoglobin A1c (HbA1c), hypoglycemic events, insulin doses, hospital complications, comorbidities, and hospital setting (medical vs surgical admission). International Classification of Diseases, 9th and 10th Revisions (ICD-9/10) codes were used to determine diagnosis of T2D, comorbidities, and complications.

From our initial dataset, we identified 16,366 patients who were treated with SSI during hospitalization. We excluded patients who were admitted to the intensive care unit (ICU) or placed on intravenous insulin, patients with missing admission BG values, and patients with a length of stay less than 1 day. To prevent inclusion of patients presenting in diabetic ketoacidosis or hyperosmolar hyperglycemic syndrome, we excluded patients with an admission BG >500 mg/dL. We then excluded 6,739 patients who received basal insulin within the first 2 days of hospitalization, as well as 943 patients who were treated with noninsulin (oral or injectable) antidiabetic agents. Our final dataset included 8,095 patients (Appendix Figure).

Patients in the SSI cohort included all patients who were treated with short-acting insulin only (regular insulin or rapid-acting [lispro, aspart, glulisine] insulin analogs) during the first 2 days of hospitalization. Patients who remained on only short-acting insulin during the entire hospitalization were defined as continuous SSI patients. Patients who subsequently received basal insulin after day 2 of hospitalization were defined as patients who transitioned to basal. Patients were stratified according to admission BG levels (first BG available on day of admission) and HbA1c (when available during index admission). We compared the baseline characteristics and clinical outcomes of patients who remained on SSI alone throughout the entirety of hospitalization with those of patients who required transition to basal insulin. The mean hospital BG was calculated by taking the average of all BG measurements during the hospital stay. We defined hypoglycemia as a BG <70 mg/dL and severe hypoglycemia as BG <40 mg/dL. Repeated hypoglycemia values were excluded if they occurred within a period of 2 hours.

Outcome Measures

The primary outcome was the percentage of patients with T2D achieving target glycemic control with SSI therapy, defined as mean hospital BG between 70 and 180 mg/dL without hypoglycemia <70 mg/dL during hospital stay. This threshold was determined based on 2019 ADA recommendations targeting hospital BG <180 mg/dL and avoidance of hypoglycemia.²³

Statistical Analysis

Patients were stratified according to continuous SSI versus transitioned to basal treatment. Patients who remained on continuous SSI were further categorized into four categories based on admission BG: <140 mg/dL, 140 to 180 mg/dL, 180 to 250 mg/dL, and ≥250 mg/dL. Clinical characteristics were compared using Wilcoxon rank-sum tests (if continuous) and chi-square tests or Fisher exact tests (if categorical). We then compared the clinical outcomes among continuous SSI patients with different admission BG levels (<140 mg/dL, 140-180 mg/dL, 180-250 mg/dL, and ≥250 mg/dL) and with different HbA1c levels (<7%, 7%-8%, 8%-9%, ≥9%). Within each scenario, logistic regression for the outcome of poor glycemic control, defined as mean hospital BG >180 mg/dL, was performed to evaluate the HbA1c levels and admission BG levels controlling for other factors (age, gender, body mass index [BMI], race, setting [medicine versus surgery] and Charlson Comorbidity Index score). A P value < .05 was regarded as statistically significant. All analyses were performed based on available cases and conducted in SAS version 9.4 (SAS Institute Inc.).

Among 25,813 adult patients with T2D, 8,095 patients (31.4%) were treated with SSI alone during the first 2 days of hospitalization. Of those patients treated with SSI, 6,903 (85%) remained on continuous SSI alone during the entire hospitalization, and 1,192 (15%) were transitioned to basal insulin. The clinical characteristics of these patients on continuous SSI and those who transitioned to basal insulin are shown in Table 1. Patients who transitioned to basal insulin had significantly higher mean (SD) admission BG (191.8 [88.2] mg/dL vs 156.4 [65.4] mg/dL, P < .001) and higher mean (SD) HbA1c (8.1% [2.0%] vs 7.01% [1.5%], P < .001), compared with those who remained on continuous SSI. Patients who transitioned to basal insulin were also younger and more likely to have chronic kidney disease (CKD), but less likely to have congestive heart failure, coronary artery disease, or chronic obstructive pulmonary disease (COPD). The Charlson Comorbidity Index score was significantly higher for patients who transitioned to basal (4.4 [2.5]) than for those who remained on continuous SSI (4.1 [2.5], P < .001). There were no significant differences among sex, BMI, or glomerular filtration rate (GFR) on admission. Of those transitioned to basal insulin, 53% achieved a mean hospitalization BG <180 mg/dL, compared with 82% of those on continuous SSI. The overall rate of hypoglycemia in the continuous SSI group was 8% compared with 18% in those transitioned to basal insulin.

Of the patients who remained on continuous SSI throughout the hospitalization, 3,319 patients (48%) had admission BG <140 mg/dL, 1,671 patients (24%) had admission BG 140 to 180 mg/dL, and 1,913 patients (28%) had admission BG >180 mg/dL. Only 9% of patients who remained on continuous SSI had admission BG ≥250 mg/dL. Patients with admission BG <140 mg/dL were older, had lower BMI and HbA1c, had higher rates of COPD and CKD, and were more likely to be admitted to a surgical service compared with patients with admission BG >140 mg/dL (P < .05 for all; Table 2).

Hospital glycemic control for patients on continuous SSI according to admission BG is displayed in Table 3. Among patients who remained on continuous SSI, 96% of patients with admission BG <140 mg/dL had a mean hospital BG <180 mg/dL; of them, 86% achieved target control without hypoglycemia. Similar rates of target control were achieved in patients with admission BG 140 to 180 mg/dL (83%), in contrast to patients with admission BG ≥250 mg/dL, of whom only 18% achieved target control (P < .001). These findings parallel those seen in patients transitioned to basal insulin. Of patients in the transition group admitted with BG <140 mg/dL and <180 mg/dL, 88.5% and 84.6% had mean hospital BG <180 mg/dL, respectively, while 69.1% and 68.9% had mean BG between 70 and 180 mg/dL without hypoglycemia. The overall frequency of hypoglycemia <70 mg/dL among patients on continuous SSI was 8% and was more common in patients with admission BG <140 mg/dL (10%) compared with patients with higher admission glucose levels (BG 140-180 mg/dL [4%], 180-250 mg/dL [4%], or ≥250 mg/dL [6%], P < .001). There was no difference in rates of severe hypoglycemia <40 mg/dL among groups.

HbA1c data were available for 2,560 of the patients on continuous SSI (Table 3). Mean hospital BG increased significantly with increasing HbA1c values. Patients admitted with HbA1c <7% had lower mean (SD) hospital BG (132.2 [28.2] mg/dL) and were more likely to achieve target glucose control during hospitalization (85%) compared with those with HbA1c 7% to 8% (mean BG, 148.7 [30.8] mg/dL; 80% target control), HbA1c 8% to 9% (mean BG, 169.1 [37.9] mg/dL; 61% target control), or HbA1c ≥9% (mean BG, 194.9 [53.4] mg/dL; 38% target control) (P < .001).

In a logistic regression analysis adjusted for age, gender, BMI, race, setting (medicine vs surgery), and Charlson Comorbidity Index score, the odds of poor glycemic control increased with higher admission BG (admission BG 140-180 mg/dL: odds ratio [OR], 1.8; 95% CI, 1.5-2.2; admission BG 180-250 mg/dL: OR, 3.7; 95% CI, 3.1-4.4; admission BG ≥250 mg/dL: OR, 7.2; 95% CI, 5.8-9.0; reference admission BG <140 mg/dL; Figure). Similarly, the logistic regression analysis showed greater odds of poor in-hospital glycemic control with increasing HbA1c (OR, 6.1; 95% CI, 4.3-8.8 for HbA1c >9% compared with HbA1c <7%).

This large retrospective cohort study examined the effectiveness of SSI for glycemic control in noncritical inpatients with T2D. Our results indicate that SSI is still widely used in our hospital system, with 31.4% of our initial cohort managed with SSI alone. We found that 86% of patients with BG <140 mg/dL and 83% of patients with BG 140 to 180 mg/dL achieved glycemic control without hypoglycemia when managed with SSI alone, compared with 53% of those admitted with BG 180 to 250 mg/dL and only 18% of those with admission BG ≥250 mg/dL. This high success rate of achieving optimal BG control with SSI alone is comparable to that seen with transition to basal insulin and may explain the prevalent use of SSI for the management of patients with T2D and mild to moderate hyperglycemia.

Published clinical guideline recommendations promoting the use of basal-bolus insulin treatment algorithms are based on the results of a few RCTs that compared the efficacy of SSI vs a basal-bolus insulin regimen. These studies reported significantly lower mean daily BG concentration with basal or basal-bolus insulin therapy compared with SSI.^10,11,24 However, it is interesting to note that the mean admission BG of patients treated with SSI in these RCTs ranged from 184 to 225 mg/dL. Patients in these trials were excluded if admission BG was <140 mg/dL.^10,11,24 This is in contrast to our study evaluating real-world data in non–critically ill settings in which we found that 48% of patients treated with SSI had admission BG <140 mg/dL, and nearly 75% had admission BG <180 mg/dL. This suggests that by nature of study design, most RCTs excluded the population of patients who do achieve good glycemic control with SSI and may have contributed to the perception that basal insulin is preferable in all populations.

Our analysis indicates that healthcare professionals should consider admission BG when selecting the type of insulin regimen to manage patients with T2D in the hospital. Our results suggest that SSI may be appropriate for many patients with admission BG <180 mg/dL and should be avoided as monotherapy in patients with admission BG ≥180 mg/dL, as the proportion of patients achieving target control decreased with increasing admission BG. More importantly, if a patient is not controlled with SSI alone, intensification of therapy with the addition of basal insulin is indicated to achieve glycemic control. In addition, we found that the admission HbA1c is an appropriate marker to consider as well, with hospital glycemic control deteriorating with increasing HbA1c values, paralleling the admission BG. The main limitation to widespread use of HbA1c for therapeutic decision-making is access to values at time of patient admission; in our population, only 37% of patients had an HbA1c value available during the index hospitalization.

Previous publications have reported that hypoglycemia carries significant safety concerns, especially among a hospitalized population.^25-27 As such, we included hypoglycemia as an important metric in our definition of target glycemic control rather than simply using mean hospital BG or number of hyperglycemic events to define treatment effectiveness. We did find a higher rate of hypoglycemia in patients with moderate admission BG treated with SSI compared with those with higher admission BG; however, few patients overall experienced clinically significant (<54 mg/dL) or severe (<40 mg/dL) hypoglycemia.

In our population, only 15% of patients started on SSI received additional basal insulin during hospitalization. This finding is similar to data reported in the Rabbit 2 trial, in which 14% of patients failed SSI alone, with a higher failure rate among those with higher BG on admission.¹⁰ Given the observational nature of this study, we cannot definitively state why certain patients in our population required additional basal insulin, but we can hypothesize that these patients admitted with BG ≥180 mg/dL had higher treatment failure rates and greater rates of hyperglycemia, therefore receiving intensified insulin therapy as clinically indicated at the discretion of the treating physician. Patients who transitioned from SSI to basal insulin had significantly higher admission BG and HbA1c compared with patients who remained on SSI alone. We noted that the rates of hypoglycemia were higher in the group that transitioned to basal (18% vs 8%) and similar to rates reported in previous RCTs.^11,24

This observational study takes advantage of a large, diverse study population and a combination of medicine and surgery patients in a real-world setting. We acknowledge several limitations in our study. Our primary data were observational in nature, and as such, some baseline patient characteristics were notably different between groups, suggesting selection bias for treatment allocation to SSI. We do not know which patients were managed by primary teams compared with specialized diabetes consult services, which may also influence treatment regimens. We did not have access to information about patients’ at-home diabetes medication regimens or duration of diabetes, both of which have been shown in prior publications to affect an individual’s overall hospital glycemic control. Data on HbA1c values were available for only approximately one-third of patients. In addition, our study did not include patients without a history of diabetes who developed stress-induced hyperglycemia, a population that may benefit from conservative therapy such as SSI.²⁸ A diagnosis of CKD was defined based on ICD 9/10 codes and not on admission estimated GFR. More specific data regarding stage of CKD or changes in renal function over the duration of hospitalization are not available, which could influence insulin prescribing practice. In addition, we defined the basal group as patients prescribed any form of basal insulin (NPH, glargine, detemir or degludec), and we do not have information on the use of prandial versus correction doses of rapid-acting insulin in the basal insulin–treated group.

In conclusion, our observational study indicates that the use of SSI results in appropriate target glycemic control for most noncritical medicine and surgery patients with admission BG <180 mg/dL. In agreement with previous RCTs, our study confirms that SSI as monotherapy is frequently inadequate in patients with significant hyperglycemia >180 mg/dL.^10,11,24,29 We propose that an individualized approach to inpatient glycemic management is imperative, and cautious use of SSI may be a viable option for certain patients with mild hyperglycemia and admission BG <180 mg/dL. Further observational and randomized studies are needed to confirm the efficacy of SSI therapy in T2D patients with mild hyperglycemia. By identifying which subset of patients can be safely managed with SSI alone, we can better understand which patients will require escalation of therapy with intensive glucose management.

Sepsis is the leading cause of in-hospital mortality in the United States.¹ Sepsis is present on admission in 85% of cases, and each hour delay in antibiotic treatment is associated with 4% to 7% increased odds of mortality.^2,3 Prompt identification and treatment of sepsis is essential for reducing morbidity and mortality, but identifying sepsis during triage is challenging.²

Risk stratification scores that rely solely on data readily available at the bedside have been developed to quickly identify those at greatest risk of poor outcomes from sepsis in real time. The quick Sequential Organ Failure Assessment (qSOFA) score, the National Early Warning System (NEWS2), and the Shock Index are easy-to-calculate measures that use routinely collected clinical data that are not subject to laboratory delay. These scores can be incorporated into electronic health record (EHR)-based alerts and can be calculated longitudinally to track the risk of poor outcomes over time. qSOFA was developed to quantify patient risk at bedside in non-intensive care unit (ICU) settings, but there is no consensus about its ability to predict adverse outcomes such as mortality and ICU admission.^4-6 The United Kingdom’s National Health Service uses NEWS2 to identify patients at risk for sepsis.⁷ NEWS has been shown to have similar or better sensitivity in identifying poorer outcomes in sepsis patients compared with systemic inflammatory response syndrome (SIRS) criteria and qSOFA.^4,8-11 However, since the latest update of NEWS2 in 2017, there has been little study of its predictive ability. The Shock Index is a simple bedside score (heart rate divided by systolic blood pressure) that was developed to detect changes in cardiovascular performance before systemic shock onset. Although it was not developed for infection and has not been regularly applied in the sepsis literature, the Shock Index might be useful for identifying patients at increased risk of poor outcomes. Patients with higher and sustained Shock Index scores are more likely to experience morbidity, such as hyperlactatemia, vasopressor use, and organ failure, and also have an increased risk of mortality.^12-14

Although the predictive abilities of these bedside risk stratification scores have been assessed individually using standard binary cut-points, the comparative performance of qSOFA, the Shock Index, and NEWS2 has not been evaluated in patients presenting to an emergency department (ED) with suspected sepsis. Our objective was to provide a head-to-head comparison of the test characteristics of qSOFA, the Shock Index, and NEWS2 calculated at ED triage for predicting in-hospital mortality and ED-to-ICU admission in patients with suspected sepsis to help health systems and providers select screening measures.

Design and Setting

We conducted a retrospective cohort study of ED patients who presented with suspected sepsis to the University of California San Francisco (UCSF) Helen Diller Medical Center at Parnassus Heights between June 1, 2012, and December 31, 2018. Our institution is a 785-bed academic teaching hospital with approximately 30,000 ED encounters per year. The study was approved with a waiver of informed consent by the UCSF Human Research Protection Program.

Participants

We use an Epic-based EHR platform (Epic 2017, Epic Systems Corporation) for clinical care, which was implemented on June 1, 2012. All data elements were obtained from Clarity, the relational database that stores Epic’s inpatient data. The study included encounters for patients age ≥18 years who had blood cultures ordered within 24 hours of ED presentation and administration of intravenous antibiotics within 24 hours. Repeat encounters were treated independently in our analysis.

Outcomes and Measures

We compared the ability of qSOFA, the Shock Index, and NEWS2 to predict in-hospital mortality and admission to the ICU from the ED (ED-to-ICU admission). We used the most abnormal vital signs and clinical assessments gathered within 30 minutes of ED presentation to identify patients who were qSOFA-positive, Shock Index-positive, and NEWS2-positive based on standard cut-points of risk. Data elements used to calculate qSOFA, Shock Index, and NEWS2 included blood pressure, heart rate, respiratory rate, Glasgow Coma Scale (GCS) score, oxygen saturation, requirement for supplemental oxygen, and temperature (Table 1). Patients were considered positive if they had a qSOFAscore ≥2, Shock Index of >0.7, and NEWS2 ≥5 based on triage vital signs.^7,15,16 We considered patients to have altered mental status, a criterion used for NEWS2, if they had a GCS score <15 instead of using the “alert, verbal, confusion, pain, unresponsive” scale, which is not captured in our EHR, a method that has been used in earlier studies.^17,18 Missing assessments were considered normal. Although our primary analysis focused on the scores calculated within 30 minutes of ED presentation, we performed a sensitivity analysis examining scores calculated within 1 hour of ED presentation in the event of a delay in gathering triage vital sign data.

We compared demographic and clinical characteristics of patients who were positive for qSOFA, the Shock Index, and NEWS2. Demographic data were extracted from the EHR and included primary language, age, sex, and insurance status. All International Classification of Diseases (ICD)-9/10 diagnosis codes were pulled from Clarity billing tables. We used the Elixhauser comorbidity groupings¹⁹ of ICD-9/10 codes present on admission to identify preexisting comorbidities and underlying organ dysfunction. To estimate burden of comorbid illnesses, we calculated the validated van Walraven comorbidity index,²⁰ which provides an estimated risk of in-hospital death based on documented Elixhauser comorbidities. Admission level of care (acute, stepdown, or intensive care) was collected for inpatient admissions to assess initial illness severity.²¹ We also evaluated discharge disposition and in-hospital mortality. Index blood culture results were collected, and dates and timestamps of mechanical ventilation, fluid, vasopressor, and antibiotic administration were obtained for the duration of the encounter.

UCSF uses an automated, real-time, algorithm-based severe sepsis alert that is triggered when a patient meets ≥2 SIRS criteria and again when the patient meets severe sepsis or septic shock criteria (ie, ≥2 SIRS criteria in addition to end-organ dysfunction and/or fluid nonresponsive hypotension). This sepsis screening alert was in use for the duration of our study.²²

Statistical Analysis

We performed a subgroup analysis among those who were diagnosed with sepsis, according to the 2016 Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria. Sepsis is defined as a change in Sequential Organ Failure Assessment (SOFA) score of ≥2 points within the first 48 hours.²³ Additionally, patients meeting Sepsis-3 criteria needed to (1) receive ≥4 days of sequential antibiotic therapy or experience death or discharge to hospice before 4 days of antibiotic therapy or (2) have a validated sepsis discharge billing code. These parameters were added to increase the specificity of our sample.²⁴

All statistical analyses were conducted using Stata 14 (StataCorp). We summarized differences in demographic and clinical characteristics among the populations meeting each severity score but elected not to conduct hypothesis testing because patients could be positive for one or more scores. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each score to predict in-hospital mortality and ED-to-ICU admission. To allow comparison with other studies, we also created a composite outcome of either in-hospital mortality or ED-to-ICU admission. To assess score discrimination to predict in-hospital mortality and ED-to-ICU admission, we calculated the area under the receiver operating characteristic curve (AUROC) along with asymptotic normal 95% CI using the “roctab” command considering a binary cut-point, as well as the full range of scores measured in the cohort. The AUROC range from 0.50 to 1.00 and a score in the 0.70 to 0.80 range can be considered fair.²⁵ We assessed significant differences between severity score AUROCs using the DeLong method²⁶ implemented through Stata 14’s “roccomp” command. As a sensitivity analysis, we explored whether the standard cut-points for qSOFA, the Shock Index, and NEWS2 provided the highest AUROC in our population by calculating test characteristics for several score cut-points.

Within our sample 23,837 ED patients had blood cultures ordered within 24 hours of ED presentation and were considered to have suspected sepsis. The mean age of the cohort was 60.8 years, and 1,612 (6.8%) had positive blood cultures. A total of 12,928 patients (54.2%) were found to have sepsis. We documented 1,427 in-hospital deaths (6.0%) and 3,149 (13.2%) ED-to-ICU admissions. At ED triage 1,921 (8.1%) were qSOFA-positive, 4,273 (17.9%) were Shock Index-positive, and 11,832 (49.6%) were NEWS2-positive. At ED triage, blood pressure, heart rate, respiratory rate, and oxygen saturated were documented in >99% of patients, 93.5% had temperature documented, and 28.5% had GCS recorded. If the window of assessment was widened to 1 hour, GCS was only documented among 44.2% of those with suspected sepsis.

Demographic Characteristics and Clinical Course

We identified significant differences when comparing demographic and clinical characteristics among patients who scored positive for the three severity measures at triage (Table 2). Although no hypothesis testing was conducted because patients could meet one or more scores, qSOFA-positive patients were older (median 70, 66, and 64 years, respectively), more likely to have Medicare as the primary payor (67.6% vs 59.7% vs 56.6%), to have chronic renal failure (26.1%, 23.1%, and 23.3%, respectively), to have a greater degree of underlying comorbidities based on the van Walraven Comorbidity Index (median 15, 12, and 11, respectively), and to be admitted to the ICU from the ED (48.1%, 36.3%, and 21.0%, respectively) compared with those positive for the Shock Index or NEWS2.

qSOFA-positive patients received antibiotics more quickly than those who were Shock Index-positive or NEWS2-positive (median 1.5, 1.8, and 2.8 hours after admission, respectively). In addition, those who were qSOFA-positive were more likely to have a positive blood culture (10.9%, 9.4%, and 8.5%, respectively) and to receive an EHR-based diagnosis of sepsis (77.0%, 69.6%, and 60.9%, respectively) than those who were Shock Index- or NEWS2-positive. Those who were qSOFA-positive also were more likely to be mechanically ventilated during their hospital stay (25.4%, 19.2%, and 10.8%, respectively) and to receive vasopressors (33.5%, 22.5%, and 12.2%, respectively). In-hospital mortality also was more common among those who were qSOFA-positive at triage (23.4%, 15.3%, and 9.2%, respectively).

Because both qSOFA and NEWS2 incorporate GCS, we explored baseline characteristics of patients with GCS documented at triage (n = 6,794). These patients were older (median age 63 and 61 years, P < .0001), more likely to be male (54.9% and 53.4%, P = .0031), more likely to have renal failure (22.8% and 20.1%, P < .0001), more likely to have liver disease (14.2% and 12.8%, P = .006), had a higher van Walraven comorbidity score on presentation (median 10 and 8, P < .0001), and were more likely to go directly to the ICU from the ED (20.2% and 10.6%, P < .0001). However, among the 6,397 GCS scores documented at triage, only 1,579 (24.7%) were abnormal.

Test Characteristics of qSOFA, Shock Index, and NEWS2 for Predicting In-hospital Mortality and ED-to-ICU Admission

Among 23,837 patients with suspected sepsis, NEWS2 had the highest sensitivity for predicting in-hospital mortality (76.0%; 95% CI, 73.7%-78.2%) and ED-to-ICU admission (78.9%; 95% CI, 77.5%-80.4%) but had the lowest specificity for in-hospital mortality (52.0%; 95% CI, 51.4%-52.7%) and for ED-to-ICU admission (54.8%; 95% CI, 54.1%-55.5%) (Table 3). qSOFA had the lowest sensitivity for in-hospital mortality (31.5%; 95% CI, 29.1%-33.9%) and ED-to-ICU admission (29.3%; 95% CI, 27.7%-30.9%) but the highest specificity for in-hospital mortality (93.4%; 95% CI, 93.1%-93.8%) and ED-to-ICU admission (95.2%; 95% CI, 94.9%-95.5%). The Shock Index had a sensitivity that fell between qSOFA and NEWS2 for in-hospital mortality (45.8%; 95% CI, 43.2%-48.5%) and ED-to-ICU admission (49.2%; 95% CI, 47.5%-51.0%). The specificity of the Shock Index also was between qSOFA and NEWS2 for in-hospital mortality (83.9%; 95% CI, 83.4%-84.3%) and ED-to-ICU admission (86.8%; 95% CI, 86.4%-87.3%). All three scores exhibited relatively low PPV, ranging from 9.2% to 23.4% for in-hospital mortality and 21.0% to 48.0% for ED-to-ICU triage. Conversely, all three scores exhibited relatively high NPV, ranging from 95.5% to 97.1% for in-hospital mortality and 89.8% to 94.5% for ED-to-ICU triage. The patterns in sensitivity and specificity for in-hospital mortality and ED-to-ICU admission were similar among the 12,928 patients who received an EHR-based sepsis diagnosis with the tests generally demonstrating lower specificities, higher PPVs, and lower NPVs (Table 3).

When considering a binary cutoff, the Shock Index exhibited the highest AUROC for in-hospital mortality (0.648; 95% CI, 0.635-0.662) and had a significantly higher AUROC than qSOFA (AUROC, 0.625; 95% CI, 0.612-0.637; P = .0005), but there was no difference compared with NEWS2 (AUROC, 0.640; 95% CI, 0.628-0.652; P = .2112). NEWS2 had a significantly higher AUROC than qSOFA for predicting in-hospital mortality (P = .0227). The Shock Index also exhibited the highest AUROC for ED-to-ICU admission (0.680; 95% CI, 0.617-0.689), which was significantly higher than the AUROC for qSOFA (P < .0001) and NEWS2 (P = 0.0151). NEWS2 had a significantly higher AUROC than qSOFA for predicting ED-to-ICU admission (P < .0001). Similar findings were seen in patients found to have sepsis. When considering the range of possible scores measured in our cohort, qSOFA and NEWS2 exhibited higher AUROCs for in-hospital mortality and ED-to-ICU admission than the Shock Index among patients with suspected infection and the subgroup with a sepsis diagnosis (Figure).

In this retrospective cohort study of 23,837 patients who presented to the ED with suspected sepsis, the standard qSOFA threshold was met least frequently, followed by the Shock Index and NEWS2. NEWS2 had the highest sensitivity but the lowest specificity for predicting in-hospital mortality and ED-to-ICU admission, making it a challenging bedside risk stratification scale for identifying patients at risk of poor clinical outcomes. When comparing predictive performance among the three scales, qSOFA had the highest specificity and the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission in this cohort of patients with suspected sepsis. These trends in sensitivity, specificity, and AUROC were consistent among those who met EHR criteria for a sepsis diagnosis. In the analysis of the three scoring systems using all available cut-points, qSOFA and NEWS2 had the highest AUROCs, followed by the Shock Index.

Considering the rapid progression from organ dysfunction to death in sepsis patients, as well as the difficulty establishing a sepsis diagnosis at triage,²³ providers must quickly identify patients at increased risk of poor outcomes when they present to the ED. Sepsis alerts often are built using SIRS criteria,²⁷ including the one used for sepsis surveillance at UCSF since 2012,²² but the white blood cell count criterion is subject to a laboratory lag and could lead to a delay in identification. Implementation of a point-of-care bedside score alert that uses readily available clinical data could allow providers to identify patients at greatest risk of poor outcomes immediately at ED presentation and triage, which motivated us to explore the predictive performance of qSOFA, the Shock Index, and NEWS2.

Our study is the first to provide a head-to-head comparison of the predictive performance of qSOFA, the Shock Index, and NEWS2, three easy-to-calculate bedside risk scores that use EHR data collected among patients with suspected sepsis. The Sepsis-3 guidelines recommend qSOFA to quickly identify non-ICU patients at greatest risk of poor outcomes because the measure exhibited predictive performance similar to the more extensive SOFA score outside the ICU.^16,23 Although some studies have confirmed qSOFA’s high predictive performance,^28-31 our test characteristics and AUROC findings are in line with other published analyses.^4,6,10,17 The UK National Health Service is using NEWS2 to screen for patients at risk of poor outcomes from sepsis. Several analyses that assessed the predictive ability of NEWS have reported estimates in line with our findings.^4,10,32 The Shock Index was introduced in 1967 and provided a metric to evaluate hemodynamic stability based on heart rate and systolic blood pressure.³³ The Shock Index has been studied in several contexts, including sepsis,³⁴ and studies show that a sustained Shock Index is associated with increased odds of vasopressor administration, higher prevalence of hyperlactatemia, and increased risk of poor outcomes in the ICU.^13,14

For our study, we were particularly interested in exploring how the Shock Index would compare with more frequently used severity scores such as qSOFA and NEWS2 among patients with suspected sepsis, given the simplicity of its calculation and the easy availability of required data. In our cohort of 23,837 patients, only 159 people had missing blood pressure and only 71 had omitted heart rate. In contrast, both qSOFA and NEWS2 include an assessment of level of consciousness that can be subject to variability in assessment methods and EHR documentation across institutions.¹¹ In our cohort, GCS within 30 minutes of ED presentation was missing in 72 patients, which could have led to incomplete calculation of qSOFA and NEWS2 if a missing value was not actually within normal limits.

Several investigations relate qSOFA to NEWS but few compare qSOFA with the newer NEWS2, and even fewer evaluate the Shock Index with any of these scores.^{10,11,18,29,35-37} In general, studies have shown that NEWS exhibits a higher AUROC for predicting mortality, sepsis with organ dysfunction, and ICU admission, often as a composite outcome.^{4,11,18,37,38} A handful of studies compare the Shock Index to SIRS; however, little has been done to compare the Shock Index to qSOFA or NEWS2, scores that have been used specifically for sepsis and might be more predictive of poor outcomes than SIRS.³³ In our study, the Shock Index had a higher AUROC than either qSOFA or NEWS2 for predicting in-hospital mortality and ED-to-ICU admission measured as separate outcomes and as a composite outcome using standard cut-points for these scores.

When selecting a severity score to apply in an institution, it is important to carefully evaluate the score’s test characteristics, in addition to considering the availability of reliable data. Tests with high sensitivity and NPV for the population being studied can be useful to rule out disease or risk of poor outcome, while tests with high specificity and PPV can be useful to rule in disease or risk of poor outcome.³⁹ When considering specificity, qSOFA’s performance was superior to the Shock Index and NEWS2 in our study, but a small percentage of the population was identified using a cut-point of qSOFA ≥2. If we used qSOFA and applied this standard cut-point at our institution, we could be confident that those identified were at increased risk, but we would miss a significant number of patients who would experience a poor outcome. When considering sensitivity, performance of NEWS2 was superior to qSOFA and the Shock Index in our study, but one-half of the population was identified using a cut-point of NEWS2 ≥5. If we were to apply this standard NEWS2 cut-point at our institution, we would assume that one-half of our population was at risk, which might drive resource use towards patients who will not experience a poor outcome. Although none of the scores exhibited a robust AUROC measure, the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission when using the standard binary cut-point, and its sensitivity and specificity is between that of qSOFA and NEWS2, potentially making it a score to use in settings where qSOFA and NEWS2 score components, such as altered mentation, are not reliably collected. Finally, our sensitivity analysis varying the binary cut-point of each score within our population demonstrated that the standard cut-points might not be as useful within a specific population and might need to be tailored for implementation, balancing sensitivity, specificity, PPV, and NPV to meet local priorities and ICU capacity.

Our study has limitations. It is a single-center, retrospective analysis, factors that could reduce generalizability. However, it does include a large and diverse patient population spanning several years. Missing GCS data could have affected the predictive ability of qSOFA and NEWS2 in our cohort. We could not reliably perform imputation of GCS because of the high missingness and therefore we assumed missing was normal, as was done in the Sepsis-3 derivation studies.¹⁶ Previous studies have attempted to impute GCS and have not observed improved performance of qSOFA to predict mortality.⁴⁰ Because manually collected variables such as GCS are less reliably documented in the EHR, there might be limitations in their use for triage risk scores.

Although the current analysis focused on the predictive performance of qSOFA, the Shock Index, and NEWS2 at triage, performance of these scores could affect the ED team’s treatment decisions before handoff to the hospitalist team and the expected level of care the patient will receive after in-patient admission. These tests also have the advantage of being easy to calculate at the bedside over time, which could provide an objective assessment of longitudinal predicted prognosis. Future work should assess the longitudinal performance of each of these scores among those with suspected sepsis and to determine the impact using these scores would have on clinical and resource utilization outcomes.

Local priorities should drive selection of a screening tool, balancing sensitivity, specificity, PPV, and NPV to achieve the institution’s goals. qSOFA, Shock Index, and NEWS2 are risk stratification tools that can be easily implemented at ED triage using data available at the bedside. Although none of these scores performed strongly when comparing AUROCs, qSOFA was highly specific for identifying patients with poor outcomes, and NEWS2 was the most sensitive for ruling out those at high risk among patients with suspected sepsis. The Shock Index exhibited a sensitivity and specificity that fell between qSOFA and NEWS2 and also might be considered to identify those at increased risk, given its ease of implementation, particularly in settings where altered mentation is unreliably or inconsistently documented.

The authors thank the UCSF Division of Hospital Medicine Data Core for their assistance with data acquisition.

Sepsis is the leading cause of in-hospital mortality in the United States.¹ Sepsis is present on admission in 85% of cases, and each hour delay in antibiotic treatment is associated with 4% to 7% increased odds of mortality.^2,3 Prompt identification and treatment of sepsis is essential for reducing morbidity and mortality, but identifying sepsis during triage is challenging.²

Risk stratification scores that rely solely on data readily available at the bedside have been developed to quickly identify those at greatest risk of poor outcomes from sepsis in real time. The quick Sequential Organ Failure Assessment (qSOFA) score, the National Early Warning System (NEWS2), and the Shock Index are easy-to-calculate measures that use routinely collected clinical data that are not subject to laboratory delay. These scores can be incorporated into electronic health record (EHR)-based alerts and can be calculated longitudinally to track the risk of poor outcomes over time. qSOFA was developed to quantify patient risk at bedside in non-intensive care unit (ICU) settings, but there is no consensus about its ability to predict adverse outcomes such as mortality and ICU admission.^4-6 The United Kingdom’s National Health Service uses NEWS2 to identify patients at risk for sepsis.⁷ NEWS has been shown to have similar or better sensitivity in identifying poorer outcomes in sepsis patients compared with systemic inflammatory response syndrome (SIRS) criteria and qSOFA.^4,8-11 However, since the latest update of NEWS2 in 2017, there has been little study of its predictive ability. The Shock Index is a simple bedside score (heart rate divided by systolic blood pressure) that was developed to detect changes in cardiovascular performance before systemic shock onset. Although it was not developed for infection and has not been regularly applied in the sepsis literature, the Shock Index might be useful for identifying patients at increased risk of poor outcomes. Patients with higher and sustained Shock Index scores are more likely to experience morbidity, such as hyperlactatemia, vasopressor use, and organ failure, and also have an increased risk of mortality.^12-14

Although the predictive abilities of these bedside risk stratification scores have been assessed individually using standard binary cut-points, the comparative performance of qSOFA, the Shock Index, and NEWS2 has not been evaluated in patients presenting to an emergency department (ED) with suspected sepsis. Our objective was to provide a head-to-head comparison of the test characteristics of qSOFA, the Shock Index, and NEWS2 calculated at ED triage for predicting in-hospital mortality and ED-to-ICU admission in patients with suspected sepsis to help health systems and providers select screening measures.

Design and Setting

We conducted a retrospective cohort study of ED patients who presented with suspected sepsis to the University of California San Francisco (UCSF) Helen Diller Medical Center at Parnassus Heights between June 1, 2012, and December 31, 2018. Our institution is a 785-bed academic teaching hospital with approximately 30,000 ED encounters per year. The study was approved with a waiver of informed consent by the UCSF Human Research Protection Program.

Participants

We use an Epic-based EHR platform (Epic 2017, Epic Systems Corporation) for clinical care, which was implemented on June 1, 2012. All data elements were obtained from Clarity, the relational database that stores Epic’s inpatient data. The study included encounters for patients age ≥18 years who had blood cultures ordered within 24 hours of ED presentation and administration of intravenous antibiotics within 24 hours. Repeat encounters were treated independently in our analysis.

Outcomes and Measures

We compared the ability of qSOFA, the Shock Index, and NEWS2 to predict in-hospital mortality and admission to the ICU from the ED (ED-to-ICU admission). We used the most abnormal vital signs and clinical assessments gathered within 30 minutes of ED presentation to identify patients who were qSOFA-positive, Shock Index-positive, and NEWS2-positive based on standard cut-points of risk. Data elements used to calculate qSOFA, Shock Index, and NEWS2 included blood pressure, heart rate, respiratory rate, Glasgow Coma Scale (GCS) score, oxygen saturation, requirement for supplemental oxygen, and temperature (Table 1). Patients were considered positive if they had a qSOFAscore ≥2, Shock Index of >0.7, and NEWS2 ≥5 based on triage vital signs.^7,15,16 We considered patients to have altered mental status, a criterion used for NEWS2, if they had a GCS score <15 instead of using the “alert, verbal, confusion, pain, unresponsive” scale, which is not captured in our EHR, a method that has been used in earlier studies.^17,18 Missing assessments were considered normal. Although our primary analysis focused on the scores calculated within 30 minutes of ED presentation, we performed a sensitivity analysis examining scores calculated within 1 hour of ED presentation in the event of a delay in gathering triage vital sign data.

We compared demographic and clinical characteristics of patients who were positive for qSOFA, the Shock Index, and NEWS2. Demographic data were extracted from the EHR and included primary language, age, sex, and insurance status. All International Classification of Diseases (ICD)-9/10 diagnosis codes were pulled from Clarity billing tables. We used the Elixhauser comorbidity groupings¹⁹ of ICD-9/10 codes present on admission to identify preexisting comorbidities and underlying organ dysfunction. To estimate burden of comorbid illnesses, we calculated the validated van Walraven comorbidity index,²⁰ which provides an estimated risk of in-hospital death based on documented Elixhauser comorbidities. Admission level of care (acute, stepdown, or intensive care) was collected for inpatient admissions to assess initial illness severity.²¹ We also evaluated discharge disposition and in-hospital mortality. Index blood culture results were collected, and dates and timestamps of mechanical ventilation, fluid, vasopressor, and antibiotic administration were obtained for the duration of the encounter.

UCSF uses an automated, real-time, algorithm-based severe sepsis alert that is triggered when a patient meets ≥2 SIRS criteria and again when the patient meets severe sepsis or septic shock criteria (ie, ≥2 SIRS criteria in addition to end-organ dysfunction and/or fluid nonresponsive hypotension). This sepsis screening alert was in use for the duration of our study.²²

Statistical Analysis

We performed a subgroup analysis among those who were diagnosed with sepsis, according to the 2016 Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria. Sepsis is defined as a change in Sequential Organ Failure Assessment (SOFA) score of ≥2 points within the first 48 hours.²³ Additionally, patients meeting Sepsis-3 criteria needed to (1) receive ≥4 days of sequential antibiotic therapy or experience death or discharge to hospice before 4 days of antibiotic therapy or (2) have a validated sepsis discharge billing code. These parameters were added to increase the specificity of our sample.²⁴

All statistical analyses were conducted using Stata 14 (StataCorp). We summarized differences in demographic and clinical characteristics among the populations meeting each severity score but elected not to conduct hypothesis testing because patients could be positive for one or more scores. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each score to predict in-hospital mortality and ED-to-ICU admission. To allow comparison with other studies, we also created a composite outcome of either in-hospital mortality or ED-to-ICU admission. To assess score discrimination to predict in-hospital mortality and ED-to-ICU admission, we calculated the area under the receiver operating characteristic curve (AUROC) along with asymptotic normal 95% CI using the “roctab” command considering a binary cut-point, as well as the full range of scores measured in the cohort. The AUROC range from 0.50 to 1.00 and a score in the 0.70 to 0.80 range can be considered fair.²⁵ We assessed significant differences between severity score AUROCs using the DeLong method²⁶ implemented through Stata 14’s “roccomp” command. As a sensitivity analysis, we explored whether the standard cut-points for qSOFA, the Shock Index, and NEWS2 provided the highest AUROC in our population by calculating test characteristics for several score cut-points.

Within our sample 23,837 ED patients had blood cultures ordered within 24 hours of ED presentation and were considered to have suspected sepsis. The mean age of the cohort was 60.8 years, and 1,612 (6.8%) had positive blood cultures. A total of 12,928 patients (54.2%) were found to have sepsis. We documented 1,427 in-hospital deaths (6.0%) and 3,149 (13.2%) ED-to-ICU admissions. At ED triage 1,921 (8.1%) were qSOFA-positive, 4,273 (17.9%) were Shock Index-positive, and 11,832 (49.6%) were NEWS2-positive. At ED triage, blood pressure, heart rate, respiratory rate, and oxygen saturated were documented in >99% of patients, 93.5% had temperature documented, and 28.5% had GCS recorded. If the window of assessment was widened to 1 hour, GCS was only documented among 44.2% of those with suspected sepsis.

Demographic Characteristics and Clinical Course

We identified significant differences when comparing demographic and clinical characteristics among patients who scored positive for the three severity measures at triage (Table 2). Although no hypothesis testing was conducted because patients could meet one or more scores, qSOFA-positive patients were older (median 70, 66, and 64 years, respectively), more likely to have Medicare as the primary payor (67.6% vs 59.7% vs 56.6%), to have chronic renal failure (26.1%, 23.1%, and 23.3%, respectively), to have a greater degree of underlying comorbidities based on the van Walraven Comorbidity Index (median 15, 12, and 11, respectively), and to be admitted to the ICU from the ED (48.1%, 36.3%, and 21.0%, respectively) compared with those positive for the Shock Index or NEWS2.

qSOFA-positive patients received antibiotics more quickly than those who were Shock Index-positive or NEWS2-positive (median 1.5, 1.8, and 2.8 hours after admission, respectively). In addition, those who were qSOFA-positive were more likely to have a positive blood culture (10.9%, 9.4%, and 8.5%, respectively) and to receive an EHR-based diagnosis of sepsis (77.0%, 69.6%, and 60.9%, respectively) than those who were Shock Index- or NEWS2-positive. Those who were qSOFA-positive also were more likely to be mechanically ventilated during their hospital stay (25.4%, 19.2%, and 10.8%, respectively) and to receive vasopressors (33.5%, 22.5%, and 12.2%, respectively). In-hospital mortality also was more common among those who were qSOFA-positive at triage (23.4%, 15.3%, and 9.2%, respectively).

Because both qSOFA and NEWS2 incorporate GCS, we explored baseline characteristics of patients with GCS documented at triage (n = 6,794). These patients were older (median age 63 and 61 years, P < .0001), more likely to be male (54.9% and 53.4%, P = .0031), more likely to have renal failure (22.8% and 20.1%, P < .0001), more likely to have liver disease (14.2% and 12.8%, P = .006), had a higher van Walraven comorbidity score on presentation (median 10 and 8, P < .0001), and were more likely to go directly to the ICU from the ED (20.2% and 10.6%, P < .0001). However, among the 6,397 GCS scores documented at triage, only 1,579 (24.7%) were abnormal.

Test Characteristics of qSOFA, Shock Index, and NEWS2 for Predicting In-hospital Mortality and ED-to-ICU Admission

Among 23,837 patients with suspected sepsis, NEWS2 had the highest sensitivity for predicting in-hospital mortality (76.0%; 95% CI, 73.7%-78.2%) and ED-to-ICU admission (78.9%; 95% CI, 77.5%-80.4%) but had the lowest specificity for in-hospital mortality (52.0%; 95% CI, 51.4%-52.7%) and for ED-to-ICU admission (54.8%; 95% CI, 54.1%-55.5%) (Table 3). qSOFA had the lowest sensitivity for in-hospital mortality (31.5%; 95% CI, 29.1%-33.9%) and ED-to-ICU admission (29.3%; 95% CI, 27.7%-30.9%) but the highest specificity for in-hospital mortality (93.4%; 95% CI, 93.1%-93.8%) and ED-to-ICU admission (95.2%; 95% CI, 94.9%-95.5%). The Shock Index had a sensitivity that fell between qSOFA and NEWS2 for in-hospital mortality (45.8%; 95% CI, 43.2%-48.5%) and ED-to-ICU admission (49.2%; 95% CI, 47.5%-51.0%). The specificity of the Shock Index also was between qSOFA and NEWS2 for in-hospital mortality (83.9%; 95% CI, 83.4%-84.3%) and ED-to-ICU admission (86.8%; 95% CI, 86.4%-87.3%). All three scores exhibited relatively low PPV, ranging from 9.2% to 23.4% for in-hospital mortality and 21.0% to 48.0% for ED-to-ICU triage. Conversely, all three scores exhibited relatively high NPV, ranging from 95.5% to 97.1% for in-hospital mortality and 89.8% to 94.5% for ED-to-ICU triage. The patterns in sensitivity and specificity for in-hospital mortality and ED-to-ICU admission were similar among the 12,928 patients who received an EHR-based sepsis diagnosis with the tests generally demonstrating lower specificities, higher PPVs, and lower NPVs (Table 3).

When considering a binary cutoff, the Shock Index exhibited the highest AUROC for in-hospital mortality (0.648; 95% CI, 0.635-0.662) and had a significantly higher AUROC than qSOFA (AUROC, 0.625; 95% CI, 0.612-0.637; P = .0005), but there was no difference compared with NEWS2 (AUROC, 0.640; 95% CI, 0.628-0.652; P = .2112). NEWS2 had a significantly higher AUROC than qSOFA for predicting in-hospital mortality (P = .0227). The Shock Index also exhibited the highest AUROC for ED-to-ICU admission (0.680; 95% CI, 0.617-0.689), which was significantly higher than the AUROC for qSOFA (P < .0001) and NEWS2 (P = 0.0151). NEWS2 had a significantly higher AUROC than qSOFA for predicting ED-to-ICU admission (P < .0001). Similar findings were seen in patients found to have sepsis. When considering the range of possible scores measured in our cohort, qSOFA and NEWS2 exhibited higher AUROCs for in-hospital mortality and ED-to-ICU admission than the Shock Index among patients with suspected infection and the subgroup with a sepsis diagnosis (Figure).

In this retrospective cohort study of 23,837 patients who presented to the ED with suspected sepsis, the standard qSOFA threshold was met least frequently, followed by the Shock Index and NEWS2. NEWS2 had the highest sensitivity but the lowest specificity for predicting in-hospital mortality and ED-to-ICU admission, making it a challenging bedside risk stratification scale for identifying patients at risk of poor clinical outcomes. When comparing predictive performance among the three scales, qSOFA had the highest specificity and the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission in this cohort of patients with suspected sepsis. These trends in sensitivity, specificity, and AUROC were consistent among those who met EHR criteria for a sepsis diagnosis. In the analysis of the three scoring systems using all available cut-points, qSOFA and NEWS2 had the highest AUROCs, followed by the Shock Index.

Considering the rapid progression from organ dysfunction to death in sepsis patients, as well as the difficulty establishing a sepsis diagnosis at triage,²³ providers must quickly identify patients at increased risk of poor outcomes when they present to the ED. Sepsis alerts often are built using SIRS criteria,²⁷ including the one used for sepsis surveillance at UCSF since 2012,²² but the white blood cell count criterion is subject to a laboratory lag and could lead to a delay in identification. Implementation of a point-of-care bedside score alert that uses readily available clinical data could allow providers to identify patients at greatest risk of poor outcomes immediately at ED presentation and triage, which motivated us to explore the predictive performance of qSOFA, the Shock Index, and NEWS2.

Our study is the first to provide a head-to-head comparison of the predictive performance of qSOFA, the Shock Index, and NEWS2, three easy-to-calculate bedside risk scores that use EHR data collected among patients with suspected sepsis. The Sepsis-3 guidelines recommend qSOFA to quickly identify non-ICU patients at greatest risk of poor outcomes because the measure exhibited predictive performance similar to the more extensive SOFA score outside the ICU.^16,23 Although some studies have confirmed qSOFA’s high predictive performance,^28-31 our test characteristics and AUROC findings are in line with other published analyses.^4,6,10,17 The UK National Health Service is using NEWS2 to screen for patients at risk of poor outcomes from sepsis. Several analyses that assessed the predictive ability of NEWS have reported estimates in line with our findings.^4,10,32 The Shock Index was introduced in 1967 and provided a metric to evaluate hemodynamic stability based on heart rate and systolic blood pressure.³³ The Shock Index has been studied in several contexts, including sepsis,³⁴ and studies show that a sustained Shock Index is associated with increased odds of vasopressor administration, higher prevalence of hyperlactatemia, and increased risk of poor outcomes in the ICU.^13,14

For our study, we were particularly interested in exploring how the Shock Index would compare with more frequently used severity scores such as qSOFA and NEWS2 among patients with suspected sepsis, given the simplicity of its calculation and the easy availability of required data. In our cohort of 23,837 patients, only 159 people had missing blood pressure and only 71 had omitted heart rate. In contrast, both qSOFA and NEWS2 include an assessment of level of consciousness that can be subject to variability in assessment methods and EHR documentation across institutions.¹¹ In our cohort, GCS within 30 minutes of ED presentation was missing in 72 patients, which could have led to incomplete calculation of qSOFA and NEWS2 if a missing value was not actually within normal limits.

Several investigations relate qSOFA to NEWS but few compare qSOFA with the newer NEWS2, and even fewer evaluate the Shock Index with any of these scores.^{10,11,18,29,35-37} In general, studies have shown that NEWS exhibits a higher AUROC for predicting mortality, sepsis with organ dysfunction, and ICU admission, often as a composite outcome.^{4,11,18,37,38} A handful of studies compare the Shock Index to SIRS; however, little has been done to compare the Shock Index to qSOFA or NEWS2, scores that have been used specifically for sepsis and might be more predictive of poor outcomes than SIRS.³³ In our study, the Shock Index had a higher AUROC than either qSOFA or NEWS2 for predicting in-hospital mortality and ED-to-ICU admission measured as separate outcomes and as a composite outcome using standard cut-points for these scores.

When selecting a severity score to apply in an institution, it is important to carefully evaluate the score’s test characteristics, in addition to considering the availability of reliable data. Tests with high sensitivity and NPV for the population being studied can be useful to rule out disease or risk of poor outcome, while tests with high specificity and PPV can be useful to rule in disease or risk of poor outcome.³⁹ When considering specificity, qSOFA’s performance was superior to the Shock Index and NEWS2 in our study, but a small percentage of the population was identified using a cut-point of qSOFA ≥2. If we used qSOFA and applied this standard cut-point at our institution, we could be confident that those identified were at increased risk, but we would miss a significant number of patients who would experience a poor outcome. When considering sensitivity, performance of NEWS2 was superior to qSOFA and the Shock Index in our study, but one-half of the population was identified using a cut-point of NEWS2 ≥5. If we were to apply this standard NEWS2 cut-point at our institution, we would assume that one-half of our population was at risk, which might drive resource use towards patients who will not experience a poor outcome. Although none of the scores exhibited a robust AUROC measure, the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission when using the standard binary cut-point, and its sensitivity and specificity is between that of qSOFA and NEWS2, potentially making it a score to use in settings where qSOFA and NEWS2 score components, such as altered mentation, are not reliably collected. Finally, our sensitivity analysis varying the binary cut-point of each score within our population demonstrated that the standard cut-points might not be as useful within a specific population and might need to be tailored for implementation, balancing sensitivity, specificity, PPV, and NPV to meet local priorities and ICU capacity.

Our study has limitations. It is a single-center, retrospective analysis, factors that could reduce generalizability. However, it does include a large and diverse patient population spanning several years. Missing GCS data could have affected the predictive ability of qSOFA and NEWS2 in our cohort. We could not reliably perform imputation of GCS because of the high missingness and therefore we assumed missing was normal, as was done in the Sepsis-3 derivation studies.¹⁶ Previous studies have attempted to impute GCS and have not observed improved performance of qSOFA to predict mortality.⁴⁰ Because manually collected variables such as GCS are less reliably documented in the EHR, there might be limitations in their use for triage risk scores.

Although the current analysis focused on the predictive performance of qSOFA, the Shock Index, and NEWS2 at triage, performance of these scores could affect the ED team’s treatment decisions before handoff to the hospitalist team and the expected level of care the patient will receive after in-patient admission. These tests also have the advantage of being easy to calculate at the bedside over time, which could provide an objective assessment of longitudinal predicted prognosis. Future work should assess the longitudinal performance of each of these scores among those with suspected sepsis and to determine the impact using these scores would have on clinical and resource utilization outcomes.

Local priorities should drive selection of a screening tool, balancing sensitivity, specificity, PPV, and NPV to achieve the institution’s goals. qSOFA, Shock Index, and NEWS2 are risk stratification tools that can be easily implemented at ED triage using data available at the bedside. Although none of these scores performed strongly when comparing AUROCs, qSOFA was highly specific for identifying patients with poor outcomes, and NEWS2 was the most sensitive for ruling out those at high risk among patients with suspected sepsis. The Shock Index exhibited a sensitivity and specificity that fell between qSOFA and NEWS2 and also might be considered to identify those at increased risk, given its ease of implementation, particularly in settings where altered mentation is unreliably or inconsistently documented.

The authors thank the UCSF Division of Hospital Medicine Data Core for their assistance with data acquisition.

Sepsis is the leading cause of in-hospital mortality in the United States.¹ Sepsis is present on admission in 85% of cases, and each hour delay in antibiotic treatment is associated with 4% to 7% increased odds of mortality.^2,3 Prompt identification and treatment of sepsis is essential for reducing morbidity and mortality, but identifying sepsis during triage is challenging.²

Risk stratification scores that rely solely on data readily available at the bedside have been developed to quickly identify those at greatest risk of poor outcomes from sepsis in real time. The quick Sequential Organ Failure Assessment (qSOFA) score, the National Early Warning System (NEWS2), and the Shock Index are easy-to-calculate measures that use routinely collected clinical data that are not subject to laboratory delay. These scores can be incorporated into electronic health record (EHR)-based alerts and can be calculated longitudinally to track the risk of poor outcomes over time. qSOFA was developed to quantify patient risk at bedside in non-intensive care unit (ICU) settings, but there is no consensus about its ability to predict adverse outcomes such as mortality and ICU admission.^4-6 The United Kingdom’s National Health Service uses NEWS2 to identify patients at risk for sepsis.⁷ NEWS has been shown to have similar or better sensitivity in identifying poorer outcomes in sepsis patients compared with systemic inflammatory response syndrome (SIRS) criteria and qSOFA.^4,8-11 However, since the latest update of NEWS2 in 2017, there has been little study of its predictive ability. The Shock Index is a simple bedside score (heart rate divided by systolic blood pressure) that was developed to detect changes in cardiovascular performance before systemic shock onset. Although it was not developed for infection and has not been regularly applied in the sepsis literature, the Shock Index might be useful for identifying patients at increased risk of poor outcomes. Patients with higher and sustained Shock Index scores are more likely to experience morbidity, such as hyperlactatemia, vasopressor use, and organ failure, and also have an increased risk of mortality.^12-14

Although the predictive abilities of these bedside risk stratification scores have been assessed individually using standard binary cut-points, the comparative performance of qSOFA, the Shock Index, and NEWS2 has not been evaluated in patients presenting to an emergency department (ED) with suspected sepsis. Our objective was to provide a head-to-head comparison of the test characteristics of qSOFA, the Shock Index, and NEWS2 calculated at ED triage for predicting in-hospital mortality and ED-to-ICU admission in patients with suspected sepsis to help health systems and providers select screening measures.

Design and Setting

We conducted a retrospective cohort study of ED patients who presented with suspected sepsis to the University of California San Francisco (UCSF) Helen Diller Medical Center at Parnassus Heights between June 1, 2012, and December 31, 2018. Our institution is a 785-bed academic teaching hospital with approximately 30,000 ED encounters per year. The study was approved with a waiver of informed consent by the UCSF Human Research Protection Program.

Participants

We use an Epic-based EHR platform (Epic 2017, Epic Systems Corporation) for clinical care, which was implemented on June 1, 2012. All data elements were obtained from Clarity, the relational database that stores Epic’s inpatient data. The study included encounters for patients age ≥18 years who had blood cultures ordered within 24 hours of ED presentation and administration of intravenous antibiotics within 24 hours. Repeat encounters were treated independently in our analysis.

Outcomes and Measures

We compared the ability of qSOFA, the Shock Index, and NEWS2 to predict in-hospital mortality and admission to the ICU from the ED (ED-to-ICU admission). We used the most abnormal vital signs and clinical assessments gathered within 30 minutes of ED presentation to identify patients who were qSOFA-positive, Shock Index-positive, and NEWS2-positive based on standard cut-points of risk. Data elements used to calculate qSOFA, Shock Index, and NEWS2 included blood pressure, heart rate, respiratory rate, Glasgow Coma Scale (GCS) score, oxygen saturation, requirement for supplemental oxygen, and temperature (Table 1). Patients were considered positive if they had a qSOFAscore ≥2, Shock Index of >0.7, and NEWS2 ≥5 based on triage vital signs.^7,15,16 We considered patients to have altered mental status, a criterion used for NEWS2, if they had a GCS score <15 instead of using the “alert, verbal, confusion, pain, unresponsive” scale, which is not captured in our EHR, a method that has been used in earlier studies.^17,18 Missing assessments were considered normal. Although our primary analysis focused on the scores calculated within 30 minutes of ED presentation, we performed a sensitivity analysis examining scores calculated within 1 hour of ED presentation in the event of a delay in gathering triage vital sign data.

We compared demographic and clinical characteristics of patients who were positive for qSOFA, the Shock Index, and NEWS2. Demographic data were extracted from the EHR and included primary language, age, sex, and insurance status. All International Classification of Diseases (ICD)-9/10 diagnosis codes were pulled from Clarity billing tables. We used the Elixhauser comorbidity groupings¹⁹ of ICD-9/10 codes present on admission to identify preexisting comorbidities and underlying organ dysfunction. To estimate burden of comorbid illnesses, we calculated the validated van Walraven comorbidity index,²⁰ which provides an estimated risk of in-hospital death based on documented Elixhauser comorbidities. Admission level of care (acute, stepdown, or intensive care) was collected for inpatient admissions to assess initial illness severity.²¹ We also evaluated discharge disposition and in-hospital mortality. Index blood culture results were collected, and dates and timestamps of mechanical ventilation, fluid, vasopressor, and antibiotic administration were obtained for the duration of the encounter.

UCSF uses an automated, real-time, algorithm-based severe sepsis alert that is triggered when a patient meets ≥2 SIRS criteria and again when the patient meets severe sepsis or septic shock criteria (ie, ≥2 SIRS criteria in addition to end-organ dysfunction and/or fluid nonresponsive hypotension). This sepsis screening alert was in use for the duration of our study.²²

Statistical Analysis

We performed a subgroup analysis among those who were diagnosed with sepsis, according to the 2016 Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria. Sepsis is defined as a change in Sequential Organ Failure Assessment (SOFA) score of ≥2 points within the first 48 hours.²³ Additionally, patients meeting Sepsis-3 criteria needed to (1) receive ≥4 days of sequential antibiotic therapy or experience death or discharge to hospice before 4 days of antibiotic therapy or (2) have a validated sepsis discharge billing code. These parameters were added to increase the specificity of our sample.²⁴

All statistical analyses were conducted using Stata 14 (StataCorp). We summarized differences in demographic and clinical characteristics among the populations meeting each severity score but elected not to conduct hypothesis testing because patients could be positive for one or more scores. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each score to predict in-hospital mortality and ED-to-ICU admission. To allow comparison with other studies, we also created a composite outcome of either in-hospital mortality or ED-to-ICU admission. To assess score discrimination to predict in-hospital mortality and ED-to-ICU admission, we calculated the area under the receiver operating characteristic curve (AUROC) along with asymptotic normal 95% CI using the “roctab” command considering a binary cut-point, as well as the full range of scores measured in the cohort. The AUROC range from 0.50 to 1.00 and a score in the 0.70 to 0.80 range can be considered fair.²⁵ We assessed significant differences between severity score AUROCs using the DeLong method²⁶ implemented through Stata 14’s “roccomp” command. As a sensitivity analysis, we explored whether the standard cut-points for qSOFA, the Shock Index, and NEWS2 provided the highest AUROC in our population by calculating test characteristics for several score cut-points.

Within our sample 23,837 ED patients had blood cultures ordered within 24 hours of ED presentation and were considered to have suspected sepsis. The mean age of the cohort was 60.8 years, and 1,612 (6.8%) had positive blood cultures. A total of 12,928 patients (54.2%) were found to have sepsis. We documented 1,427 in-hospital deaths (6.0%) and 3,149 (13.2%) ED-to-ICU admissions. At ED triage 1,921 (8.1%) were qSOFA-positive, 4,273 (17.9%) were Shock Index-positive, and 11,832 (49.6%) were NEWS2-positive. At ED triage, blood pressure, heart rate, respiratory rate, and oxygen saturated were documented in >99% of patients, 93.5% had temperature documented, and 28.5% had GCS recorded. If the window of assessment was widened to 1 hour, GCS was only documented among 44.2% of those with suspected sepsis.

Demographic Characteristics and Clinical Course

We identified significant differences when comparing demographic and clinical characteristics among patients who scored positive for the three severity measures at triage (Table 2). Although no hypothesis testing was conducted because patients could meet one or more scores, qSOFA-positive patients were older (median 70, 66, and 64 years, respectively), more likely to have Medicare as the primary payor (67.6% vs 59.7% vs 56.6%), to have chronic renal failure (26.1%, 23.1%, and 23.3%, respectively), to have a greater degree of underlying comorbidities based on the van Walraven Comorbidity Index (median 15, 12, and 11, respectively), and to be admitted to the ICU from the ED (48.1%, 36.3%, and 21.0%, respectively) compared with those positive for the Shock Index or NEWS2.

qSOFA-positive patients received antibiotics more quickly than those who were Shock Index-positive or NEWS2-positive (median 1.5, 1.8, and 2.8 hours after admission, respectively). In addition, those who were qSOFA-positive were more likely to have a positive blood culture (10.9%, 9.4%, and 8.5%, respectively) and to receive an EHR-based diagnosis of sepsis (77.0%, 69.6%, and 60.9%, respectively) than those who were Shock Index- or NEWS2-positive. Those who were qSOFA-positive also were more likely to be mechanically ventilated during their hospital stay (25.4%, 19.2%, and 10.8%, respectively) and to receive vasopressors (33.5%, 22.5%, and 12.2%, respectively). In-hospital mortality also was more common among those who were qSOFA-positive at triage (23.4%, 15.3%, and 9.2%, respectively).

Because both qSOFA and NEWS2 incorporate GCS, we explored baseline characteristics of patients with GCS documented at triage (n = 6,794). These patients were older (median age 63 and 61 years, P < .0001), more likely to be male (54.9% and 53.4%, P = .0031), more likely to have renal failure (22.8% and 20.1%, P < .0001), more likely to have liver disease (14.2% and 12.8%, P = .006), had a higher van Walraven comorbidity score on presentation (median 10 and 8, P < .0001), and were more likely to go directly to the ICU from the ED (20.2% and 10.6%, P < .0001). However, among the 6,397 GCS scores documented at triage, only 1,579 (24.7%) were abnormal.

Test Characteristics of qSOFA, Shock Index, and NEWS2 for Predicting In-hospital Mortality and ED-to-ICU Admission

Among 23,837 patients with suspected sepsis, NEWS2 had the highest sensitivity for predicting in-hospital mortality (76.0%; 95% CI, 73.7%-78.2%) and ED-to-ICU admission (78.9%; 95% CI, 77.5%-80.4%) but had the lowest specificity for in-hospital mortality (52.0%; 95% CI, 51.4%-52.7%) and for ED-to-ICU admission (54.8%; 95% CI, 54.1%-55.5%) (Table 3). qSOFA had the lowest sensitivity for in-hospital mortality (31.5%; 95% CI, 29.1%-33.9%) and ED-to-ICU admission (29.3%; 95% CI, 27.7%-30.9%) but the highest specificity for in-hospital mortality (93.4%; 95% CI, 93.1%-93.8%) and ED-to-ICU admission (95.2%; 95% CI, 94.9%-95.5%). The Shock Index had a sensitivity that fell between qSOFA and NEWS2 for in-hospital mortality (45.8%; 95% CI, 43.2%-48.5%) and ED-to-ICU admission (49.2%; 95% CI, 47.5%-51.0%). The specificity of the Shock Index also was between qSOFA and NEWS2 for in-hospital mortality (83.9%; 95% CI, 83.4%-84.3%) and ED-to-ICU admission (86.8%; 95% CI, 86.4%-87.3%). All three scores exhibited relatively low PPV, ranging from 9.2% to 23.4% for in-hospital mortality and 21.0% to 48.0% for ED-to-ICU triage. Conversely, all three scores exhibited relatively high NPV, ranging from 95.5% to 97.1% for in-hospital mortality and 89.8% to 94.5% for ED-to-ICU triage. The patterns in sensitivity and specificity for in-hospital mortality and ED-to-ICU admission were similar among the 12,928 patients who received an EHR-based sepsis diagnosis with the tests generally demonstrating lower specificities, higher PPVs, and lower NPVs (Table 3).

When considering a binary cutoff, the Shock Index exhibited the highest AUROC for in-hospital mortality (0.648; 95% CI, 0.635-0.662) and had a significantly higher AUROC than qSOFA (AUROC, 0.625; 95% CI, 0.612-0.637; P = .0005), but there was no difference compared with NEWS2 (AUROC, 0.640; 95% CI, 0.628-0.652; P = .2112). NEWS2 had a significantly higher AUROC than qSOFA for predicting in-hospital mortality (P = .0227). The Shock Index also exhibited the highest AUROC for ED-to-ICU admission (0.680; 95% CI, 0.617-0.689), which was significantly higher than the AUROC for qSOFA (P < .0001) and NEWS2 (P = 0.0151). NEWS2 had a significantly higher AUROC than qSOFA for predicting ED-to-ICU admission (P < .0001). Similar findings were seen in patients found to have sepsis. When considering the range of possible scores measured in our cohort, qSOFA and NEWS2 exhibited higher AUROCs for in-hospital mortality and ED-to-ICU admission than the Shock Index among patients with suspected infection and the subgroup with a sepsis diagnosis (Figure).

In this retrospective cohort study of 23,837 patients who presented to the ED with suspected sepsis, the standard qSOFA threshold was met least frequently, followed by the Shock Index and NEWS2. NEWS2 had the highest sensitivity but the lowest specificity for predicting in-hospital mortality and ED-to-ICU admission, making it a challenging bedside risk stratification scale for identifying patients at risk of poor clinical outcomes. When comparing predictive performance among the three scales, qSOFA had the highest specificity and the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission in this cohort of patients with suspected sepsis. These trends in sensitivity, specificity, and AUROC were consistent among those who met EHR criteria for a sepsis diagnosis. In the analysis of the three scoring systems using all available cut-points, qSOFA and NEWS2 had the highest AUROCs, followed by the Shock Index.

Considering the rapid progression from organ dysfunction to death in sepsis patients, as well as the difficulty establishing a sepsis diagnosis at triage,²³ providers must quickly identify patients at increased risk of poor outcomes when they present to the ED. Sepsis alerts often are built using SIRS criteria,²⁷ including the one used for sepsis surveillance at UCSF since 2012,²² but the white blood cell count criterion is subject to a laboratory lag and could lead to a delay in identification. Implementation of a point-of-care bedside score alert that uses readily available clinical data could allow providers to identify patients at greatest risk of poor outcomes immediately at ED presentation and triage, which motivated us to explore the predictive performance of qSOFA, the Shock Index, and NEWS2.

Our study is the first to provide a head-to-head comparison of the predictive performance of qSOFA, the Shock Index, and NEWS2, three easy-to-calculate bedside risk scores that use EHR data collected among patients with suspected sepsis. The Sepsis-3 guidelines recommend qSOFA to quickly identify non-ICU patients at greatest risk of poor outcomes because the measure exhibited predictive performance similar to the more extensive SOFA score outside the ICU.^16,23 Although some studies have confirmed qSOFA’s high predictive performance,^28-31 our test characteristics and AUROC findings are in line with other published analyses.^4,6,10,17 The UK National Health Service is using NEWS2 to screen for patients at risk of poor outcomes from sepsis. Several analyses that assessed the predictive ability of NEWS have reported estimates in line with our findings.^4,10,32 The Shock Index was introduced in 1967 and provided a metric to evaluate hemodynamic stability based on heart rate and systolic blood pressure.³³ The Shock Index has been studied in several contexts, including sepsis,³⁴ and studies show that a sustained Shock Index is associated with increased odds of vasopressor administration, higher prevalence of hyperlactatemia, and increased risk of poor outcomes in the ICU.^13,14

For our study, we were particularly interested in exploring how the Shock Index would compare with more frequently used severity scores such as qSOFA and NEWS2 among patients with suspected sepsis, given the simplicity of its calculation and the easy availability of required data. In our cohort of 23,837 patients, only 159 people had missing blood pressure and only 71 had omitted heart rate. In contrast, both qSOFA and NEWS2 include an assessment of level of consciousness that can be subject to variability in assessment methods and EHR documentation across institutions.¹¹ In our cohort, GCS within 30 minutes of ED presentation was missing in 72 patients, which could have led to incomplete calculation of qSOFA and NEWS2 if a missing value was not actually within normal limits.

Several investigations relate qSOFA to NEWS but few compare qSOFA with the newer NEWS2, and even fewer evaluate the Shock Index with any of these scores.^{10,11,18,29,35-37} In general, studies have shown that NEWS exhibits a higher AUROC for predicting mortality, sepsis with organ dysfunction, and ICU admission, often as a composite outcome.^{4,11,18,37,38} A handful of studies compare the Shock Index to SIRS; however, little has been done to compare the Shock Index to qSOFA or NEWS2, scores that have been used specifically for sepsis and might be more predictive of poor outcomes than SIRS.³³ In our study, the Shock Index had a higher AUROC than either qSOFA or NEWS2 for predicting in-hospital mortality and ED-to-ICU admission measured as separate outcomes and as a composite outcome using standard cut-points for these scores.

When selecting a severity score to apply in an institution, it is important to carefully evaluate the score’s test characteristics, in addition to considering the availability of reliable data. Tests with high sensitivity and NPV for the population being studied can be useful to rule out disease or risk of poor outcome, while tests with high specificity and PPV can be useful to rule in disease or risk of poor outcome.³⁹ When considering specificity, qSOFA’s performance was superior to the Shock Index and NEWS2 in our study, but a small percentage of the population was identified using a cut-point of qSOFA ≥2. If we used qSOFA and applied this standard cut-point at our institution, we could be confident that those identified were at increased risk, but we would miss a significant number of patients who would experience a poor outcome. When considering sensitivity, performance of NEWS2 was superior to qSOFA and the Shock Index in our study, but one-half of the population was identified using a cut-point of NEWS2 ≥5. If we were to apply this standard NEWS2 cut-point at our institution, we would assume that one-half of our population was at risk, which might drive resource use towards patients who will not experience a poor outcome. Although none of the scores exhibited a robust AUROC measure, the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission when using the standard binary cut-point, and its sensitivity and specificity is between that of qSOFA and NEWS2, potentially making it a score to use in settings where qSOFA and NEWS2 score components, such as altered mentation, are not reliably collected. Finally, our sensitivity analysis varying the binary cut-point of each score within our population demonstrated that the standard cut-points might not be as useful within a specific population and might need to be tailored for implementation, balancing sensitivity, specificity, PPV, and NPV to meet local priorities and ICU capacity.

Our study has limitations. It is a single-center, retrospective analysis, factors that could reduce generalizability. However, it does include a large and diverse patient population spanning several years. Missing GCS data could have affected the predictive ability of qSOFA and NEWS2 in our cohort. We could not reliably perform imputation of GCS because of the high missingness and therefore we assumed missing was normal, as was done in the Sepsis-3 derivation studies.¹⁶ Previous studies have attempted to impute GCS and have not observed improved performance of qSOFA to predict mortality.⁴⁰ Because manually collected variables such as GCS are less reliably documented in the EHR, there might be limitations in their use for triage risk scores.

Although the current analysis focused on the predictive performance of qSOFA, the Shock Index, and NEWS2 at triage, performance of these scores could affect the ED team’s treatment decisions before handoff to the hospitalist team and the expected level of care the patient will receive after in-patient admission. These tests also have the advantage of being easy to calculate at the bedside over time, which could provide an objective assessment of longitudinal predicted prognosis. Future work should assess the longitudinal performance of each of these scores among those with suspected sepsis and to determine the impact using these scores would have on clinical and resource utilization outcomes.

Local priorities should drive selection of a screening tool, balancing sensitivity, specificity, PPV, and NPV to achieve the institution’s goals. qSOFA, Shock Index, and NEWS2 are risk stratification tools that can be easily implemented at ED triage using data available at the bedside. Although none of these scores performed strongly when comparing AUROCs, qSOFA was highly specific for identifying patients with poor outcomes, and NEWS2 was the most sensitive for ruling out those at high risk among patients with suspected sepsis. The Shock Index exhibited a sensitivity and specificity that fell between qSOFA and NEWS2 and also might be considered to identify those at increased risk, given its ease of implementation, particularly in settings where altered mentation is unreliably or inconsistently documented.

The authors thank the UCSF Division of Hospital Medicine Data Core for their assistance with data acquisition.

The US Department of Veterans Affairs (VA) created the Home Improvements and Structural Alterations (HISA) program to help provide necessary home modifications (HMs) to veterans with disabilities (VWDs) that will facilitate the provision of medical services at home and improve home accessibility and functional independence. The Veterans Health Administration (VHA) has more than 9 million veteran enrollees; of those, 2.7 million are classified as rural or highly rural.¹ Rural veterans (RVs) possess higher rate of disability compared with that of urban veterans.^2-5 RVs have unequal access to screening of ambulatory care sensitive conditions (eg, hypertension, diabetes mellitus).⁶ Furthermore, RVs are at risk of poor medical outcomes due to distance from health care facilities and specialist care, which can be a barrier to emergency care when issues arise. These barriers, among others, are associated with compromised health quality of life and health outcomes for RVs.^3,6 The HISA program may be key to decreasing falls and other serious mishaps in the home. Therefore, understanding use of the HISA program by RVs is important. However, to date little information has been available regarding use of HISA benefits by RVs or characteristics of RVs who receive HISA benefits.

HISA Alterations Program

HISA was initially developed by VA to improve veterans’ transition from acute medical care to home.^7,8 However, to obtain HISA grants currently, there is an average 3 to 6 months application process.⁷ Through the HISA program, VWDs can be prescribed the following HMs, including (but not limited to): flooring replacement, permanent ramps, roll-in showers, installation of central air-conditioning systems, improved lighting, kitchen/bathroom modifications, and home inspections. The HMs prescribed depend on an assessment of medical need by health care providers (HCPs).⁸

As time passed and the veteran population aged, the program now primarily helps ensure the ability to enter into essential areas and safety in the home.⁵ The amount of a HISA payment is based on whether a veteran’s health condition is related to military service as defined by the VHA service connection medical evaluation process. Barriers to obtaining a HISA HM can include difficulty in navigating the evaluation process and difficulty in finding a qualified contractor or builder to do the HM.⁷

This article aims to: (1) Detail the sociodemographic and clinical characteristics of rural HISA users (RHUs); (2) report on HISA usage patterns in number, types, and cost of HMs; (3) compare use amid the diverse VA medical centers (VAMCs) and related complexity levels and Veterans Integrated Service Networks (VISNs); and (4) examine the relationship between travel time/distance and HISA utilization. The long-term goal is to provide accurate information to researchers, HM administrators, health care providers and policy makers on HISA program utilization by rural VWDs, which may help improve its use and bring awareness of its users. This study was approved by the affiliate University of Florida Institutional Review Board and VA research and development committee at the North Florida/South Georgia Veterans Health System.

Methods

Data were obtained from 3 VA sources: the National Prosthetics Patient Database (NPPD), the VHA Medical Inpatient Dataset, and the VHA Outpatient Dataset.⁷ The NPPD is a national administrative database that contains information on prosthetic-associated products ordered by HCPs for patients, such as portable ramps, handrails, home oxygen equipment, and orthotic and prosthetic apparatus. Data obtained from the NPPD included cost of HMs, clinical characteristics, VISN, and VAMC. VA facilities are categorized into complexity levels 1a, 1b, 1c, 2, and 3. Complexity level 1a to 1c VAMCs address medical cases that entail “heightening involvedness,” meaning a larger number of patients presented with medical concerns needing medical specialists. Complexity levels 2 and 3 have fewer resources, lower patient numbers, and less medically complex patients. Finally, the VHA Medical Inpatient and Outpatient Datasets administrated by VA Informatics and Computing Infrastructure, consist of in-depth health services national data on inpatient and outpatient encounters and procedures.

The study cohort was divided into those with service-connected conditions (Class 1) or those with conditions not related to military service (Class 2). If veterans were identified in both classes, they were assigned to Class 1. The cost variable is determined by using the veterans’ classification. Class 1 veterans receive a lifetime limit of $6800, and Class 2 veterans receive a lifetime limit of $2000. A Class 2 veteran with ≥ 50% disability rating is eligible for a HISA lifetime limit of $6800. Whenever a value exceeds allowed limit of $6800 or $2000, due to data entry error or other reasons, the study team reassigned the cost value to the maximum allowed value.

Travel distance and time were derived by loading patient zip codes and HISA facility locations into the geographical information system program and using the nearest facility and find-route tools. These tools used a road network that simulates real-world driving conditions to calculate distance.

Study Variables

VWDs of any age, gender, and race/ethnicity who qualified for HISA and received HMs from fiscal year ( FY) 2015 through FY 2018 were identified (N = 30,823). Most VWDs were nonrural subjects (n = 19,970), and 43 had no Federal Information Processing System data. The final study cohort consisted of 10,810 HISA recipients. The NPPD, inpatient and outpatient data were merged by scrambled social security numbers to retrieve the following data: age, gender, race, ethnicity, marital status, Class (1 or 2), mean and total number of inpatient days, and type of HMs prescribed.

We also recorded rurality using the VA Rural-Urban Commuting Areas (RUCA) system, but we combined the rural and highly rural designation.¹ Census tracts with a RUCA score of 10.0 are deemed highly rural, the remainder are considered rural except those with a RUCA score of 1.0 or 1.1. Travel time and distance from a veteran’s home to the VA facility that provided the HISA prescription were determined from zip codes. The current study focuses on VAMCs prescribing stations (affiliated sites of administrative parent medical facilities) where the HISA users obtained the HM, not the parent station (administrative parent medical facilities).

HISA Utilization

To characterize HISA utilization geographically and over time, the number of users were mapped by county. Areas where users were increasing (hot spots) or decreasing (cold spots) also were mapped. The maps were created using Environmental Systems Research Institute ArcGIS Pro 2.2.1 software. We chose to use natural breaks (Jenks) data classification method in a choropleth to symbolize the change over time map. We then used the Getis Ord GI* optimized hot spot analysis tool in the ArcGIS Pro spatial statistics tool set to generate the hot/cold spot maps. This tool identifies clusters of high values (hot spots) and low values (cold spots) creating a new output layer, RHUs by county, with a Z score, P value, and CI for each county. The Gi Bin field classifies statistically significant hot and cold spots. Counties sorted into the ± 3 category (bin) have a clustering characteristic (eg, with neighboring counties) that is statistically significant with a 99% CI; the ± 2 bin indicates a 95% CI for those county clustering sorted therein; ± 1 reflects a 90% CI; and 0 bin contains county features that have no statistical significant clustering with neighboring counties.

Data Analysis

Data were cleaned and analyzed using SAS 9.4 and R 3.5.3. Descriptive statistics are provided for sociodemographic characteristics, clinical characteristics, and class. ANOVA and t tests were used to compare continuous variables between groups, while χ² and Fisher exact tests were used for dichotomous and categorical outcome variables. The threshold for statistical significance for these tests was set at α = .001.

Results

There were 10,810 RHUs from FY 2015 through FY 2018 and HISA utilization increased each year (Figure 1). Although some years may show usage decreases relative to previous fiscal years, the cumulative trends showed an increase relative to FY 2015 for both Classes of RVs (Figure 2). There was a 45.4% increase from FY 2015 to FY 2018 with a mean 13.6% yearly increase. Class 1 increased 21.0% and Class 2 increased 39.5% from FY 2015 to FY 2016 (Figure 3).

Most RHUs were male, White, and married. Class 1 and Class 2 RHUs differed significantly by age, race, marital status, and disability conditions: Class 1 RHUs were aged 6.6 years younger with a mean age of 69.1 years compared with 75.7 years for Class 2 users. For Class 1 RHUs, a plurality (29.4%) were aged 65 to 69 years; while a plurality (41.4%) of Class 2 users were aged ≥ 80 years. Musculoskeletal was the most common identified type of condition for all RHUs (Table 1).

There were 11,166 HM prescribed to RHUs (Table 2). Bathroom HMs also were the dominant HM type for all facilities regardless of complexity levels (Table 3). The San Antonio, Texas, VAMC demonstrated the highest Class 1 vs Class 2 difference in HISA use (Class 1: 87.7% and Class 2: 12.3%). Except for the Des Moines VAMC, all other VAMCs showed HISA use > 60% by Class 1.

Cost Data

Air-conditioning installation ($5007) was the costliest HM overall (Table 4), closely followed by bathroom ($4978) and kitchen modifications ($4305). Bathroom renovations were the costliest HM type for both Class 1 and Class 2, closely followed by electrical repair and air-conditioning installation for Class 1 and driveway reconstruction and wooden ramp construction for Class 2.

The mean award received for HM was $4687 (Table 5). While the number of RHUs increased from FY 2015 to FY 2016, the average cost decreased, both overall ($280) and for Class 1 ($195) and Class 2 ($153). Except for a small decline in the number of Class 2 HISA recipients from FY 2017 to FY 2018, overall, the number of RHUs continuously grew from FY 2015 to FY 2018: 977 for the overall cohort, 678 for Class 1 and 299 for Class 2. Despite the obvious gain in the number of RHUs, the average costs did not notably change over time. VISN 21 had the highest mean cost, followed by VISNs 17, 6, 22, and 20.

Travel

Travel time and distance to the HISA prescribing facility differed significantly between Class 1 and Class 2 HISA users. RHUs had to travel about 95 minutes from their place of residence to access the HISA benefits program. There were no statistically significant differences between Class 1 and 2 users with respect to travel time and distance traveled (Table 6).

The majority of Class 1 and Class 2 veterans accessed the HISA from their nearest facility. However, nearly one-quarter of both Class 1 and 2 RHUs (24% each) did not. Among the 2598 who accessed the nonnearest facility, 97 (3.7%) accessed a facility that is ≤ 40 miles. Many (44%) users traveled 40 to 100 miles, and another 43.2% traveled 100 to 200 miles from their residence to access a HM prescription. Some 2598 users (1.1%) traveled > 500 miles to access a facility.

Discussion

Although utilization of the HISA program has steadily increased, overall participation by subpopulations such as RHUs can still be improved significantly. Veterans aged ≤ 46 years who have a disability that is common to those receiving HISA benefits have low HISA utilization. Similarly, veterans with sensory disabilities also have low use. These subpopulations are among those in great need of attention and services.

A study by Lucas and Zelaya, using the 2016 National Health Interview Survey data with an aim to measure degree of vision problems, dual sensory impairment, and hearing trouble in male veterans aged ≥ 18 years, found that veterans were more likely to report dual sensory impairment and balance difficulties when compared with nonveterans.⁹ The number of female veterans is growing but had very low representation in this study.¹⁰ This emerging VHA population requires information and education on their HM benefits.

Home Modifications

The most common HM prescribed for RHUs was for the bathroom. Further investigation is warranted as to why, given the diversity of HM types that the grant covers, low prescription rates exist across most of the HM types. There may be a lack of knowledge by providers and VWD as to the range of HMs that can be awarded under the grant. It is important that HCPs and veterans receive education on HISA HM options.

Semeah and colleagues pointed out the need for an assessment of the HISA HM ordering system to ensure that multiple HMs items (eg, kitchen, air conditioning, fees, driveway, and plumbing) are listed among the forced choices shown to clinicians to select from.⁷ Poor housing in rural America is widespread: 63% of rural dwellings need renovations and/or repairs to be accessible to individuals with disabilities, with > 6.7 million rural homes having no or faulty plumbing or kitchens; yet in this study, prescriptions for these HMs accounted for < 1%.^11,12

VISN 6 had the most HISA awards with 1364, while VISN 21 had the fewest (245). Across all VISNs, Class 1 RHUs received more prescriptions than did Class 2 RHUs. Future research may seek to examine whether prescribers are fully aware of the eligibility of HM prescription to Class 2 veterans. VISN 21 ($5354); VISN 17 ($5302); and VISN 6 ($5301) had the highest mean HM expenditures. The national mean cost for HISA HMs were $4978 for bathrooms and $4305 for kitchens; for non-HISA HMs in FY 2017, the mean costs were $6362 and $12,255, respectively. A noteworthy concern is whether the maximum grant limit awards are sufficient to perform more expensive and complex HMs, such as the kitchen or major bathroom alternations.¹³

Facilities categorized as 1a, 1b, or 1c provided prescription to more than 70.7% of all RHUs. Embedded in the infrastructures of these higher complexity level facilities are substantial research and education opportunities for staff and patients and a diversity of medical specialties. Attention must be provided to less resourced facilities located in rural areas or serving many RVs that may not have the infrastructure to monitor the HISA process or make it available to VWDs. Furthermore, the top prescribing VAMCs were almost all high-complexity facilities, and all served > 100 unique RHUs. Further research may be warranted to be better understand the variation in HM prescriptions at VAMCs.

North Florida/Sough Georgia was the highest-prescribing VAMC with 39% more HM prescriptions than the second highest prescribing facility (Durham, NC). Unfortunately, the data presented here cannot establish causality for the large variance difference between the top facilities, and the skewed distribution of total RHUs across VAMCs.

Travel-Related Variables

HISA beneficiaries face significant travel-related challenges. Just 3.6% of RHUs could access a facility within 40 miles of their home and 43.2% traveled 100 to 200 miles from their home to access a HM prescription. Further exploration is warranted to understand how travel patterns impact access to or the uptake of HISA.

RVs already have problems with accessing care because of long travel time.^14,15 The choice or necessity to travel to a farther facility for HISA prescription is problematic for RVs, especially when transportation is often reported in the literature as a barrier to resources for people living in rural communities.^15-17 When patients have travel barriers, they wait longer to obtain medical services and often wait for their conditions to worsen before seeking services.^15,18 Once HM is completed, telerehabilitation is an effective delivery method used for delivering health care services to people in remote places.^18,19 Considering that HISA use has the potential to improve quality of life, afford comfort, facilitate the accomplishment of activities of daily living for RVs, it is important that future studies examine how existing telehealth technologies can be used to improve HISA access.

Future Directions

County-level analyses is warranted in future studies exploring potential variables associated with HISA use; for example, county-level rates of primary care physicians and other HCPs. Future research should explore how long distance travel impacts the HISA application process and HM implementation. Further research also should focus on the HISA application structure and process to identify causes of delays. The HISA application process takes a mean 6 months to complete, yet the duration of hospital stays is 1 to 3 weeks, thus it is impossible to connect HISA to hospital discharge, which was the original intent of the program. Future research can examine how telehealth services can expedite HISA obtainment and coordination of the application process. Future research also may study the possible causes of the wide variations in HM prescriptions per facility. It is also important that educational programs provide information on the array of HM items that veterans can obtain.

Conclusions

In our previous study of the HISA cohort (2011-2017), we documented that an increase in utilization of the HISA program was warranted based on the low national budgetary appropriation and identification of significant low participation by vulnerable subpopulations, including veterans residing in rural areas or having returned from recent conflicts.⁷ The present study documents national utilization patterns, demographic profiles, and clinical characteristics of RHUs from FY 2015 through FY 2018, data that may be useful to policy makers and HISA administrators in predicting future use and users. It is important to note that the data and information presented in this article identify trends. The work in no way establishes a gold standard or any targeted goal of utilization. Future research could focus on conceptualizing or theorizing what steps are necessary to set such a gold standard of utilization rate and steps toward achievement.

Acknowledgments

This research was supported by grant 15521 from the US Department of Veterans Affairs, Office of Rural Health . Furthermore, the research was supported in part by grant K12 HD055929 from the National Institutes of Health.

The US Department of Veterans Affairs (VA) created the Home Improvements and Structural Alterations (HISA) program to help provide necessary home modifications (HMs) to veterans with disabilities (VWDs) that will facilitate the provision of medical services at home and improve home accessibility and functional independence. The Veterans Health Administration (VHA) has more than 9 million veteran enrollees; of those, 2.7 million are classified as rural or highly rural.¹ Rural veterans (RVs) possess higher rate of disability compared with that of urban veterans.^2-5 RVs have unequal access to screening of ambulatory care sensitive conditions (eg, hypertension, diabetes mellitus).⁶ Furthermore, RVs are at risk of poor medical outcomes due to distance from health care facilities and specialist care, which can be a barrier to emergency care when issues arise. These barriers, among others, are associated with compromised health quality of life and health outcomes for RVs.^3,6 The HISA program may be key to decreasing falls and other serious mishaps in the home. Therefore, understanding use of the HISA program by RVs is important. However, to date little information has been available regarding use of HISA benefits by RVs or characteristics of RVs who receive HISA benefits.

HISA Alterations Program

HISA was initially developed by VA to improve veterans’ transition from acute medical care to home.^7,8 However, to obtain HISA grants currently, there is an average 3 to 6 months application process.⁷ Through the HISA program, VWDs can be prescribed the following HMs, including (but not limited to): flooring replacement, permanent ramps, roll-in showers, installation of central air-conditioning systems, improved lighting, kitchen/bathroom modifications, and home inspections. The HMs prescribed depend on an assessment of medical need by health care providers (HCPs).⁸

As time passed and the veteran population aged, the program now primarily helps ensure the ability to enter into essential areas and safety in the home.⁵ The amount of a HISA payment is based on whether a veteran’s health condition is related to military service as defined by the VHA service connection medical evaluation process. Barriers to obtaining a HISA HM can include difficulty in navigating the evaluation process and difficulty in finding a qualified contractor or builder to do the HM.⁷

This article aims to: (1) Detail the sociodemographic and clinical characteristics of rural HISA users (RHUs); (2) report on HISA usage patterns in number, types, and cost of HMs; (3) compare use amid the diverse VA medical centers (VAMCs) and related complexity levels and Veterans Integrated Service Networks (VISNs); and (4) examine the relationship between travel time/distance and HISA utilization. The long-term goal is to provide accurate information to researchers, HM administrators, health care providers and policy makers on HISA program utilization by rural VWDs, which may help improve its use and bring awareness of its users. This study was approved by the affiliate University of Florida Institutional Review Board and VA research and development committee at the North Florida/South Georgia Veterans Health System.

Methods

Data were obtained from 3 VA sources: the National Prosthetics Patient Database (NPPD), the VHA Medical Inpatient Dataset, and the VHA Outpatient Dataset.⁷ The NPPD is a national administrative database that contains information on prosthetic-associated products ordered by HCPs for patients, such as portable ramps, handrails, home oxygen equipment, and orthotic and prosthetic apparatus. Data obtained from the NPPD included cost of HMs, clinical characteristics, VISN, and VAMC. VA facilities are categorized into complexity levels 1a, 1b, 1c, 2, and 3. Complexity level 1a to 1c VAMCs address medical cases that entail “heightening involvedness,” meaning a larger number of patients presented with medical concerns needing medical specialists. Complexity levels 2 and 3 have fewer resources, lower patient numbers, and less medically complex patients. Finally, the VHA Medical Inpatient and Outpatient Datasets administrated by VA Informatics and Computing Infrastructure, consist of in-depth health services national data on inpatient and outpatient encounters and procedures.

The study cohort was divided into those with service-connected conditions (Class 1) or those with conditions not related to military service (Class 2). If veterans were identified in both classes, they were assigned to Class 1. The cost variable is determined by using the veterans’ classification. Class 1 veterans receive a lifetime limit of $6800, and Class 2 veterans receive a lifetime limit of $2000. A Class 2 veteran with ≥ 50% disability rating is eligible for a HISA lifetime limit of $6800. Whenever a value exceeds allowed limit of $6800 or $2000, due to data entry error or other reasons, the study team reassigned the cost value to the maximum allowed value.

Travel distance and time were derived by loading patient zip codes and HISA facility locations into the geographical information system program and using the nearest facility and find-route tools. These tools used a road network that simulates real-world driving conditions to calculate distance.

Study Variables

VWDs of any age, gender, and race/ethnicity who qualified for HISA and received HMs from fiscal year ( FY) 2015 through FY 2018 were identified (N = 30,823). Most VWDs were nonrural subjects (n = 19,970), and 43 had no Federal Information Processing System data. The final study cohort consisted of 10,810 HISA recipients. The NPPD, inpatient and outpatient data were merged by scrambled social security numbers to retrieve the following data: age, gender, race, ethnicity, marital status, Class (1 or 2), mean and total number of inpatient days, and type of HMs prescribed.

We also recorded rurality using the VA Rural-Urban Commuting Areas (RUCA) system, but we combined the rural and highly rural designation.¹ Census tracts with a RUCA score of 10.0 are deemed highly rural, the remainder are considered rural except those with a RUCA score of 1.0 or 1.1. Travel time and distance from a veteran’s home to the VA facility that provided the HISA prescription were determined from zip codes. The current study focuses on VAMCs prescribing stations (affiliated sites of administrative parent medical facilities) where the HISA users obtained the HM, not the parent station (administrative parent medical facilities).

HISA Utilization

To characterize HISA utilization geographically and over time, the number of users were mapped by county. Areas where users were increasing (hot spots) or decreasing (cold spots) also were mapped. The maps were created using Environmental Systems Research Institute ArcGIS Pro 2.2.1 software. We chose to use natural breaks (Jenks) data classification method in a choropleth to symbolize the change over time map. We then used the Getis Ord GI* optimized hot spot analysis tool in the ArcGIS Pro spatial statistics tool set to generate the hot/cold spot maps. This tool identifies clusters of high values (hot spots) and low values (cold spots) creating a new output layer, RHUs by county, with a Z score, P value, and CI for each county. The Gi Bin field classifies statistically significant hot and cold spots. Counties sorted into the ± 3 category (bin) have a clustering characteristic (eg, with neighboring counties) that is statistically significant with a 99% CI; the ± 2 bin indicates a 95% CI for those county clustering sorted therein; ± 1 reflects a 90% CI; and 0 bin contains county features that have no statistical significant clustering with neighboring counties.

Data Analysis

Data were cleaned and analyzed using SAS 9.4 and R 3.5.3. Descriptive statistics are provided for sociodemographic characteristics, clinical characteristics, and class. ANOVA and t tests were used to compare continuous variables between groups, while χ² and Fisher exact tests were used for dichotomous and categorical outcome variables. The threshold for statistical significance for these tests was set at α = .001.

Results

There were 10,810 RHUs from FY 2015 through FY 2018 and HISA utilization increased each year (Figure 1). Although some years may show usage decreases relative to previous fiscal years, the cumulative trends showed an increase relative to FY 2015 for both Classes of RVs (Figure 2). There was a 45.4% increase from FY 2015 to FY 2018 with a mean 13.6% yearly increase. Class 1 increased 21.0% and Class 2 increased 39.5% from FY 2015 to FY 2016 (Figure 3).

Most RHUs were male, White, and married. Class 1 and Class 2 RHUs differed significantly by age, race, marital status, and disability conditions: Class 1 RHUs were aged 6.6 years younger with a mean age of 69.1 years compared with 75.7 years for Class 2 users. For Class 1 RHUs, a plurality (29.4%) were aged 65 to 69 years; while a plurality (41.4%) of Class 2 users were aged ≥ 80 years. Musculoskeletal was the most common identified type of condition for all RHUs (Table 1).

There were 11,166 HM prescribed to RHUs (Table 2). Bathroom HMs also were the dominant HM type for all facilities regardless of complexity levels (Table 3). The San Antonio, Texas, VAMC demonstrated the highest Class 1 vs Class 2 difference in HISA use (Class 1: 87.7% and Class 2: 12.3%). Except for the Des Moines VAMC, all other VAMCs showed HISA use > 60% by Class 1.

Cost Data

Air-conditioning installation ($5007) was the costliest HM overall (Table 4), closely followed by bathroom ($4978) and kitchen modifications ($4305). Bathroom renovations were the costliest HM type for both Class 1 and Class 2, closely followed by electrical repair and air-conditioning installation for Class 1 and driveway reconstruction and wooden ramp construction for Class 2.

The mean award received for HM was $4687 (Table 5). While the number of RHUs increased from FY 2015 to FY 2016, the average cost decreased, both overall ($280) and for Class 1 ($195) and Class 2 ($153). Except for a small decline in the number of Class 2 HISA recipients from FY 2017 to FY 2018, overall, the number of RHUs continuously grew from FY 2015 to FY 2018: 977 for the overall cohort, 678 for Class 1 and 299 for Class 2. Despite the obvious gain in the number of RHUs, the average costs did not notably change over time. VISN 21 had the highest mean cost, followed by VISNs 17, 6, 22, and 20.

Travel

Travel time and distance to the HISA prescribing facility differed significantly between Class 1 and Class 2 HISA users. RHUs had to travel about 95 minutes from their place of residence to access the HISA benefits program. There were no statistically significant differences between Class 1 and 2 users with respect to travel time and distance traveled (Table 6).

The majority of Class 1 and Class 2 veterans accessed the HISA from their nearest facility. However, nearly one-quarter of both Class 1 and 2 RHUs (24% each) did not. Among the 2598 who accessed the nonnearest facility, 97 (3.7%) accessed a facility that is ≤ 40 miles. Many (44%) users traveled 40 to 100 miles, and another 43.2% traveled 100 to 200 miles from their residence to access a HM prescription. Some 2598 users (1.1%) traveled > 500 miles to access a facility.

Discussion

Although utilization of the HISA program has steadily increased, overall participation by subpopulations such as RHUs can still be improved significantly. Veterans aged ≤ 46 years who have a disability that is common to those receiving HISA benefits have low HISA utilization. Similarly, veterans with sensory disabilities also have low use. These subpopulations are among those in great need of attention and services.

A study by Lucas and Zelaya, using the 2016 National Health Interview Survey data with an aim to measure degree of vision problems, dual sensory impairment, and hearing trouble in male veterans aged ≥ 18 years, found that veterans were more likely to report dual sensory impairment and balance difficulties when compared with nonveterans.⁹ The number of female veterans is growing but had very low representation in this study.¹⁰ This emerging VHA population requires information and education on their HM benefits.

Home Modifications

The most common HM prescribed for RHUs was for the bathroom. Further investigation is warranted as to why, given the diversity of HM types that the grant covers, low prescription rates exist across most of the HM types. There may be a lack of knowledge by providers and VWD as to the range of HMs that can be awarded under the grant. It is important that HCPs and veterans receive education on HISA HM options.

Semeah and colleagues pointed out the need for an assessment of the HISA HM ordering system to ensure that multiple HMs items (eg, kitchen, air conditioning, fees, driveway, and plumbing) are listed among the forced choices shown to clinicians to select from.⁷ Poor housing in rural America is widespread: 63% of rural dwellings need renovations and/or repairs to be accessible to individuals with disabilities, with > 6.7 million rural homes having no or faulty plumbing or kitchens; yet in this study, prescriptions for these HMs accounted for < 1%.^11,12

VISN 6 had the most HISA awards with 1364, while VISN 21 had the fewest (245). Across all VISNs, Class 1 RHUs received more prescriptions than did Class 2 RHUs. Future research may seek to examine whether prescribers are fully aware of the eligibility of HM prescription to Class 2 veterans. VISN 21 ($5354); VISN 17 ($5302); and VISN 6 ($5301) had the highest mean HM expenditures. The national mean cost for HISA HMs were $4978 for bathrooms and $4305 for kitchens; for non-HISA HMs in FY 2017, the mean costs were $6362 and $12,255, respectively. A noteworthy concern is whether the maximum grant limit awards are sufficient to perform more expensive and complex HMs, such as the kitchen or major bathroom alternations.¹³

Facilities categorized as 1a, 1b, or 1c provided prescription to more than 70.7% of all RHUs. Embedded in the infrastructures of these higher complexity level facilities are substantial research and education opportunities for staff and patients and a diversity of medical specialties. Attention must be provided to less resourced facilities located in rural areas or serving many RVs that may not have the infrastructure to monitor the HISA process or make it available to VWDs. Furthermore, the top prescribing VAMCs were almost all high-complexity facilities, and all served > 100 unique RHUs. Further research may be warranted to be better understand the variation in HM prescriptions at VAMCs.

North Florida/Sough Georgia was the highest-prescribing VAMC with 39% more HM prescriptions than the second highest prescribing facility (Durham, NC). Unfortunately, the data presented here cannot establish causality for the large variance difference between the top facilities, and the skewed distribution of total RHUs across VAMCs.

Travel-Related Variables

HISA beneficiaries face significant travel-related challenges. Just 3.6% of RHUs could access a facility within 40 miles of their home and 43.2% traveled 100 to 200 miles from their home to access a HM prescription. Further exploration is warranted to understand how travel patterns impact access to or the uptake of HISA.

RVs already have problems with accessing care because of long travel time.^14,15 The choice or necessity to travel to a farther facility for HISA prescription is problematic for RVs, especially when transportation is often reported in the literature as a barrier to resources for people living in rural communities.^15-17 When patients have travel barriers, they wait longer to obtain medical services and often wait for their conditions to worsen before seeking services.^15,18 Once HM is completed, telerehabilitation is an effective delivery method used for delivering health care services to people in remote places.^18,19 Considering that HISA use has the potential to improve quality of life, afford comfort, facilitate the accomplishment of activities of daily living for RVs, it is important that future studies examine how existing telehealth technologies can be used to improve HISA access.

Future Directions

County-level analyses is warranted in future studies exploring potential variables associated with HISA use; for example, county-level rates of primary care physicians and other HCPs. Future research should explore how long distance travel impacts the HISA application process and HM implementation. Further research also should focus on the HISA application structure and process to identify causes of delays. The HISA application process takes a mean 6 months to complete, yet the duration of hospital stays is 1 to 3 weeks, thus it is impossible to connect HISA to hospital discharge, which was the original intent of the program. Future research can examine how telehealth services can expedite HISA obtainment and coordination of the application process. Future research also may study the possible causes of the wide variations in HM prescriptions per facility. It is also important that educational programs provide information on the array of HM items that veterans can obtain.

Conclusions

In our previous study of the HISA cohort (2011-2017), we documented that an increase in utilization of the HISA program was warranted based on the low national budgetary appropriation and identification of significant low participation by vulnerable subpopulations, including veterans residing in rural areas or having returned from recent conflicts.⁷ The present study documents national utilization patterns, demographic profiles, and clinical characteristics of RHUs from FY 2015 through FY 2018, data that may be useful to policy makers and HISA administrators in predicting future use and users. It is important to note that the data and information presented in this article identify trends. The work in no way establishes a gold standard or any targeted goal of utilization. Future research could focus on conceptualizing or theorizing what steps are necessary to set such a gold standard of utilization rate and steps toward achievement.

Acknowledgments

This research was supported by grant 15521 from the US Department of Veterans Affairs, Office of Rural Health . Furthermore, the research was supported in part by grant K12 HD055929 from the National Institutes of Health.

The US Department of Veterans Affairs (VA) created the Home Improvements and Structural Alterations (HISA) program to help provide necessary home modifications (HMs) to veterans with disabilities (VWDs) that will facilitate the provision of medical services at home and improve home accessibility and functional independence. The Veterans Health Administration (VHA) has more than 9 million veteran enrollees; of those, 2.7 million are classified as rural or highly rural.¹ Rural veterans (RVs) possess higher rate of disability compared with that of urban veterans.^2-5 RVs have unequal access to screening of ambulatory care sensitive conditions (eg, hypertension, diabetes mellitus).⁶ Furthermore, RVs are at risk of poor medical outcomes due to distance from health care facilities and specialist care, which can be a barrier to emergency care when issues arise. These barriers, among others, are associated with compromised health quality of life and health outcomes for RVs.^3,6 The HISA program may be key to decreasing falls and other serious mishaps in the home. Therefore, understanding use of the HISA program by RVs is important. However, to date little information has been available regarding use of HISA benefits by RVs or characteristics of RVs who receive HISA benefits.

HISA Alterations Program

HISA was initially developed by VA to improve veterans’ transition from acute medical care to home.^7,8 However, to obtain HISA grants currently, there is an average 3 to 6 months application process.⁷ Through the HISA program, VWDs can be prescribed the following HMs, including (but not limited to): flooring replacement, permanent ramps, roll-in showers, installation of central air-conditioning systems, improved lighting, kitchen/bathroom modifications, and home inspections. The HMs prescribed depend on an assessment of medical need by health care providers (HCPs).⁸

As time passed and the veteran population aged, the program now primarily helps ensure the ability to enter into essential areas and safety in the home.⁵ The amount of a HISA payment is based on whether a veteran’s health condition is related to military service as defined by the VHA service connection medical evaluation process. Barriers to obtaining a HISA HM can include difficulty in navigating the evaluation process and difficulty in finding a qualified contractor or builder to do the HM.⁷

This article aims to: (1) Detail the sociodemographic and clinical characteristics of rural HISA users (RHUs); (2) report on HISA usage patterns in number, types, and cost of HMs; (3) compare use amid the diverse VA medical centers (VAMCs) and related complexity levels and Veterans Integrated Service Networks (VISNs); and (4) examine the relationship between travel time/distance and HISA utilization. The long-term goal is to provide accurate information to researchers, HM administrators, health care providers and policy makers on HISA program utilization by rural VWDs, which may help improve its use and bring awareness of its users. This study was approved by the affiliate University of Florida Institutional Review Board and VA research and development committee at the North Florida/South Georgia Veterans Health System.

Methods

Data were obtained from 3 VA sources: the National Prosthetics Patient Database (NPPD), the VHA Medical Inpatient Dataset, and the VHA Outpatient Dataset.⁷ The NPPD is a national administrative database that contains information on prosthetic-associated products ordered by HCPs for patients, such as portable ramps, handrails, home oxygen equipment, and orthotic and prosthetic apparatus. Data obtained from the NPPD included cost of HMs, clinical characteristics, VISN, and VAMC. VA facilities are categorized into complexity levels 1a, 1b, 1c, 2, and 3. Complexity level 1a to 1c VAMCs address medical cases that entail “heightening involvedness,” meaning a larger number of patients presented with medical concerns needing medical specialists. Complexity levels 2 and 3 have fewer resources, lower patient numbers, and less medically complex patients. Finally, the VHA Medical Inpatient and Outpatient Datasets administrated by VA Informatics and Computing Infrastructure, consist of in-depth health services national data on inpatient and outpatient encounters and procedures.

The study cohort was divided into those with service-connected conditions (Class 1) or those with conditions not related to military service (Class 2). If veterans were identified in both classes, they were assigned to Class 1. The cost variable is determined by using the veterans’ classification. Class 1 veterans receive a lifetime limit of $6800, and Class 2 veterans receive a lifetime limit of $2000. A Class 2 veteran with ≥ 50% disability rating is eligible for a HISA lifetime limit of $6800. Whenever a value exceeds allowed limit of $6800 or $2000, due to data entry error or other reasons, the study team reassigned the cost value to the maximum allowed value.

Travel distance and time were derived by loading patient zip codes and HISA facility locations into the geographical information system program and using the nearest facility and find-route tools. These tools used a road network that simulates real-world driving conditions to calculate distance.

Study Variables

VWDs of any age, gender, and race/ethnicity who qualified for HISA and received HMs from fiscal year ( FY) 2015 through FY 2018 were identified (N = 30,823). Most VWDs were nonrural subjects (n = 19,970), and 43 had no Federal Information Processing System data. The final study cohort consisted of 10,810 HISA recipients. The NPPD, inpatient and outpatient data were merged by scrambled social security numbers to retrieve the following data: age, gender, race, ethnicity, marital status, Class (1 or 2), mean and total number of inpatient days, and type of HMs prescribed.

We also recorded rurality using the VA Rural-Urban Commuting Areas (RUCA) system, but we combined the rural and highly rural designation.¹ Census tracts with a RUCA score of 10.0 are deemed highly rural, the remainder are considered rural except those with a RUCA score of 1.0 or 1.1. Travel time and distance from a veteran’s home to the VA facility that provided the HISA prescription were determined from zip codes. The current study focuses on VAMCs prescribing stations (affiliated sites of administrative parent medical facilities) where the HISA users obtained the HM, not the parent station (administrative parent medical facilities).

HISA Utilization

To characterize HISA utilization geographically and over time, the number of users were mapped by county. Areas where users were increasing (hot spots) or decreasing (cold spots) also were mapped. The maps were created using Environmental Systems Research Institute ArcGIS Pro 2.2.1 software. We chose to use natural breaks (Jenks) data classification method in a choropleth to symbolize the change over time map. We then used the Getis Ord GI* optimized hot spot analysis tool in the ArcGIS Pro spatial statistics tool set to generate the hot/cold spot maps. This tool identifies clusters of high values (hot spots) and low values (cold spots) creating a new output layer, RHUs by county, with a Z score, P value, and CI for each county. The Gi Bin field classifies statistically significant hot and cold spots. Counties sorted into the ± 3 category (bin) have a clustering characteristic (eg, with neighboring counties) that is statistically significant with a 99% CI; the ± 2 bin indicates a 95% CI for those county clustering sorted therein; ± 1 reflects a 90% CI; and 0 bin contains county features that have no statistical significant clustering with neighboring counties.

Data Analysis

Data were cleaned and analyzed using SAS 9.4 and R 3.5.3. Descriptive statistics are provided for sociodemographic characteristics, clinical characteristics, and class. ANOVA and t tests were used to compare continuous variables between groups, while χ² and Fisher exact tests were used for dichotomous and categorical outcome variables. The threshold for statistical significance for these tests was set at α = .001.

Results

There were 10,810 RHUs from FY 2015 through FY 2018 and HISA utilization increased each year (Figure 1). Although some years may show usage decreases relative to previous fiscal years, the cumulative trends showed an increase relative to FY 2015 for both Classes of RVs (Figure 2). There was a 45.4% increase from FY 2015 to FY 2018 with a mean 13.6% yearly increase. Class 1 increased 21.0% and Class 2 increased 39.5% from FY 2015 to FY 2016 (Figure 3).

Most RHUs were male, White, and married. Class 1 and Class 2 RHUs differed significantly by age, race, marital status, and disability conditions: Class 1 RHUs were aged 6.6 years younger with a mean age of 69.1 years compared with 75.7 years for Class 2 users. For Class 1 RHUs, a plurality (29.4%) were aged 65 to 69 years; while a plurality (41.4%) of Class 2 users were aged ≥ 80 years. Musculoskeletal was the most common identified type of condition for all RHUs (Table 1).

There were 11,166 HM prescribed to RHUs (Table 2). Bathroom HMs also were the dominant HM type for all facilities regardless of complexity levels (Table 3). The San Antonio, Texas, VAMC demonstrated the highest Class 1 vs Class 2 difference in HISA use (Class 1: 87.7% and Class 2: 12.3%). Except for the Des Moines VAMC, all other VAMCs showed HISA use > 60% by Class 1.

Cost Data

Air-conditioning installation ($5007) was the costliest HM overall (Table 4), closely followed by bathroom ($4978) and kitchen modifications ($4305). Bathroom renovations were the costliest HM type for both Class 1 and Class 2, closely followed by electrical repair and air-conditioning installation for Class 1 and driveway reconstruction and wooden ramp construction for Class 2.

The mean award received for HM was $4687 (Table 5). While the number of RHUs increased from FY 2015 to FY 2016, the average cost decreased, both overall ($280) and for Class 1 ($195) and Class 2 ($153). Except for a small decline in the number of Class 2 HISA recipients from FY 2017 to FY 2018, overall, the number of RHUs continuously grew from FY 2015 to FY 2018: 977 for the overall cohort, 678 for Class 1 and 299 for Class 2. Despite the obvious gain in the number of RHUs, the average costs did not notably change over time. VISN 21 had the highest mean cost, followed by VISNs 17, 6, 22, and 20.

Travel

Travel time and distance to the HISA prescribing facility differed significantly between Class 1 and Class 2 HISA users. RHUs had to travel about 95 minutes from their place of residence to access the HISA benefits program. There were no statistically significant differences between Class 1 and 2 users with respect to travel time and distance traveled (Table 6).

The majority of Class 1 and Class 2 veterans accessed the HISA from their nearest facility. However, nearly one-quarter of both Class 1 and 2 RHUs (24% each) did not. Among the 2598 who accessed the nonnearest facility, 97 (3.7%) accessed a facility that is ≤ 40 miles. Many (44%) users traveled 40 to 100 miles, and another 43.2% traveled 100 to 200 miles from their residence to access a HM prescription. Some 2598 users (1.1%) traveled > 500 miles to access a facility.

Discussion

Although utilization of the HISA program has steadily increased, overall participation by subpopulations such as RHUs can still be improved significantly. Veterans aged ≤ 46 years who have a disability that is common to those receiving HISA benefits have low HISA utilization. Similarly, veterans with sensory disabilities also have low use. These subpopulations are among those in great need of attention and services.

A study by Lucas and Zelaya, using the 2016 National Health Interview Survey data with an aim to measure degree of vision problems, dual sensory impairment, and hearing trouble in male veterans aged ≥ 18 years, found that veterans were more likely to report dual sensory impairment and balance difficulties when compared with nonveterans.⁹ The number of female veterans is growing but had very low representation in this study.¹⁰ This emerging VHA population requires information and education on their HM benefits.

Home Modifications

The most common HM prescribed for RHUs was for the bathroom. Further investigation is warranted as to why, given the diversity of HM types that the grant covers, low prescription rates exist across most of the HM types. There may be a lack of knowledge by providers and VWD as to the range of HMs that can be awarded under the grant. It is important that HCPs and veterans receive education on HISA HM options.

Semeah and colleagues pointed out the need for an assessment of the HISA HM ordering system to ensure that multiple HMs items (eg, kitchen, air conditioning, fees, driveway, and plumbing) are listed among the forced choices shown to clinicians to select from.⁷ Poor housing in rural America is widespread: 63% of rural dwellings need renovations and/or repairs to be accessible to individuals with disabilities, with > 6.7 million rural homes having no or faulty plumbing or kitchens; yet in this study, prescriptions for these HMs accounted for < 1%.^11,12

VISN 6 had the most HISA awards with 1364, while VISN 21 had the fewest (245). Across all VISNs, Class 1 RHUs received more prescriptions than did Class 2 RHUs. Future research may seek to examine whether prescribers are fully aware of the eligibility of HM prescription to Class 2 veterans. VISN 21 ($5354); VISN 17 ($5302); and VISN 6 ($5301) had the highest mean HM expenditures. The national mean cost for HISA HMs were $4978 for bathrooms and $4305 for kitchens; for non-HISA HMs in FY 2017, the mean costs were $6362 and $12,255, respectively. A noteworthy concern is whether the maximum grant limit awards are sufficient to perform more expensive and complex HMs, such as the kitchen or major bathroom alternations.¹³

Facilities categorized as 1a, 1b, or 1c provided prescription to more than 70.7% of all RHUs. Embedded in the infrastructures of these higher complexity level facilities are substantial research and education opportunities for staff and patients and a diversity of medical specialties. Attention must be provided to less resourced facilities located in rural areas or serving many RVs that may not have the infrastructure to monitor the HISA process or make it available to VWDs. Furthermore, the top prescribing VAMCs were almost all high-complexity facilities, and all served > 100 unique RHUs. Further research may be warranted to be better understand the variation in HM prescriptions at VAMCs.

North Florida/Sough Georgia was the highest-prescribing VAMC with 39% more HM prescriptions than the second highest prescribing facility (Durham, NC). Unfortunately, the data presented here cannot establish causality for the large variance difference between the top facilities, and the skewed distribution of total RHUs across VAMCs.

Travel-Related Variables

HISA beneficiaries face significant travel-related challenges. Just 3.6% of RHUs could access a facility within 40 miles of their home and 43.2% traveled 100 to 200 miles from their home to access a HM prescription. Further exploration is warranted to understand how travel patterns impact access to or the uptake of HISA.

RVs already have problems with accessing care because of long travel time.^14,15 The choice or necessity to travel to a farther facility for HISA prescription is problematic for RVs, especially when transportation is often reported in the literature as a barrier to resources for people living in rural communities.^15-17 When patients have travel barriers, they wait longer to obtain medical services and often wait for their conditions to worsen before seeking services.^15,18 Once HM is completed, telerehabilitation is an effective delivery method used for delivering health care services to people in remote places.^18,19 Considering that HISA use has the potential to improve quality of life, afford comfort, facilitate the accomplishment of activities of daily living for RVs, it is important that future studies examine how existing telehealth technologies can be used to improve HISA access.

Future Directions

County-level analyses is warranted in future studies exploring potential variables associated with HISA use; for example, county-level rates of primary care physicians and other HCPs. Future research should explore how long distance travel impacts the HISA application process and HM implementation. Further research also should focus on the HISA application structure and process to identify causes of delays. The HISA application process takes a mean 6 months to complete, yet the duration of hospital stays is 1 to 3 weeks, thus it is impossible to connect HISA to hospital discharge, which was the original intent of the program. Future research can examine how telehealth services can expedite HISA obtainment and coordination of the application process. Future research also may study the possible causes of the wide variations in HM prescriptions per facility. It is also important that educational programs provide information on the array of HM items that veterans can obtain.

Conclusions

In our previous study of the HISA cohort (2011-2017), we documented that an increase in utilization of the HISA program was warranted based on the low national budgetary appropriation and identification of significant low participation by vulnerable subpopulations, including veterans residing in rural areas or having returned from recent conflicts.⁷ The present study documents national utilization patterns, demographic profiles, and clinical characteristics of RHUs from FY 2015 through FY 2018, data that may be useful to policy makers and HISA administrators in predicting future use and users. It is important to note that the data and information presented in this article identify trends. The work in no way establishes a gold standard or any targeted goal of utilization. Future research could focus on conceptualizing or theorizing what steps are necessary to set such a gold standard of utilization rate and steps toward achievement.

Acknowledgments

This research was supported by grant 15521 from the US Department of Veterans Affairs, Office of Rural Health . Furthermore, the research was supported in part by grant K12 HD055929 from the National Institutes of Health.

Risk calculators can be of great value in guiding clinical decision making, patient-centered precision medicine, and resource allocation.¹ Several perioperative risk prediction models have emerged in recent decades that estimate specific hazards (eg, cardiovascular complications after noncardiac surgery) with varying accuracy and utility. In the perioperative sphere, the time windows are often limited to an index hospitalization or 30 days following surgery or discharge.^2-9 Although longer periods are of interest to patients, families, and health systems, few widely used or validated models are designed to look beyond this very narrow window.^10,11 In addition, perioperative risk prediction models do not routinely incorporate parameters of a wide variety of health or demographic domains, such as patterns of health care, health care utilization, or medication use.

In 2013, in response to the need for near real-time information to guide delivery of enhanced care management services, the Veterans Health Administration (VHA) Office of Informatics and Analytics developed automated risk prediction models that used detailed electronic health record (EHR) data. These models were used to report Care Assessment Need (CAN) scores each week for all VHA enrollees and include data from a wide array of health domains. These CAN scores predict the risk for hospitalization, death, or either event within 90 days and 1 year.^12,13 Each score is reported as both a predicted probability (0-1) and as a percentile in relation to all other VHA enrollees (a value between 1 and 99).¹³ The data used to calculate CAN scores are listed in Table 1.¹²

Methods

Following approval of the Durham Veterans Affairs Medical Center Institutional Review Board, all necessary data were extracted from the VHA Corporate Data Warehouse (CDW) repository.¹⁴ Informed consent was waived due to the minimal risk nature of the study.

We used Current Procedural Terminology codes (27438, 27446, 27447, 27486, 27487, 27488) and International Classification of Diseases, 9th edition clinical modification procedure codes (81.54, 81.55, 81.59, 00.80-00.84) to identify all veterans who had undergone primary or revision TKR between July 2014 and December 2015 in VHA Veterans Integrated Service Network 1 (Maine, Vermont, New Hampshire, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, West Virginia, Virginia, North Carolina). Because we focused on outcomes following hospital discharge, patients who died before discharge were excluded from the analysis. Preoperative CAN 1-year mortality score was chosen as the measure under the assumption that long-term survival may be the most meaningful of the 4 possible CAN score measures.

Our primary objective was to determine distribution of preoperative CAN scores in the study population. Our secondary was to study relationships among the preoperative CAN 1-year mortality scores and 1-year mortality and hospital LOS.

Study Variables

For each patient, we extracted the date of index surgery. The primary exposure or independent variable was the CAN score in the week prior to this date. Because prior study has shown that CAN scores trajectories do not significantly change over time, the date-stamped CAN scores in the week before surgery represent what would have been available to clinicians in a preoperative setting.¹⁵ Since CAN scores are refreshed and overwritten every week, we extracted archived scores from the CDW.

For the 1-year survival outcome, the primary dependent variable, we queried the vital status files in the CDW for the date of death if applicable. We confirmed survival beyond 1 year by examining vital signs in the CDW for a minimum of 2 independent encounters beyond 1 year after the date of discharge. To compute the index LOS, the secondary outcome, we computed the difference between the date of admission and date of hospital discharge.

Statistical Methods

The parameters and performance of the multivariable logistic regression models developed to compute the various CAN mortality and hospitalization risk scores have been previously described.¹² Briefly, Wang and colleagues created parsimonious regression models using backward selection. Model discrimination was evaluated using C (concordance)-statistic. Model calibration was assessed by comparing predicted vs observed event rates by risk deciles and performing Cox proportional hazards regression.

We plotted histograms to display preoperative CAN scores as a simple measure of distribution (Figure 1). We also examined the cumulative proportion of patients at each preoperative CAN 1-year mortality score.

Results

We identified 8206 patients who had undergone a TKR over the 18-month study period. The overall mean (SD) for age was 65 (8.41) years; 93% were male, and 78% were White veterans. Patient demographics are well described in a previous publication.^16,17

In terms of model parameters for the CAN score models, C-statistics for the 90-day outcome models were as follows: 0.833 for the model predicting hospitalization (95% CI, 0.832-0.834); 0.865 for the model predicting death (95% CI, 0.863-0.876); and 0.811 for the model predicting either event (95% CI, 0.810-0.812). C-statistics for the 1-year outcome models were 0.809 for the model predicting hospitalization (95% CI, 0.808-0.810); 0.851 for the model predicting death (95% CI, 0.849-0.852); and 0.787 for the model predicting either event (95% CI, 0.786-0.787). Models were well calibrated with α = 0 and β = 1, demonstrating strong agreement between observed and predicted event rates.

The distribution of preoperative CAN 1-year mortality scores was close to normal (median, 50; interquartile range, 40; mean [SD] 48 [25.6]) (eTable). The original CAN score models were developed having an equal number of patients in each strata and as such, are normally distributed.¹² Our cohort was similar in pattern of distribution. Distributions of the remaining preoperative CAN scores (90-day mortality, 1-year hospitalization, 90-day hospitalization) are shown in Figures 2, 3, and 4. Not surprisingly, histograms for both 90-day and 1-year hospitalization were skewed toward higher scores, indicating that these patients were expected to be hospitalized in the near future.

Discussion

CAN scores are automatically generated each week by EHR-based multivariable risk models. These scores have excellent predictive accuracy for 90-day and 1-year mortality and hospitalization and are routinely used by VHA primary care teams to assist with clinical operations.¹³ We studied the distribution of CAN 1-year mortality scores in a preoperative context and examined relationships of the preoperative CAN 1-year mortality scores with postoperative mortality and LOS in 8206 veterans who underwent TKR.

There are several noteworthy findings. First, the overall 1-year mortality rate observed following TKR (1.4%) was similar to other published reports.^18,19 Not surprisingly, preoperative CAN 1-year mortality scores were significantly higher in veterans who died compared with those of survivors. The majority of patients who died had a preoperative CAN 1-year mortality score > 75 while most who survived had a preoperative CAN 1-year mortality score < 45 (P < .001). Interestingly, the same scores showed a nonlinear correlation with LOS. Index LOS was about 4 days in patients in the highest tercile of CAN scores vs 2.5 days in the lowest tercile, but the initial increase in LOS was detected at a CAN score of about 55 to 60.

In addition, mortality rate varied widely in different segments of the population when grouped according to preoperative CAN scores. One-year mortality rates in the highest tercile reached 2%, about 4-fold higher than that of lower terciles (0.5%). Examination of the Kaplan-Meier curves showed that this difference in mortality between the highest tercile and the lower 2 groups appears soon after discharge and continues to increase over time, suggesting that the factors contributing to the increased mortality are present at the time of discharge and persist beyond the postoperative period. In summary, although CAN scores were not designed for use in the perioperative context, we found that preoperative CAN 1-year mortality scores are broadly predictive of mortality, but especially for increases in LOS following elective TKA, both increases in hospital LOS following elective TKA and mortality over the year after TKA.

Our findings raise several important questions. The decision to undergo elective surgery is complex. Arguably, individuals who undergo elective knee replacement should be healthy enough to undergo, recover, and reap the benefits from a procedure that does not extend life. The distribution of preoperative CAN 1-year mortality scores for our study population was similar to that of the general VHA enrollee population with similar measured mortality rates (≤ 0.5% vs ≥ 1.7% in the low and high terciles, respectively).¹ Further study comparing outcomes in matched cohorts who did and did not undergo joint replacement would be of interest. In lieu of this, though, the association of high but not extreme CAN scores with increased hospital LOS may potentially be used to guide allocation of resources to this group, obviating the increased cost and risk to which this group is exposed. And the additional insight afforded by CAN scores may enhance shared decision-making models by identifying patients at the very highest risk (eg, 1-year mortality CAN score ≥ 90), patients who conceivably might not survive long enough to recover from and enjoy their reconstructed knee, who might in the long run be harmed by undergoing the procedure.

Many total joint arthroplasties are performed in older patients, a population in which frailty is increasingly recognized as a significant risk factor for poor outcomes.^20,21 CAN scores reliably identify high-risk patients and have been shown to correlate with frailty in this group.²² Multiple authors have reported improved outcomes with cost reductions after implementation of programs targeting modifiable risk factors in high-risk surgical candidates.^23-25 A preoperative assessment that includes the CAN score may be valuable in identifying patients who would benefit most from prehabilitation programs or other interventions designed to blunt the impact of frailty. It is true that many elements used to calculate the CAN score would not be considered modifiable, especially in the short term. However, specific contributors to frailty, such as nutritional status and polypharmacy might be potential candidates. As with all multivariable risk prediction models, there are multiple paths to a high CAN score, and further research to identify clinically relevant subgroups may help inform efforts to improve perioperative care within this population.

Hospital LOS is of intense interest for many reasons, not least its utility as a surrogate for cost and increased risk for immediate perioperative adverse events, such as multidrug-resistant hospital acquired infections, need for postacute facility-based rehabilitation, and deconditioning that increase risks of falls and fractures in the older population.^26-29 In addition, its importance is magnified due to the COVID-19 pandemic context in which restarting elective surgery programs has changed traditional criteria by which patients are scheduled for surgery.

We have shown that elevated CAN scores are able to identify patients at risk for extended hospital stays and, as such, may be useful additional data in allocating scarce operating room time and other resources for optimal patient and health care provider safety.^30,31 Individual surgeons and hospital systems would, of course, decide which patients should be triaged to go first, based on local priorities; however, choosing lower risk patients with minimal risk of morbidity and mortality while pursuing prehabilitation for higher risk patients is a reasonable approach.

Limitations

Our study has several limitations. Only a single surgical procedure was included, albeit the most common one performed in the VHA. In addition, no information was available concerning the precise clinical course for these patients, such as the duration of surgery, anesthetic technique, and management of acute, perioperative course. Although the assumption was made that patients received standard care in a manner such that these factors would not significantly affect either their mortality or their LOS out of proportion to their preoperative clinical status, confounding cannot be excluded. Therefore, further study is necessary to determine whether CAN scores can accurately predict mortality and/or LOS for patients undergoing other procedures. Further, a clinical trial is required to assess whether systematic provision of the CAN score at the point of surgery would impact care and, more important, impact outcomes. In addition, multivariable analyses were not performed, including and excluding various components of the CAN score models. Currently, CAN scores could be made available to the surgical/anesthesia communities at minimal or no cost and are updated automatically. Model calibration and discrimination in this particular setting were not validated.

Because our interest is in leveraging an existing resource to a current clinical and operational problem rather than in creating or validating a new tool, we chose to test the simple bivariate relationship between preoperative CAN scores and outcomes. We chose the preoperative 1-year mortality CAN score from among the 4 options under the assumption that long-term survival is the most meaningful of the 4 candidate outcomes. Finally, while the CAN scores are currently only calculated and generated for patients cared for within the VHA, few data elements are unavailable to civilian health systems. The most problematic would be documentation of actual prescription filling, but this is a topic of increasing interest to the medical and academic communities and access to such information we hope will improve.^32-34

Conclusions

Although designed for use by VHA primary care teams, CAN scores also may have value for perioperative clinicians, predicting mortality and prolonged hospital LOS in those with elevated 1-year mortality scores. Advantages of CAN scores relative to other perioperative risk calculators lies in their ability to predict long-term rather than 30-day survival and that they are automatically generated on a near-real-time basis for all patients who receive care in VHA ambulatory clinics. Further study is needed to determine practical utility in shared decision making, preoperative evaluation and optimization, and perioperative resource allocation.

Acknowledgments

This work was supported by the US Department of Veterans Affairs (VA) National Center for Patient Safety, Field Office 10A4E, through the Patient Safety Center of Inquiry at the Durham VA Medical Center in North Carolina. The study also received support from the Center of Innovation to Accelerate Discovery and Practice Transformation (CIN 13-410) at the Durham VA Health Care System.

Risk calculators can be of great value in guiding clinical decision making, patient-centered precision medicine, and resource allocation.¹ Several perioperative risk prediction models have emerged in recent decades that estimate specific hazards (eg, cardiovascular complications after noncardiac surgery) with varying accuracy and utility. In the perioperative sphere, the time windows are often limited to an index hospitalization or 30 days following surgery or discharge.^2-9 Although longer periods are of interest to patients, families, and health systems, few widely used or validated models are designed to look beyond this very narrow window.^10,11 In addition, perioperative risk prediction models do not routinely incorporate parameters of a wide variety of health or demographic domains, such as patterns of health care, health care utilization, or medication use.

In 2013, in response to the need for near real-time information to guide delivery of enhanced care management services, the Veterans Health Administration (VHA) Office of Informatics and Analytics developed automated risk prediction models that used detailed electronic health record (EHR) data. These models were used to report Care Assessment Need (CAN) scores each week for all VHA enrollees and include data from a wide array of health domains. These CAN scores predict the risk for hospitalization, death, or either event within 90 days and 1 year.^12,13 Each score is reported as both a predicted probability (0-1) and as a percentile in relation to all other VHA enrollees (a value between 1 and 99).¹³ The data used to calculate CAN scores are listed in Table 1.¹²

Methods

Following approval of the Durham Veterans Affairs Medical Center Institutional Review Board, all necessary data were extracted from the VHA Corporate Data Warehouse (CDW) repository.¹⁴ Informed consent was waived due to the minimal risk nature of the study.

We used Current Procedural Terminology codes (27438, 27446, 27447, 27486, 27487, 27488) and International Classification of Diseases, 9th edition clinical modification procedure codes (81.54, 81.55, 81.59, 00.80-00.84) to identify all veterans who had undergone primary or revision TKR between July 2014 and December 2015 in VHA Veterans Integrated Service Network 1 (Maine, Vermont, New Hampshire, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, West Virginia, Virginia, North Carolina). Because we focused on outcomes following hospital discharge, patients who died before discharge were excluded from the analysis. Preoperative CAN 1-year mortality score was chosen as the measure under the assumption that long-term survival may be the most meaningful of the 4 possible CAN score measures.

Our primary objective was to determine distribution of preoperative CAN scores in the study population. Our secondary was to study relationships among the preoperative CAN 1-year mortality scores and 1-year mortality and hospital LOS.

Study Variables

For each patient, we extracted the date of index surgery. The primary exposure or independent variable was the CAN score in the week prior to this date. Because prior study has shown that CAN scores trajectories do not significantly change over time, the date-stamped CAN scores in the week before surgery represent what would have been available to clinicians in a preoperative setting.¹⁵ Since CAN scores are refreshed and overwritten every week, we extracted archived scores from the CDW.

For the 1-year survival outcome, the primary dependent variable, we queried the vital status files in the CDW for the date of death if applicable. We confirmed survival beyond 1 year by examining vital signs in the CDW for a minimum of 2 independent encounters beyond 1 year after the date of discharge. To compute the index LOS, the secondary outcome, we computed the difference between the date of admission and date of hospital discharge.

Statistical Methods

The parameters and performance of the multivariable logistic regression models developed to compute the various CAN mortality and hospitalization risk scores have been previously described.¹² Briefly, Wang and colleagues created parsimonious regression models using backward selection. Model discrimination was evaluated using C (concordance)-statistic. Model calibration was assessed by comparing predicted vs observed event rates by risk deciles and performing Cox proportional hazards regression.

We plotted histograms to display preoperative CAN scores as a simple measure of distribution (Figure 1). We also examined the cumulative proportion of patients at each preoperative CAN 1-year mortality score.

Results

We identified 8206 patients who had undergone a TKR over the 18-month study period. The overall mean (SD) for age was 65 (8.41) years; 93% were male, and 78% were White veterans. Patient demographics are well described in a previous publication.^16,17

In terms of model parameters for the CAN score models, C-statistics for the 90-day outcome models were as follows: 0.833 for the model predicting hospitalization (95% CI, 0.832-0.834); 0.865 for the model predicting death (95% CI, 0.863-0.876); and 0.811 for the model predicting either event (95% CI, 0.810-0.812). C-statistics for the 1-year outcome models were 0.809 for the model predicting hospitalization (95% CI, 0.808-0.810); 0.851 for the model predicting death (95% CI, 0.849-0.852); and 0.787 for the model predicting either event (95% CI, 0.786-0.787). Models were well calibrated with α = 0 and β = 1, demonstrating strong agreement between observed and predicted event rates.

The distribution of preoperative CAN 1-year mortality scores was close to normal (median, 50; interquartile range, 40; mean [SD] 48 [25.6]) (eTable). The original CAN score models were developed having an equal number of patients in each strata and as such, are normally distributed.¹² Our cohort was similar in pattern of distribution. Distributions of the remaining preoperative CAN scores (90-day mortality, 1-year hospitalization, 90-day hospitalization) are shown in Figures 2, 3, and 4. Not surprisingly, histograms for both 90-day and 1-year hospitalization were skewed toward higher scores, indicating that these patients were expected to be hospitalized in the near future.

Discussion

CAN scores are automatically generated each week by EHR-based multivariable risk models. These scores have excellent predictive accuracy for 90-day and 1-year mortality and hospitalization and are routinely used by VHA primary care teams to assist with clinical operations.¹³ We studied the distribution of CAN 1-year mortality scores in a preoperative context and examined relationships of the preoperative CAN 1-year mortality scores with postoperative mortality and LOS in 8206 veterans who underwent TKR.

There are several noteworthy findings. First, the overall 1-year mortality rate observed following TKR (1.4%) was similar to other published reports.^18,19 Not surprisingly, preoperative CAN 1-year mortality scores were significantly higher in veterans who died compared with those of survivors. The majority of patients who died had a preoperative CAN 1-year mortality score > 75 while most who survived had a preoperative CAN 1-year mortality score < 45 (P < .001). Interestingly, the same scores showed a nonlinear correlation with LOS. Index LOS was about 4 days in patients in the highest tercile of CAN scores vs 2.5 days in the lowest tercile, but the initial increase in LOS was detected at a CAN score of about 55 to 60.

In addition, mortality rate varied widely in different segments of the population when grouped according to preoperative CAN scores. One-year mortality rates in the highest tercile reached 2%, about 4-fold higher than that of lower terciles (0.5%). Examination of the Kaplan-Meier curves showed that this difference in mortality between the highest tercile and the lower 2 groups appears soon after discharge and continues to increase over time, suggesting that the factors contributing to the increased mortality are present at the time of discharge and persist beyond the postoperative period. In summary, although CAN scores were not designed for use in the perioperative context, we found that preoperative CAN 1-year mortality scores are broadly predictive of mortality, but especially for increases in LOS following elective TKA, both increases in hospital LOS following elective TKA and mortality over the year after TKA.

Our findings raise several important questions. The decision to undergo elective surgery is complex. Arguably, individuals who undergo elective knee replacement should be healthy enough to undergo, recover, and reap the benefits from a procedure that does not extend life. The distribution of preoperative CAN 1-year mortality scores for our study population was similar to that of the general VHA enrollee population with similar measured mortality rates (≤ 0.5% vs ≥ 1.7% in the low and high terciles, respectively).¹ Further study comparing outcomes in matched cohorts who did and did not undergo joint replacement would be of interest. In lieu of this, though, the association of high but not extreme CAN scores with increased hospital LOS may potentially be used to guide allocation of resources to this group, obviating the increased cost and risk to which this group is exposed. And the additional insight afforded by CAN scores may enhance shared decision-making models by identifying patients at the very highest risk (eg, 1-year mortality CAN score ≥ 90), patients who conceivably might not survive long enough to recover from and enjoy their reconstructed knee, who might in the long run be harmed by undergoing the procedure.

Many total joint arthroplasties are performed in older patients, a population in which frailty is increasingly recognized as a significant risk factor for poor outcomes.^20,21 CAN scores reliably identify high-risk patients and have been shown to correlate with frailty in this group.²² Multiple authors have reported improved outcomes with cost reductions after implementation of programs targeting modifiable risk factors in high-risk surgical candidates.^23-25 A preoperative assessment that includes the CAN score may be valuable in identifying patients who would benefit most from prehabilitation programs or other interventions designed to blunt the impact of frailty. It is true that many elements used to calculate the CAN score would not be considered modifiable, especially in the short term. However, specific contributors to frailty, such as nutritional status and polypharmacy might be potential candidates. As with all multivariable risk prediction models, there are multiple paths to a high CAN score, and further research to identify clinically relevant subgroups may help inform efforts to improve perioperative care within this population.

Hospital LOS is of intense interest for many reasons, not least its utility as a surrogate for cost and increased risk for immediate perioperative adverse events, such as multidrug-resistant hospital acquired infections, need for postacute facility-based rehabilitation, and deconditioning that increase risks of falls and fractures in the older population.^26-29 In addition, its importance is magnified due to the COVID-19 pandemic context in which restarting elective surgery programs has changed traditional criteria by which patients are scheduled for surgery.

We have shown that elevated CAN scores are able to identify patients at risk for extended hospital stays and, as such, may be useful additional data in allocating scarce operating room time and other resources for optimal patient and health care provider safety.^30,31 Individual surgeons and hospital systems would, of course, decide which patients should be triaged to go first, based on local priorities; however, choosing lower risk patients with minimal risk of morbidity and mortality while pursuing prehabilitation for higher risk patients is a reasonable approach.

Limitations

Our study has several limitations. Only a single surgical procedure was included, albeit the most common one performed in the VHA. In addition, no information was available concerning the precise clinical course for these patients, such as the duration of surgery, anesthetic technique, and management of acute, perioperative course. Although the assumption was made that patients received standard care in a manner such that these factors would not significantly affect either their mortality or their LOS out of proportion to their preoperative clinical status, confounding cannot be excluded. Therefore, further study is necessary to determine whether CAN scores can accurately predict mortality and/or LOS for patients undergoing other procedures. Further, a clinical trial is required to assess whether systematic provision of the CAN score at the point of surgery would impact care and, more important, impact outcomes. In addition, multivariable analyses were not performed, including and excluding various components of the CAN score models. Currently, CAN scores could be made available to the surgical/anesthesia communities at minimal or no cost and are updated automatically. Model calibration and discrimination in this particular setting were not validated.

Because our interest is in leveraging an existing resource to a current clinical and operational problem rather than in creating or validating a new tool, we chose to test the simple bivariate relationship between preoperative CAN scores and outcomes. We chose the preoperative 1-year mortality CAN score from among the 4 options under the assumption that long-term survival is the most meaningful of the 4 candidate outcomes. Finally, while the CAN scores are currently only calculated and generated for patients cared for within the VHA, few data elements are unavailable to civilian health systems. The most problematic would be documentation of actual prescription filling, but this is a topic of increasing interest to the medical and academic communities and access to such information we hope will improve.^32-34

Conclusions

Although designed for use by VHA primary care teams, CAN scores also may have value for perioperative clinicians, predicting mortality and prolonged hospital LOS in those with elevated 1-year mortality scores. Advantages of CAN scores relative to other perioperative risk calculators lies in their ability to predict long-term rather than 30-day survival and that they are automatically generated on a near-real-time basis for all patients who receive care in VHA ambulatory clinics. Further study is needed to determine practical utility in shared decision making, preoperative evaluation and optimization, and perioperative resource allocation.

Acknowledgments

This work was supported by the US Department of Veterans Affairs (VA) National Center for Patient Safety, Field Office 10A4E, through the Patient Safety Center of Inquiry at the Durham VA Medical Center in North Carolina. The study also received support from the Center of Innovation to Accelerate Discovery and Practice Transformation (CIN 13-410) at the Durham VA Health Care System.

Risk calculators can be of great value in guiding clinical decision making, patient-centered precision medicine, and resource allocation.¹ Several perioperative risk prediction models have emerged in recent decades that estimate specific hazards (eg, cardiovascular complications after noncardiac surgery) with varying accuracy and utility. In the perioperative sphere, the time windows are often limited to an index hospitalization or 30 days following surgery or discharge.^2-9 Although longer periods are of interest to patients, families, and health systems, few widely used or validated models are designed to look beyond this very narrow window.^10,11 In addition, perioperative risk prediction models do not routinely incorporate parameters of a wide variety of health or demographic domains, such as patterns of health care, health care utilization, or medication use.

In 2013, in response to the need for near real-time information to guide delivery of enhanced care management services, the Veterans Health Administration (VHA) Office of Informatics and Analytics developed automated risk prediction models that used detailed electronic health record (EHR) data. These models were used to report Care Assessment Need (CAN) scores each week for all VHA enrollees and include data from a wide array of health domains. These CAN scores predict the risk for hospitalization, death, or either event within 90 days and 1 year.^12,13 Each score is reported as both a predicted probability (0-1) and as a percentile in relation to all other VHA enrollees (a value between 1 and 99).¹³ The data used to calculate CAN scores are listed in Table 1.¹²

Methods

Following approval of the Durham Veterans Affairs Medical Center Institutional Review Board, all necessary data were extracted from the VHA Corporate Data Warehouse (CDW) repository.¹⁴ Informed consent was waived due to the minimal risk nature of the study.

We used Current Procedural Terminology codes (27438, 27446, 27447, 27486, 27487, 27488) and International Classification of Diseases, 9th edition clinical modification procedure codes (81.54, 81.55, 81.59, 00.80-00.84) to identify all veterans who had undergone primary or revision TKR between July 2014 and December 2015 in VHA Veterans Integrated Service Network 1 (Maine, Vermont, New Hampshire, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, West Virginia, Virginia, North Carolina). Because we focused on outcomes following hospital discharge, patients who died before discharge were excluded from the analysis. Preoperative CAN 1-year mortality score was chosen as the measure under the assumption that long-term survival may be the most meaningful of the 4 possible CAN score measures.

Our primary objective was to determine distribution of preoperative CAN scores in the study population. Our secondary was to study relationships among the preoperative CAN 1-year mortality scores and 1-year mortality and hospital LOS.

Study Variables

For each patient, we extracted the date of index surgery. The primary exposure or independent variable was the CAN score in the week prior to this date. Because prior study has shown that CAN scores trajectories do not significantly change over time, the date-stamped CAN scores in the week before surgery represent what would have been available to clinicians in a preoperative setting.¹⁵ Since CAN scores are refreshed and overwritten every week, we extracted archived scores from the CDW.

For the 1-year survival outcome, the primary dependent variable, we queried the vital status files in the CDW for the date of death if applicable. We confirmed survival beyond 1 year by examining vital signs in the CDW for a minimum of 2 independent encounters beyond 1 year after the date of discharge. To compute the index LOS, the secondary outcome, we computed the difference between the date of admission and date of hospital discharge.

Statistical Methods

The parameters and performance of the multivariable logistic regression models developed to compute the various CAN mortality and hospitalization risk scores have been previously described.¹² Briefly, Wang and colleagues created parsimonious regression models using backward selection. Model discrimination was evaluated using C (concordance)-statistic. Model calibration was assessed by comparing predicted vs observed event rates by risk deciles and performing Cox proportional hazards regression.

We plotted histograms to display preoperative CAN scores as a simple measure of distribution (Figure 1). We also examined the cumulative proportion of patients at each preoperative CAN 1-year mortality score.

Results

We identified 8206 patients who had undergone a TKR over the 18-month study period. The overall mean (SD) for age was 65 (8.41) years; 93% were male, and 78% were White veterans. Patient demographics are well described in a previous publication.^16,17

In terms of model parameters for the CAN score models, C-statistics for the 90-day outcome models were as follows: 0.833 for the model predicting hospitalization (95% CI, 0.832-0.834); 0.865 for the model predicting death (95% CI, 0.863-0.876); and 0.811 for the model predicting either event (95% CI, 0.810-0.812). C-statistics for the 1-year outcome models were 0.809 for the model predicting hospitalization (95% CI, 0.808-0.810); 0.851 for the model predicting death (95% CI, 0.849-0.852); and 0.787 for the model predicting either event (95% CI, 0.786-0.787). Models were well calibrated with α = 0 and β = 1, demonstrating strong agreement between observed and predicted event rates.

The distribution of preoperative CAN 1-year mortality scores was close to normal (median, 50; interquartile range, 40; mean [SD] 48 [25.6]) (eTable). The original CAN score models were developed having an equal number of patients in each strata and as such, are normally distributed.¹² Our cohort was similar in pattern of distribution. Distributions of the remaining preoperative CAN scores (90-day mortality, 1-year hospitalization, 90-day hospitalization) are shown in Figures 2, 3, and 4. Not surprisingly, histograms for both 90-day and 1-year hospitalization were skewed toward higher scores, indicating that these patients were expected to be hospitalized in the near future.

Discussion

CAN scores are automatically generated each week by EHR-based multivariable risk models. These scores have excellent predictive accuracy for 90-day and 1-year mortality and hospitalization and are routinely used by VHA primary care teams to assist with clinical operations.¹³ We studied the distribution of CAN 1-year mortality scores in a preoperative context and examined relationships of the preoperative CAN 1-year mortality scores with postoperative mortality and LOS in 8206 veterans who underwent TKR.

There are several noteworthy findings. First, the overall 1-year mortality rate observed following TKR (1.4%) was similar to other published reports.^18,19 Not surprisingly, preoperative CAN 1-year mortality scores were significantly higher in veterans who died compared with those of survivors. The majority of patients who died had a preoperative CAN 1-year mortality score > 75 while most who survived had a preoperative CAN 1-year mortality score < 45 (P < .001). Interestingly, the same scores showed a nonlinear correlation with LOS. Index LOS was about 4 days in patients in the highest tercile of CAN scores vs 2.5 days in the lowest tercile, but the initial increase in LOS was detected at a CAN score of about 55 to 60.

In addition, mortality rate varied widely in different segments of the population when grouped according to preoperative CAN scores. One-year mortality rates in the highest tercile reached 2%, about 4-fold higher than that of lower terciles (0.5%). Examination of the Kaplan-Meier curves showed that this difference in mortality between the highest tercile and the lower 2 groups appears soon after discharge and continues to increase over time, suggesting that the factors contributing to the increased mortality are present at the time of discharge and persist beyond the postoperative period. In summary, although CAN scores were not designed for use in the perioperative context, we found that preoperative CAN 1-year mortality scores are broadly predictive of mortality, but especially for increases in LOS following elective TKA, both increases in hospital LOS following elective TKA and mortality over the year after TKA.

Our findings raise several important questions. The decision to undergo elective surgery is complex. Arguably, individuals who undergo elective knee replacement should be healthy enough to undergo, recover, and reap the benefits from a procedure that does not extend life. The distribution of preoperative CAN 1-year mortality scores for our study population was similar to that of the general VHA enrollee population with similar measured mortality rates (≤ 0.5% vs ≥ 1.7% in the low and high terciles, respectively).¹ Further study comparing outcomes in matched cohorts who did and did not undergo joint replacement would be of interest. In lieu of this, though, the association of high but not extreme CAN scores with increased hospital LOS may potentially be used to guide allocation of resources to this group, obviating the increased cost and risk to which this group is exposed. And the additional insight afforded by CAN scores may enhance shared decision-making models by identifying patients at the very highest risk (eg, 1-year mortality CAN score ≥ 90), patients who conceivably might not survive long enough to recover from and enjoy their reconstructed knee, who might in the long run be harmed by undergoing the procedure.

Many total joint arthroplasties are performed in older patients, a population in which frailty is increasingly recognized as a significant risk factor for poor outcomes.^20,21 CAN scores reliably identify high-risk patients and have been shown to correlate with frailty in this group.²² Multiple authors have reported improved outcomes with cost reductions after implementation of programs targeting modifiable risk factors in high-risk surgical candidates.^23-25 A preoperative assessment that includes the CAN score may be valuable in identifying patients who would benefit most from prehabilitation programs or other interventions designed to blunt the impact of frailty. It is true that many elements used to calculate the CAN score would not be considered modifiable, especially in the short term. However, specific contributors to frailty, such as nutritional status and polypharmacy might be potential candidates. As with all multivariable risk prediction models, there are multiple paths to a high CAN score, and further research to identify clinically relevant subgroups may help inform efforts to improve perioperative care within this population.

Hospital LOS is of intense interest for many reasons, not least its utility as a surrogate for cost and increased risk for immediate perioperative adverse events, such as multidrug-resistant hospital acquired infections, need for postacute facility-based rehabilitation, and deconditioning that increase risks of falls and fractures in the older population.^26-29 In addition, its importance is magnified due to the COVID-19 pandemic context in which restarting elective surgery programs has changed traditional criteria by which patients are scheduled for surgery.

We have shown that elevated CAN scores are able to identify patients at risk for extended hospital stays and, as such, may be useful additional data in allocating scarce operating room time and other resources for optimal patient and health care provider safety.^30,31 Individual surgeons and hospital systems would, of course, decide which patients should be triaged to go first, based on local priorities; however, choosing lower risk patients with minimal risk of morbidity and mortality while pursuing prehabilitation for higher risk patients is a reasonable approach.

Limitations

Our study has several limitations. Only a single surgical procedure was included, albeit the most common one performed in the VHA. In addition, no information was available concerning the precise clinical course for these patients, such as the duration of surgery, anesthetic technique, and management of acute, perioperative course. Although the assumption was made that patients received standard care in a manner such that these factors would not significantly affect either their mortality or their LOS out of proportion to their preoperative clinical status, confounding cannot be excluded. Therefore, further study is necessary to determine whether CAN scores can accurately predict mortality and/or LOS for patients undergoing other procedures. Further, a clinical trial is required to assess whether systematic provision of the CAN score at the point of surgery would impact care and, more important, impact outcomes. In addition, multivariable analyses were not performed, including and excluding various components of the CAN score models. Currently, CAN scores could be made available to the surgical/anesthesia communities at minimal or no cost and are updated automatically. Model calibration and discrimination in this particular setting were not validated.

Because our interest is in leveraging an existing resource to a current clinical and operational problem rather than in creating or validating a new tool, we chose to test the simple bivariate relationship between preoperative CAN scores and outcomes. We chose the preoperative 1-year mortality CAN score from among the 4 options under the assumption that long-term survival is the most meaningful of the 4 candidate outcomes. Finally, while the CAN scores are currently only calculated and generated for patients cared for within the VHA, few data elements are unavailable to civilian health systems. The most problematic would be documentation of actual prescription filling, but this is a topic of increasing interest to the medical and academic communities and access to such information we hope will improve.^32-34

Conclusions

Although designed for use by VHA primary care teams, CAN scores also may have value for perioperative clinicians, predicting mortality and prolonged hospital LOS in those with elevated 1-year mortality scores. Advantages of CAN scores relative to other perioperative risk calculators lies in their ability to predict long-term rather than 30-day survival and that they are automatically generated on a near-real-time basis for all patients who receive care in VHA ambulatory clinics. Further study is needed to determine practical utility in shared decision making, preoperative evaluation and optimization, and perioperative resource allocation.

Acknowledgments

This work was supported by the US Department of Veterans Affairs (VA) National Center for Patient Safety, Field Office 10A4E, through the Patient Safety Center of Inquiry at the Durham VA Medical Center in North Carolina. The study also received support from the Center of Innovation to Accelerate Discovery and Practice Transformation (CIN 13-410) at the Durham VA Health Care System.

Women make up an estimated 10% of the veteran population.¹ The US Department of Veterans Affairs (VA) projected that there would be an increase of 18,000 female veterans per year for 10 years based on 2015 data. The number of women veterans enrolled in the VA health care increased from 397,024 to 729,989 (83.9%) between 2005 and 2015.² This rise in the number of enrolled women veterans also increased the demand for female-specific health care services, such as breast reduction surgery, a reconstructive procedure provided at the Malcom Randall VA Medical Center (MRVAMC) federal teaching hospital in Gainesville, Florida.

Patients who experience symptomatic macromastia will report a history of neck and shoulder pain, shoulder grooving from bra straps, inframammary intertrigo, difficulty finding clothes that fit, and discomfort participating in sports. For the treatment of symptomatic macromastia, patients report a high satisfaction rate after breast reduction surgery.^3-5 Unfortunately, the complications from the surgery can significantly disrupt a woman’s life due to previously unplanned hospital admissions, clinic appointments, wound care, time off work, and poor aesthetic outcome. Faculty awareness of a large number of complications for patients after breast reduction surgery prompted the MRVAMC Plastic Surgery Service to establish a stricter surgical screening protocol using body mass index (BMI) values and negative nicotine status to help patients be healthier and reduce the potential risk before offering surgery. A medical literature search did not find an existing study on veteran-specific breast reduction surgery.

Methods

The University of Florida and North Florida/South Georgia Veterans Health System Institutional Review Board approved a retrospective chart review of all breast reduction surgeries performed at MRVAMC over a 20-year period (July 1, 2000-June 30, 2020). Electronic health records were queried for all primary bilateral breast reduction surgeries performed for symptomatic macromastia using Current Procedural Terminology code 19318. Potentially modifiable or predictable risk factors for wound complications were recorded: nicotine status, BMI, diabetes mellitus (DM) status, skin incision pattern, and pedicle location. Skin incision patterns were either vertical (periareolar plus a vertical scar from the areola to the inframammary fold) or traditional Wise pattern (also known as anchor pattern: periareolar scar, vertical scar to inframammary fold, plus a horizontal scar along the inframammary fold) as seen in Figures 1 and 2. The pedicle is the source of blood supply to the nipple, which was documented as either from the inferior aspect or the superior or superior/medial aspect.

For this study, the blood supply from the superior and superior/medial was logged in the same category. Records were reviewed 3 months after surgery for documentation of local wound complications, such as hematoma, infection, wound breakdown, skin necrosis, and nipple necrosis. Major complications were defined as requiring an unplanned hospital admission or urgent return to the operating room. A χ² test using a P value of < .05 was used to determine statistical significance between the incidence of wound complications and the individually identifiable variables.

Results

One hundred fifteen bilateral breast reduction surgeries were performed at MRVAMC over a 20-year period. Patient median age was 43 years. Median combined specimen weight was 1272 g. Forty-eight (41.7%) wound complications were documented, including 8 (7%) major complications. Most complications were identified in the first 7 years of the study before the new protocol and consult template became active. The new template resulted in the local complication rate dropping from 62% (July 2000-June 2007) to 26% (July 2007-June 2020). BMI > 32 (P = .03) and active nicotine use (P = .004) were found to be statistically significant independent risk factors for wound complications. Median BMI for all patients was 30. DM status (P = .22), skin incision pattern (P = .25), and pedicle location (P = .13) were not found to be predictors of wound complications (Table). There was no significant change in the incidence of major complications before and after the new protocols were enforced.

Discussion

Breast reduction surgery is an elective reconstructive option to treat symptomatic macromastia. There are several accepted ways to do the reduction surgical procedure where the blood supply (pedicle) to the nipple can vary and the visible scars can be in a horizontal, vertical, or Wise pattern. Technique is usually based on surgeon training, comfort, and preference. There are several known complications specific to this operation that include asymmetry, changes in nipple sensation, unattractive scars, diminished ability to breastfeed, and wound complications.^5-7 Wound complications include seroma, hematoma, dehiscence, infection, wound breakdown, skin necrosis, and nipple necrosis.

This study focused on wound complications with the objective of identifying and modifying risk factors. Two known risk factors documented in the literature, nicotine use and obesity, already had been addressed by our service, and results were known anecdotally but had not been previously verified. This study also looked at other potential risk factors, including the pedicle location, skin incision, and DM status.

Residents or fellows participated in all the surgeries. An outcome analysis from The American College of Surgeons National Surgical Quality Improvement Program database from 2005 to 2011 found that resident participation was associated with morbidity, including wound complications.⁸ This study was performed at a federal hospital with a complexity level 1a rating, which is designated based on the highest level of patient volume, risk, teaching, research, intensive care unit beds, and specialty services.⁹ The hospital is closely affiliated with a level 1 trauma center and teaching hospital; therefore, resident and fellow participation is not a modifiable risk factor.

Limitations and Strengths

This retrospective review comprised multiple different surgeons, including faculty and trainees, who were involved in the consultation, surgery, and postoperative care of the patients over a 20-year period; therefore, consistency in documentation is lacking. In addition, we were limited to only the information available on the charts. For example, wound size and laterality were not consistently documented. The MRVAMC complication rate was consistent with the current literature (range, 14-52%).^12,18,20,24

The major strength of the study is that the veterans tend to stay within the VA, which makes complications easier to identify and follow. Patients who do not present initially to their surgeon due to travel limitations will typically contact their primary care provider or present to their local VA urgent care or emergency department provider, who will route the patient back to the surgical specialty service through the electronic health record.

Conclusions

Breast reduction surgery has a high wound complication rate, which can be predicted and improved on so that patients can receive their indicated surgical procedure with minimal inconvenience and downtime. This review confirms that preoperative weight loss and nicotine cessation were the appropriate focus of the MRVAMC plastic surgery service’s efforts to achieve a safer surgical experience. We will continue to enforce our protocol and encourage patients who are interested in breast reduction surgery and fall outside the requirements to work with their primary care provider on smoking cessation and weight loss through better nutrition and physical activity.

Acknowledgment
This manuscript is the result of work supported with resources and the use of facilities at the North Florida/South Georgia Veterans Health System in Gainesville, Florida.

Women make up an estimated 10% of the veteran population.¹ The US Department of Veterans Affairs (VA) projected that there would be an increase of 18,000 female veterans per year for 10 years based on 2015 data. The number of women veterans enrolled in the VA health care increased from 397,024 to 729,989 (83.9%) between 2005 and 2015.² This rise in the number of enrolled women veterans also increased the demand for female-specific health care services, such as breast reduction surgery, a reconstructive procedure provided at the Malcom Randall VA Medical Center (MRVAMC) federal teaching hospital in Gainesville, Florida.

Patients who experience symptomatic macromastia will report a history of neck and shoulder pain, shoulder grooving from bra straps, inframammary intertrigo, difficulty finding clothes that fit, and discomfort participating in sports. For the treatment of symptomatic macromastia, patients report a high satisfaction rate after breast reduction surgery.^3-5 Unfortunately, the complications from the surgery can significantly disrupt a woman’s life due to previously unplanned hospital admissions, clinic appointments, wound care, time off work, and poor aesthetic outcome. Faculty awareness of a large number of complications for patients after breast reduction surgery prompted the MRVAMC Plastic Surgery Service to establish a stricter surgical screening protocol using body mass index (BMI) values and negative nicotine status to help patients be healthier and reduce the potential risk before offering surgery. A medical literature search did not find an existing study on veteran-specific breast reduction surgery.

Methods

The University of Florida and North Florida/South Georgia Veterans Health System Institutional Review Board approved a retrospective chart review of all breast reduction surgeries performed at MRVAMC over a 20-year period (July 1, 2000-June 30, 2020). Electronic health records were queried for all primary bilateral breast reduction surgeries performed for symptomatic macromastia using Current Procedural Terminology code 19318. Potentially modifiable or predictable risk factors for wound complications were recorded: nicotine status, BMI, diabetes mellitus (DM) status, skin incision pattern, and pedicle location. Skin incision patterns were either vertical (periareolar plus a vertical scar from the areola to the inframammary fold) or traditional Wise pattern (also known as anchor pattern: periareolar scar, vertical scar to inframammary fold, plus a horizontal scar along the inframammary fold) as seen in Figures 1 and 2. The pedicle is the source of blood supply to the nipple, which was documented as either from the inferior aspect or the superior or superior/medial aspect.

For this study, the blood supply from the superior and superior/medial was logged in the same category. Records were reviewed 3 months after surgery for documentation of local wound complications, such as hematoma, infection, wound breakdown, skin necrosis, and nipple necrosis. Major complications were defined as requiring an unplanned hospital admission or urgent return to the operating room. A χ² test using a P value of < .05 was used to determine statistical significance between the incidence of wound complications and the individually identifiable variables.

Results

One hundred fifteen bilateral breast reduction surgeries were performed at MRVAMC over a 20-year period. Patient median age was 43 years. Median combined specimen weight was 1272 g. Forty-eight (41.7%) wound complications were documented, including 8 (7%) major complications. Most complications were identified in the first 7 years of the study before the new protocol and consult template became active. The new template resulted in the local complication rate dropping from 62% (July 2000-June 2007) to 26% (July 2007-June 2020). BMI > 32 (P = .03) and active nicotine use (P = .004) were found to be statistically significant independent risk factors for wound complications. Median BMI for all patients was 30. DM status (P = .22), skin incision pattern (P = .25), and pedicle location (P = .13) were not found to be predictors of wound complications (Table). There was no significant change in the incidence of major complications before and after the new protocols were enforced.

Discussion

Breast reduction surgery is an elective reconstructive option to treat symptomatic macromastia. There are several accepted ways to do the reduction surgical procedure where the blood supply (pedicle) to the nipple can vary and the visible scars can be in a horizontal, vertical, or Wise pattern. Technique is usually based on surgeon training, comfort, and preference. There are several known complications specific to this operation that include asymmetry, changes in nipple sensation, unattractive scars, diminished ability to breastfeed, and wound complications.^5-7 Wound complications include seroma, hematoma, dehiscence, infection, wound breakdown, skin necrosis, and nipple necrosis.

This study focused on wound complications with the objective of identifying and modifying risk factors. Two known risk factors documented in the literature, nicotine use and obesity, already had been addressed by our service, and results were known anecdotally but had not been previously verified. This study also looked at other potential risk factors, including the pedicle location, skin incision, and DM status.

Residents or fellows participated in all the surgeries. An outcome analysis from The American College of Surgeons National Surgical Quality Improvement Program database from 2005 to 2011 found that resident participation was associated with morbidity, including wound complications.⁸ This study was performed at a federal hospital with a complexity level 1a rating, which is designated based on the highest level of patient volume, risk, teaching, research, intensive care unit beds, and specialty services.⁹ The hospital is closely affiliated with a level 1 trauma center and teaching hospital; therefore, resident and fellow participation is not a modifiable risk factor.

Limitations and Strengths

This retrospective review comprised multiple different surgeons, including faculty and trainees, who were involved in the consultation, surgery, and postoperative care of the patients over a 20-year period; therefore, consistency in documentation is lacking. In addition, we were limited to only the information available on the charts. For example, wound size and laterality were not consistently documented. The MRVAMC complication rate was consistent with the current literature (range, 14-52%).^12,18,20,24

The major strength of the study is that the veterans tend to stay within the VA, which makes complications easier to identify and follow. Patients who do not present initially to their surgeon due to travel limitations will typically contact their primary care provider or present to their local VA urgent care or emergency department provider, who will route the patient back to the surgical specialty service through the electronic health record.

Conclusions

Breast reduction surgery has a high wound complication rate, which can be predicted and improved on so that patients can receive their indicated surgical procedure with minimal inconvenience and downtime. This review confirms that preoperative weight loss and nicotine cessation were the appropriate focus of the MRVAMC plastic surgery service’s efforts to achieve a safer surgical experience. We will continue to enforce our protocol and encourage patients who are interested in breast reduction surgery and fall outside the requirements to work with their primary care provider on smoking cessation and weight loss through better nutrition and physical activity.

Acknowledgment
This manuscript is the result of work supported with resources and the use of facilities at the North Florida/South Georgia Veterans Health System in Gainesville, Florida.

Women make up an estimated 10% of the veteran population.¹ The US Department of Veterans Affairs (VA) projected that there would be an increase of 18,000 female veterans per year for 10 years based on 2015 data. The number of women veterans enrolled in the VA health care increased from 397,024 to 729,989 (83.9%) between 2005 and 2015.² This rise in the number of enrolled women veterans also increased the demand for female-specific health care services, such as breast reduction surgery, a reconstructive procedure provided at the Malcom Randall VA Medical Center (MRVAMC) federal teaching hospital in Gainesville, Florida.

Patients who experience symptomatic macromastia will report a history of neck and shoulder pain, shoulder grooving from bra straps, inframammary intertrigo, difficulty finding clothes that fit, and discomfort participating in sports. For the treatment of symptomatic macromastia, patients report a high satisfaction rate after breast reduction surgery.^3-5 Unfortunately, the complications from the surgery can significantly disrupt a woman’s life due to previously unplanned hospital admissions, clinic appointments, wound care, time off work, and poor aesthetic outcome. Faculty awareness of a large number of complications for patients after breast reduction surgery prompted the MRVAMC Plastic Surgery Service to establish a stricter surgical screening protocol using body mass index (BMI) values and negative nicotine status to help patients be healthier and reduce the potential risk before offering surgery. A medical literature search did not find an existing study on veteran-specific breast reduction surgery.

Methods

The University of Florida and North Florida/South Georgia Veterans Health System Institutional Review Board approved a retrospective chart review of all breast reduction surgeries performed at MRVAMC over a 20-year period (July 1, 2000-June 30, 2020). Electronic health records were queried for all primary bilateral breast reduction surgeries performed for symptomatic macromastia using Current Procedural Terminology code 19318. Potentially modifiable or predictable risk factors for wound complications were recorded: nicotine status, BMI, diabetes mellitus (DM) status, skin incision pattern, and pedicle location. Skin incision patterns were either vertical (periareolar plus a vertical scar from the areola to the inframammary fold) or traditional Wise pattern (also known as anchor pattern: periareolar scar, vertical scar to inframammary fold, plus a horizontal scar along the inframammary fold) as seen in Figures 1 and 2. The pedicle is the source of blood supply to the nipple, which was documented as either from the inferior aspect or the superior or superior/medial aspect.

For this study, the blood supply from the superior and superior/medial was logged in the same category. Records were reviewed 3 months after surgery for documentation of local wound complications, such as hematoma, infection, wound breakdown, skin necrosis, and nipple necrosis. Major complications were defined as requiring an unplanned hospital admission or urgent return to the operating room. A χ² test using a P value of < .05 was used to determine statistical significance between the incidence of wound complications and the individually identifiable variables.

Results

One hundred fifteen bilateral breast reduction surgeries were performed at MRVAMC over a 20-year period. Patient median age was 43 years. Median combined specimen weight was 1272 g. Forty-eight (41.7%) wound complications were documented, including 8 (7%) major complications. Most complications were identified in the first 7 years of the study before the new protocol and consult template became active. The new template resulted in the local complication rate dropping from 62% (July 2000-June 2007) to 26% (July 2007-June 2020). BMI > 32 (P = .03) and active nicotine use (P = .004) were found to be statistically significant independent risk factors for wound complications. Median BMI for all patients was 30. DM status (P = .22), skin incision pattern (P = .25), and pedicle location (P = .13) were not found to be predictors of wound complications (Table). There was no significant change in the incidence of major complications before and after the new protocols were enforced.

Discussion

Breast reduction surgery is an elective reconstructive option to treat symptomatic macromastia. There are several accepted ways to do the reduction surgical procedure where the blood supply (pedicle) to the nipple can vary and the visible scars can be in a horizontal, vertical, or Wise pattern. Technique is usually based on surgeon training, comfort, and preference. There are several known complications specific to this operation that include asymmetry, changes in nipple sensation, unattractive scars, diminished ability to breastfeed, and wound complications.^5-7 Wound complications include seroma, hematoma, dehiscence, infection, wound breakdown, skin necrosis, and nipple necrosis.

This study focused on wound complications with the objective of identifying and modifying risk factors. Two known risk factors documented in the literature, nicotine use and obesity, already had been addressed by our service, and results were known anecdotally but had not been previously verified. This study also looked at other potential risk factors, including the pedicle location, skin incision, and DM status.

Residents or fellows participated in all the surgeries. An outcome analysis from The American College of Surgeons National Surgical Quality Improvement Program database from 2005 to 2011 found that resident participation was associated with morbidity, including wound complications.⁸ This study was performed at a federal hospital with a complexity level 1a rating, which is designated based on the highest level of patient volume, risk, teaching, research, intensive care unit beds, and specialty services.⁹ The hospital is closely affiliated with a level 1 trauma center and teaching hospital; therefore, resident and fellow participation is not a modifiable risk factor.

Limitations and Strengths

This retrospective review comprised multiple different surgeons, including faculty and trainees, who were involved in the consultation, surgery, and postoperative care of the patients over a 20-year period; therefore, consistency in documentation is lacking. In addition, we were limited to only the information available on the charts. For example, wound size and laterality were not consistently documented. The MRVAMC complication rate was consistent with the current literature (range, 14-52%).^12,18,20,24

The major strength of the study is that the veterans tend to stay within the VA, which makes complications easier to identify and follow. Patients who do not present initially to their surgeon due to travel limitations will typically contact their primary care provider or present to their local VA urgent care or emergency department provider, who will route the patient back to the surgical specialty service through the electronic health record.

Conclusions

Breast reduction surgery has a high wound complication rate, which can be predicted and improved on so that patients can receive their indicated surgical procedure with minimal inconvenience and downtime. This review confirms that preoperative weight loss and nicotine cessation were the appropriate focus of the MRVAMC plastic surgery service’s efforts to achieve a safer surgical experience. We will continue to enforce our protocol and encourage patients who are interested in breast reduction surgery and fall outside the requirements to work with their primary care provider on smoking cessation and weight loss through better nutrition and physical activity.

Acknowledgment
This manuscript is the result of work supported with resources and the use of facilities at the North Florida/South Georgia Veterans Health System in Gainesville, Florida.

According to the Centers for Disease Control and Prevention National Center for Health Statistics (NCHS), the use of prescription drugs has increased in the past half century. Although prescription drugs have played an important role in preventing, controlling, and delaying onset or progression of disease, their growth in use also has posed many risks.¹ One ramification of this growth is the occurrence of polypharmacy, which does not have a universal, clear definition. In general, it can be described as the concurrent use of multiple medications by a single patient to treat one or more medical ailments. Five or more medications taken simultaneously is the most common definition to date, but this is just one of many acceptable definitions and that varies from one health care facility to another.^1,2

Regardless of the cutoffs established to indicate polypharmacy, its incidence can result in poor and potentially harmful health outcomes. Polypharmacy increases the risk of experiencing adverse drug events (ADEs), drug-drug interactions (DDIs), geriatric-related syndromes, falls, hospitalization, and mortality. Issues with adherence may begin to unfold secondary to increased pill burden. Both the patient and the health care system may encounter financial strain, as polypharmacy can lead to unnecessary and essentially preventable costs of care. When evaluating the likelihood of polypharmacy based on age group, NCHS found that 47.5% of patients taking ≥ 5 medications were aged ≥ 65 years.^1-5 This indicates that polypharmacy is of great concern in the geriatric population, which also represents a large proportion of individuals accessing Veterans Health Administration (VHA) care.

Deprescibing

Deprescribing is the act of withdrawing or discontinuing potentially inappropriate medications (PIM), or medications used by older patients harboring ADEs that generally outweigh the clinical benefits of the drug. Deprescribing is an effective tool for managing or reducing polypharmacy. A variety of tools have been created whose sole purpose is to simplify deprescribing. Some tools explicitly identify PIM and are widely familiar in medical practice. Examples are the Beers Criteria developed in 1991 or Screening Tool to Alert Right Treatment/Screening Tool of Older Persons Prescriptions (START/STOPP) criteria created in 2003. Other tools that are less commonplace but equally as resourceful are MedStopper and Deprescribing.org. The former was launched in 2015 and is a Canadian online system that provides risk assessments for medications with guidance for tapering or stopping medications if continuation of the drug presents higher risk than benefit.^5-7 The latter is a full-fledged website developed by a physician, a pharmacist, and their research teams that serves as an exchange hub for deprescribing information.

In 2016, the VIONE (Vital, Important, Optional, Not indicated/treatment complete, and Every medication has an indication) deprescribing tool was developed by Saraswathy Battar, MD, at Central Arkansas Veterans Healthcare System (CAVHS) in Little Rock, as a system that could go beyond medication reconciliation (Table 1). Health care providers (HCPs) and pharmacists evaluate each medication that a patient has been prescribed and places each medication in a VIONE category. Prescribers may then take the opportunity to deprescribe or discontinue medications if deemed appropriate based on their clinical assessments and shared decision making.⁸ Traditionally, medication reconciliation involves the process of obtaining a complete and accurate list of medications as reported by a patient or caregiver to a HCP. VIONE encourages HCPs and pharmacists not only to ensure medication lists are accurate, but also that each medication reported is appropriate for continued use. In other words, VIONE is meant to help implement deprescribing at opportune times. More than 14,000 medications have been deprescribed using the VIONE method, resulting in more than $2,000,000 of annualized cost avoidance after just 1 year of implementation at CAVHS.⁹

Methods

Veterans receiving care at VASNHS aged ≥ 65 years with ≥ 10 active medications noted in CPRS were included in this project. PACT pharmacists and physicians were educated on the proper use of the VIONE tool prior to its implementation. Education included a 15-minute slide presentation followed by dissemination of a 1-page VIONE tool handout during a PACT all-staff clinic meeting.

Data were collected for 3 months before and after the intervention. Data were made available for assessment by the Automated Data Processing Application Coordinator (ADPAC) at VASNHS. The ADPAC created and generated an Excel spreadsheet report, which listed all medications deprescribed using the VIONE method. The primary endpoint was the total number of medications discontinued using the VIONE template and/or discontinuation menu. For the purpose of this project, appropriate discontinuation was considered any prescription deprescribed, excluding medical supplies, by pharmacists and PCPs who received VIONE education.

Results

A total of 2,442 veterans met inclusion criteria, and the VIONE method was applied to 598 between late October 2018 and January 2019. The 13 PACT pharmacists contacted at least 10 veterans each, thus at least 130 were randomly selected for telephone calls to perform polypharmacy reviews using the VIONE note template. The discontinuation menu was used if a medication qualified to be deprescribed. After 3 months, 1986 prescriptions were deprescribed using VIONE; however, 1060 prescriptions were considered appropriately deprescribed (Table 2). The 13 PACT pharmacists deprescribed 361 medications, and the 29 PACT physicians deprescribed 699 medications. These prescriptions were then separated into medication categories to determine the most common discontinued classes. Vitamins and supplements were the medication class most frequently deprescribed (19.4%), followed by pain medications (15.5%), antimicrobial agents (9.6%), antihypertensive medications (9.2%), and diabetes medications (6.4%) (Table 3). The top 5 medication categories accounted for 60% of all medications appropriately deprescribed.

The estimated annualized cost avoidance for all medications deprescribed in the 3-month project period was $84,030.46. To provide the most appropriate and accurate calculation, medication classes excluded from this figure were acute or short-term prescriptions and antimicrobial agents. Medications prescribed short-term typically are not suitable to continue for an extended period, and antimicrobial agents were excluded since they are normally associated with higher costs, and may overestimate the cost avoidance calculation for the facility.

Discussion

The outcomes for the primary and secondary endpoints of this project illustrate that using VIONE in PACT primary care clinics had a notable impact on polypharmacy and cost avoidance over a short period. This outcome can be attributed to 2 significant effects of using the deprescribing tool. VIONE’s simplicity in application allowed clinicians to incorporate daily use of the tool with minimal effort. Education was all that was required to fully enable clinicians to work together successfully and exercise collaborative practice to promote deprescribing. VIONE also elicited a cascade of favorable effects that improve patient safety and health outcomes. The tool aided in identification of PIM, which helped reduce polypharmacy and medication burden. The risk for DDIs and ADEs may decrease; therefore, the incidence of falls, need for emergency department visits or inpatient care related to polypharmacy may decline. Less complex medication regimens may alleviate issues with adherence and avoid the various consequences of polypharmacy in theory. Simplified regimens can potentially improve disease management and quality of life for patients. Further studies are needed to substantiate deprescribing and its true effect on patient adherence and better health outcomes at this time.¹⁰

Reducing polypharmacy can lead to cost savings. Based on the results of this 3-month study, we expect that VASNHS would save more than $84,000 by reducing polypharmacy among its patients. Those savings can be funneled back into the health care system, and allotted to necessary patient care, prescriptions, and health care facility needs.

Limitations

There are some important limitations to this study. Definitions of polypharmacy may vary from one health care facility to another. The cutoffs for polypharmacy may differ, causing the prevalence of polypharmacy and potential costs savings to vary. Use of VIONE may be inconsistent among users if not previously educated or properly trained. For instance, VIONE selections are listed in the same menu as the standard CPRS discontinuation options, which may lead to discontinuation of medical supplies or laboratory orders instead of prescriptions.

The method of data analysis and project design used in this study may have been subject to error. For example, the list of PCPs may have been inaccurate or outdated, which would result in an over- or underrepresentation of those who contributed to data collection. Furthermore, there is some volatility in calculating the total cost avoidance. For example, medications for chronic conditions that were only taken on an as needed basis may have overestimated savings. Either under- or overestimations could occur when parameters are adjusted on the VIONE discontinuation dashboard without appropriate guidance. With the ability to manually adjust the dashboard parameters, dissimilarities in calculations may follow.

Conclusions

The VIONE tool may be useful in improving patient safety through deprescribing and discontinuing PIM. Decreasing the number of medications being taken concomitantly by a patient and continuing only those that are imperative in their medical treatment is the first step to reducing the incidence of polypharmacy. Consequently, chances of ADEs or DDIs are lessened, especially among older individuals who are considered high risk for experiencing the detrimental effects that may ensue. These effects include geriatric-related syndromes, increased risk of fall, hospital visits or admissions, or death. Use of VIONE easily promotes collaboration among clinicians to evaluate medications eligible for discontinuation more regularly. If this deprescribing tool is continuously used, costs avoided can likely be maximized within VA health care systems.

The results of this project should serve as an incentive to push for better prescribing practices and increase deprescribing efforts. It should provoke the need for change in regimens and the subsequent discontinuation of prescriptions that are not considered vital to continue. Finally, the result of this project should substantiate the positive impact a deprescribing tool can possess to avert the issues commonly associated with polypharmacy.

According to the Centers for Disease Control and Prevention National Center for Health Statistics (NCHS), the use of prescription drugs has increased in the past half century. Although prescription drugs have played an important role in preventing, controlling, and delaying onset or progression of disease, their growth in use also has posed many risks.¹ One ramification of this growth is the occurrence of polypharmacy, which does not have a universal, clear definition. In general, it can be described as the concurrent use of multiple medications by a single patient to treat one or more medical ailments. Five or more medications taken simultaneously is the most common definition to date, but this is just one of many acceptable definitions and that varies from one health care facility to another.^1,2

Regardless of the cutoffs established to indicate polypharmacy, its incidence can result in poor and potentially harmful health outcomes. Polypharmacy increases the risk of experiencing adverse drug events (ADEs), drug-drug interactions (DDIs), geriatric-related syndromes, falls, hospitalization, and mortality. Issues with adherence may begin to unfold secondary to increased pill burden. Both the patient and the health care system may encounter financial strain, as polypharmacy can lead to unnecessary and essentially preventable costs of care. When evaluating the likelihood of polypharmacy based on age group, NCHS found that 47.5% of patients taking ≥ 5 medications were aged ≥ 65 years.^1-5 This indicates that polypharmacy is of great concern in the geriatric population, which also represents a large proportion of individuals accessing Veterans Health Administration (VHA) care.

Deprescibing

Deprescribing is the act of withdrawing or discontinuing potentially inappropriate medications (PIM), or medications used by older patients harboring ADEs that generally outweigh the clinical benefits of the drug. Deprescribing is an effective tool for managing or reducing polypharmacy. A variety of tools have been created whose sole purpose is to simplify deprescribing. Some tools explicitly identify PIM and are widely familiar in medical practice. Examples are the Beers Criteria developed in 1991 or Screening Tool to Alert Right Treatment/Screening Tool of Older Persons Prescriptions (START/STOPP) criteria created in 2003. Other tools that are less commonplace but equally as resourceful are MedStopper and Deprescribing.org. The former was launched in 2015 and is a Canadian online system that provides risk assessments for medications with guidance for tapering or stopping medications if continuation of the drug presents higher risk than benefit.^5-7 The latter is a full-fledged website developed by a physician, a pharmacist, and their research teams that serves as an exchange hub for deprescribing information.

In 2016, the VIONE (Vital, Important, Optional, Not indicated/treatment complete, and Every medication has an indication) deprescribing tool was developed by Saraswathy Battar, MD, at Central Arkansas Veterans Healthcare System (CAVHS) in Little Rock, as a system that could go beyond medication reconciliation (Table 1). Health care providers (HCPs) and pharmacists evaluate each medication that a patient has been prescribed and places each medication in a VIONE category. Prescribers may then take the opportunity to deprescribe or discontinue medications if deemed appropriate based on their clinical assessments and shared decision making.⁸ Traditionally, medication reconciliation involves the process of obtaining a complete and accurate list of medications as reported by a patient or caregiver to a HCP. VIONE encourages HCPs and pharmacists not only to ensure medication lists are accurate, but also that each medication reported is appropriate for continued use. In other words, VIONE is meant to help implement deprescribing at opportune times. More than 14,000 medications have been deprescribed using the VIONE method, resulting in more than $2,000,000 of annualized cost avoidance after just 1 year of implementation at CAVHS.⁹

Methods

Veterans receiving care at VASNHS aged ≥ 65 years with ≥ 10 active medications noted in CPRS were included in this project. PACT pharmacists and physicians were educated on the proper use of the VIONE tool prior to its implementation. Education included a 15-minute slide presentation followed by dissemination of a 1-page VIONE tool handout during a PACT all-staff clinic meeting.

Data were collected for 3 months before and after the intervention. Data were made available for assessment by the Automated Data Processing Application Coordinator (ADPAC) at VASNHS. The ADPAC created and generated an Excel spreadsheet report, which listed all medications deprescribed using the VIONE method. The primary endpoint was the total number of medications discontinued using the VIONE template and/or discontinuation menu. For the purpose of this project, appropriate discontinuation was considered any prescription deprescribed, excluding medical supplies, by pharmacists and PCPs who received VIONE education.

Results

A total of 2,442 veterans met inclusion criteria, and the VIONE method was applied to 598 between late October 2018 and January 2019. The 13 PACT pharmacists contacted at least 10 veterans each, thus at least 130 were randomly selected for telephone calls to perform polypharmacy reviews using the VIONE note template. The discontinuation menu was used if a medication qualified to be deprescribed. After 3 months, 1986 prescriptions were deprescribed using VIONE; however, 1060 prescriptions were considered appropriately deprescribed (Table 2). The 13 PACT pharmacists deprescribed 361 medications, and the 29 PACT physicians deprescribed 699 medications. These prescriptions were then separated into medication categories to determine the most common discontinued classes. Vitamins and supplements were the medication class most frequently deprescribed (19.4%), followed by pain medications (15.5%), antimicrobial agents (9.6%), antihypertensive medications (9.2%), and diabetes medications (6.4%) (Table 3). The top 5 medication categories accounted for 60% of all medications appropriately deprescribed.

The estimated annualized cost avoidance for all medications deprescribed in the 3-month project period was $84,030.46. To provide the most appropriate and accurate calculation, medication classes excluded from this figure were acute or short-term prescriptions and antimicrobial agents. Medications prescribed short-term typically are not suitable to continue for an extended period, and antimicrobial agents were excluded since they are normally associated with higher costs, and may overestimate the cost avoidance calculation for the facility.

Discussion

The outcomes for the primary and secondary endpoints of this project illustrate that using VIONE in PACT primary care clinics had a notable impact on polypharmacy and cost avoidance over a short period. This outcome can be attributed to 2 significant effects of using the deprescribing tool. VIONE’s simplicity in application allowed clinicians to incorporate daily use of the tool with minimal effort. Education was all that was required to fully enable clinicians to work together successfully and exercise collaborative practice to promote deprescribing. VIONE also elicited a cascade of favorable effects that improve patient safety and health outcomes. The tool aided in identification of PIM, which helped reduce polypharmacy and medication burden. The risk for DDIs and ADEs may decrease; therefore, the incidence of falls, need for emergency department visits or inpatient care related to polypharmacy may decline. Less complex medication regimens may alleviate issues with adherence and avoid the various consequences of polypharmacy in theory. Simplified regimens can potentially improve disease management and quality of life for patients. Further studies are needed to substantiate deprescribing and its true effect on patient adherence and better health outcomes at this time.¹⁰

Reducing polypharmacy can lead to cost savings. Based on the results of this 3-month study, we expect that VASNHS would save more than $84,000 by reducing polypharmacy among its patients. Those savings can be funneled back into the health care system, and allotted to necessary patient care, prescriptions, and health care facility needs.

Limitations

There are some important limitations to this study. Definitions of polypharmacy may vary from one health care facility to another. The cutoffs for polypharmacy may differ, causing the prevalence of polypharmacy and potential costs savings to vary. Use of VIONE may be inconsistent among users if not previously educated or properly trained. For instance, VIONE selections are listed in the same menu as the standard CPRS discontinuation options, which may lead to discontinuation of medical supplies or laboratory orders instead of prescriptions.

The method of data analysis and project design used in this study may have been subject to error. For example, the list of PCPs may have been inaccurate or outdated, which would result in an over- or underrepresentation of those who contributed to data collection. Furthermore, there is some volatility in calculating the total cost avoidance. For example, medications for chronic conditions that were only taken on an as needed basis may have overestimated savings. Either under- or overestimations could occur when parameters are adjusted on the VIONE discontinuation dashboard without appropriate guidance. With the ability to manually adjust the dashboard parameters, dissimilarities in calculations may follow.

Conclusions

The VIONE tool may be useful in improving patient safety through deprescribing and discontinuing PIM. Decreasing the number of medications being taken concomitantly by a patient and continuing only those that are imperative in their medical treatment is the first step to reducing the incidence of polypharmacy. Consequently, chances of ADEs or DDIs are lessened, especially among older individuals who are considered high risk for experiencing the detrimental effects that may ensue. These effects include geriatric-related syndromes, increased risk of fall, hospital visits or admissions, or death. Use of VIONE easily promotes collaboration among clinicians to evaluate medications eligible for discontinuation more regularly. If this deprescribing tool is continuously used, costs avoided can likely be maximized within VA health care systems.

The results of this project should serve as an incentive to push for better prescribing practices and increase deprescribing efforts. It should provoke the need for change in regimens and the subsequent discontinuation of prescriptions that are not considered vital to continue. Finally, the result of this project should substantiate the positive impact a deprescribing tool can possess to avert the issues commonly associated with polypharmacy.

According to the Centers for Disease Control and Prevention National Center for Health Statistics (NCHS), the use of prescription drugs has increased in the past half century. Although prescription drugs have played an important role in preventing, controlling, and delaying onset or progression of disease, their growth in use also has posed many risks.¹ One ramification of this growth is the occurrence of polypharmacy, which does not have a universal, clear definition. In general, it can be described as the concurrent use of multiple medications by a single patient to treat one or more medical ailments. Five or more medications taken simultaneously is the most common definition to date, but this is just one of many acceptable definitions and that varies from one health care facility to another.^1,2

Regardless of the cutoffs established to indicate polypharmacy, its incidence can result in poor and potentially harmful health outcomes. Polypharmacy increases the risk of experiencing adverse drug events (ADEs), drug-drug interactions (DDIs), geriatric-related syndromes, falls, hospitalization, and mortality. Issues with adherence may begin to unfold secondary to increased pill burden. Both the patient and the health care system may encounter financial strain, as polypharmacy can lead to unnecessary and essentially preventable costs of care. When evaluating the likelihood of polypharmacy based on age group, NCHS found that 47.5% of patients taking ≥ 5 medications were aged ≥ 65 years.^1-5 This indicates that polypharmacy is of great concern in the geriatric population, which also represents a large proportion of individuals accessing Veterans Health Administration (VHA) care.

Deprescibing

Deprescribing is the act of withdrawing or discontinuing potentially inappropriate medications (PIM), or medications used by older patients harboring ADEs that generally outweigh the clinical benefits of the drug. Deprescribing is an effective tool for managing or reducing polypharmacy. A variety of tools have been created whose sole purpose is to simplify deprescribing. Some tools explicitly identify PIM and are widely familiar in medical practice. Examples are the Beers Criteria developed in 1991 or Screening Tool to Alert Right Treatment/Screening Tool of Older Persons Prescriptions (START/STOPP) criteria created in 2003. Other tools that are less commonplace but equally as resourceful are MedStopper and Deprescribing.org. The former was launched in 2015 and is a Canadian online system that provides risk assessments for medications with guidance for tapering or stopping medications if continuation of the drug presents higher risk than benefit.^5-7 The latter is a full-fledged website developed by a physician, a pharmacist, and their research teams that serves as an exchange hub for deprescribing information.

In 2016, the VIONE (Vital, Important, Optional, Not indicated/treatment complete, and Every medication has an indication) deprescribing tool was developed by Saraswathy Battar, MD, at Central Arkansas Veterans Healthcare System (CAVHS) in Little Rock, as a system that could go beyond medication reconciliation (Table 1). Health care providers (HCPs) and pharmacists evaluate each medication that a patient has been prescribed and places each medication in a VIONE category. Prescribers may then take the opportunity to deprescribe or discontinue medications if deemed appropriate based on their clinical assessments and shared decision making.⁸ Traditionally, medication reconciliation involves the process of obtaining a complete and accurate list of medications as reported by a patient or caregiver to a HCP. VIONE encourages HCPs and pharmacists not only to ensure medication lists are accurate, but also that each medication reported is appropriate for continued use. In other words, VIONE is meant to help implement deprescribing at opportune times. More than 14,000 medications have been deprescribed using the VIONE method, resulting in more than $2,000,000 of annualized cost avoidance after just 1 year of implementation at CAVHS.⁹

Methods

Veterans receiving care at VASNHS aged ≥ 65 years with ≥ 10 active medications noted in CPRS were included in this project. PACT pharmacists and physicians were educated on the proper use of the VIONE tool prior to its implementation. Education included a 15-minute slide presentation followed by dissemination of a 1-page VIONE tool handout during a PACT all-staff clinic meeting.

Data were collected for 3 months before and after the intervention. Data were made available for assessment by the Automated Data Processing Application Coordinator (ADPAC) at VASNHS. The ADPAC created and generated an Excel spreadsheet report, which listed all medications deprescribed using the VIONE method. The primary endpoint was the total number of medications discontinued using the VIONE template and/or discontinuation menu. For the purpose of this project, appropriate discontinuation was considered any prescription deprescribed, excluding medical supplies, by pharmacists and PCPs who received VIONE education.

Results

A total of 2,442 veterans met inclusion criteria, and the VIONE method was applied to 598 between late October 2018 and January 2019. The 13 PACT pharmacists contacted at least 10 veterans each, thus at least 130 were randomly selected for telephone calls to perform polypharmacy reviews using the VIONE note template. The discontinuation menu was used if a medication qualified to be deprescribed. After 3 months, 1986 prescriptions were deprescribed using VIONE; however, 1060 prescriptions were considered appropriately deprescribed (Table 2). The 13 PACT pharmacists deprescribed 361 medications, and the 29 PACT physicians deprescribed 699 medications. These prescriptions were then separated into medication categories to determine the most common discontinued classes. Vitamins and supplements were the medication class most frequently deprescribed (19.4%), followed by pain medications (15.5%), antimicrobial agents (9.6%), antihypertensive medications (9.2%), and diabetes medications (6.4%) (Table 3). The top 5 medication categories accounted for 60% of all medications appropriately deprescribed.

The estimated annualized cost avoidance for all medications deprescribed in the 3-month project period was $84,030.46. To provide the most appropriate and accurate calculation, medication classes excluded from this figure were acute or short-term prescriptions and antimicrobial agents. Medications prescribed short-term typically are not suitable to continue for an extended period, and antimicrobial agents were excluded since they are normally associated with higher costs, and may overestimate the cost avoidance calculation for the facility.

Discussion

The outcomes for the primary and secondary endpoints of this project illustrate that using VIONE in PACT primary care clinics had a notable impact on polypharmacy and cost avoidance over a short period. This outcome can be attributed to 2 significant effects of using the deprescribing tool. VIONE’s simplicity in application allowed clinicians to incorporate daily use of the tool with minimal effort. Education was all that was required to fully enable clinicians to work together successfully and exercise collaborative practice to promote deprescribing. VIONE also elicited a cascade of favorable effects that improve patient safety and health outcomes. The tool aided in identification of PIM, which helped reduce polypharmacy and medication burden. The risk for DDIs and ADEs may decrease; therefore, the incidence of falls, need for emergency department visits or inpatient care related to polypharmacy may decline. Less complex medication regimens may alleviate issues with adherence and avoid the various consequences of polypharmacy in theory. Simplified regimens can potentially improve disease management and quality of life for patients. Further studies are needed to substantiate deprescribing and its true effect on patient adherence and better health outcomes at this time.¹⁰

Reducing polypharmacy can lead to cost savings. Based on the results of this 3-month study, we expect that VASNHS would save more than $84,000 by reducing polypharmacy among its patients. Those savings can be funneled back into the health care system, and allotted to necessary patient care, prescriptions, and health care facility needs.

Limitations

There are some important limitations to this study. Definitions of polypharmacy may vary from one health care facility to another. The cutoffs for polypharmacy may differ, causing the prevalence of polypharmacy and potential costs savings to vary. Use of VIONE may be inconsistent among users if not previously educated or properly trained. For instance, VIONE selections are listed in the same menu as the standard CPRS discontinuation options, which may lead to discontinuation of medical supplies or laboratory orders instead of prescriptions.

The method of data analysis and project design used in this study may have been subject to error. For example, the list of PCPs may have been inaccurate or outdated, which would result in an over- or underrepresentation of those who contributed to data collection. Furthermore, there is some volatility in calculating the total cost avoidance. For example, medications for chronic conditions that were only taken on an as needed basis may have overestimated savings. Either under- or overestimations could occur when parameters are adjusted on the VIONE discontinuation dashboard without appropriate guidance. With the ability to manually adjust the dashboard parameters, dissimilarities in calculations may follow.

Conclusions

The VIONE tool may be useful in improving patient safety through deprescribing and discontinuing PIM. Decreasing the number of medications being taken concomitantly by a patient and continuing only those that are imperative in their medical treatment is the first step to reducing the incidence of polypharmacy. Consequently, chances of ADEs or DDIs are lessened, especially among older individuals who are considered high risk for experiencing the detrimental effects that may ensue. These effects include geriatric-related syndromes, increased risk of fall, hospital visits or admissions, or death. Use of VIONE easily promotes collaboration among clinicians to evaluate medications eligible for discontinuation more regularly. If this deprescribing tool is continuously used, costs avoided can likely be maximized within VA health care systems.

The results of this project should serve as an incentive to push for better prescribing practices and increase deprescribing efforts. It should provoke the need for change in regimens and the subsequent discontinuation of prescriptions that are not considered vital to continue. Finally, the result of this project should substantiate the positive impact a deprescribing tool can possess to avert the issues commonly associated with polypharmacy.

Acne rosacea is a chronic inflammatory disease that may affect the facial skin, eyes, and eyelids.¹ It is characterized by transient or persistent flushing, facial erythema, and telangiectases, generally located on the central portion of the face, and may progress to papules and pustules.^2,3 At the late stage of the disease, dermal edema or fibroplasia and sebaceous gland hypertrophy may cause phymatous alterations in the skin. In 2004, the National Rosacea Society Expert Committee developed a classification system for rosacea to standardize subtypes and variants that has since been widely accepted and continues to aid in research and epidemiologic studies.⁴ The committee defined 4 subtypes based on clinical characteristics: erythematotelangiectatic (ETR), papulopustular (PPR), phymatous, and ocular rosacea.^2,3

Ocular rosacea may accompany mild, moderate, and severe dermatologic disease or may occur in the absence of diagnostic skin disease.⁵ Ocular signs include eyelid margin telangiectasia, spade-shaped infiltrates in the cornea, scleritis, and sclerokeratitis. Common symptoms include burning, stinging, light sensitivity, and foreign-body sensation. Ocular signs commonly seen in rosacea are meibomian gland dysfunction characterized by inspissation and inflammation of the meibomian glands (chalazia), conjunctivitis, honey crust and cylindrical collarette accumulation at the base of the eyelashes, irregularity of the eyelid margin architecture, and evaporative tear dysfunction.^5,6

The physiopathology of rosacea is still unknown. Potential factors include genetic predisposition, abnormal inflammation, vascular dysfunction, and involvement of several microbial agents, such as commensal Demodex mites. The number of Demodex mites on normal skin flora is less than 5/cm²; however, the increased vascular dilation and capillary permeability associated with rosacea that result from sunlight and heat exposure increase the density of Demodex folliculorum.⁷ Elevated Demodex mite density has been observed in the lumens of the sebaceous follicles in patients with rosacea. However, because the severity of the clinical manifestations of the disease is not directly associated with the density of D folliculorum, it generally is accepted that D folliculorum is not a pathogenetic but rather an exacerbating factor.⁸ It has been reported that this species of mite is mostly found on the face and around the eyelashes and scalp of patients and that it can cause ocular surface inflammation.⁸

Most studies have researched ocular manifestations of rosacea but not ocular involvement in rosacea patients with and without Demodex mite infestation. In our study, we sought to compare the ocular surface, meibomian gland characteristics, and tear film abnormalities among patients with cutaneous rosacea with and without Demodex infestation.

Materials and Methods

We conducted a retrospective study of 60 patients with cutaneous rosacea. This study was approved by the ethics committee of the local hospital (2018/002-003), and all patients provided verbal and written informed consent before participating in the study. The study was carried out according to the guidelines of the Declaration of Helsinki.

Patient Selection and Evaluation
Patients diagnosed with rosacea by a dermatologist within 6 months were included in the study. Diagnosis of the disease was made after a detailed anamnesis and dermatologic examination. Rosacea was diagnosed if patients had an itching sensation, erythema and/or erythema attacks, and papules and pustules, and fulfilled the diagnostic criteria according to the National Rosacea Society. The skin disease was classified according to the subtypes as ETR, PPR, phymatous rosacea, or ocular rosacea.

The standard skin surface biopsy method was used in 60 patients for detecting Demodex density. When more than 5 mites were detected per square centimeter, the result was recorded as positive. Thirty consecutive, newly diagnosed patients with cutaneous acne rosacea with Demodex infestation and 30 consecutive, newly diagnosed sex- and age-matched patients with acne rosacea without Demodex infestation admitted to the dermatology outpatient clinic were included to this study. The patients who did not have any known dermatologic, systemic, or ocular diseases were included in the study. Patients who met any of the following criteria were excluded from the study: prior anti-inflammatory topical and/or systemic treatment for rosacea during the last 3 months, contact lens wear, eyelid surgery, or autoimmune disease requiring treatment.

Microscopic Demodex Examination
Demodex count was determined using a standardized skin surface biopsy, which is a noninvasive method. Every patient gave samples from the cheeks. This biopsy was repeated from the same site. A drop of cyanoacrylate was placed on a clean slide, pressed against a skin lesion, held in place for 1 minute, and removed. The obtained samples were evaluated under a light microscope (Nikon E200) with oil immersion. When more than 5 mites were detected per square centimeter, the result was recorded as positive.

Ophthalmologic Examination
A complete ophthalmologic examination including visual acuity assessment, standardized slit lamp examination, and fundus examination was done for all patients. Ocular rosacea was diagnosed on detection of 1 or more of the following: watery or bloodshot appearance, foreign-body sensation, burning or stinging, dryness, itching, light sensitivity, blurred vision, telangiectases of the conjunctiva and eyelid margin, eyelid lid and periocular erythema, anterior blepharitis, meibomian gland dysfunction, or irregularity of eyelid margins. All patients were screened for the signs and symptoms of ocular rosacea and underwent other ophthalmologic examinations, including tear function tests. Tear functions were evaluated with Schirmer tests without anesthesia and fluorescein tear breakup time (TBUT). Tear film breakup time was assessed after instillation of 2% fluorescein staining under a cobalt blue filter. The time interval between the last complete blink and the appearance of the first dry spot was recorded. The mean of 3 consecutive measurements was obtained. The Schirmer test was performed without topical anesthesia using a standardized filter strip (Bio-Tech Vision Care). The amount of wetting was measured after 5 minutes. Meibomian gland expressibility was assessed by applying digital pressure to the eyelid margin.

Statistical Analysis
Statistical analysis of the study was performed with SPSS Statistics Version 22.0 (SPSS Inc). Continuous variables were reported as mean (SD), and categorical variables were reported as percentages and counts. Descriptive statistics for numerical variables were created. An independent sample t test was used for normally distributed continuous variables. The Kolmogorov-Smirnov test was used to determine normality. The Schirmer test without anesthesia and TBUT values among groups were compared using one-way analysis of variance. The differences were calculated using the multiple comparison Tukey test. P<.05 was considered statistically significant.

Results

Demographic Characteristics of Rosacea Patients
Sixty eyes of 30 newly diagnosed patients with acne rosacea with Demodex infestation and 60 eyes of 30 newly diagnosed patients with acne rosacea without Demodex infestation were enrolled in this study. The mean age (SD) of the 60 patients was 37.63 (10.01) years. The mean TBUT (SD) of the 120 eyes was 6.65 (3.44) seconds, and the mean Schirmer score (SD) was 12.59 (6.71) mm (Table 1).

Meibomian Gland Dysfunction vs Subgroup of Rosacea Patients
Thirty-four (57%) patients had blepharitis, and 18 (30%) patients had meibomitis. Thirty-five (58.3%) patients had ETR, 5 (8.3%) patients had phymatous rosacea, and 20 (33.4%) patients had PPR (Table 2). Of the Demodex-negative patients, 73.3% (22/30) had ETR, 20% (6/30) had PPR, and 6.7% (2/30) had phymatous rosacea. Of the Demodex-positive patients, 43.3% (13/30) had ETR, 46.7% (14/30) had PPR, and 10% (3/30) had phymatous rosacea (Table 3). Papulopustular rosacea was found to be significantly associated with Demodex positivity (P=.003); neither ETR nor phymatous rosacea was found to be significantly associated with Demodex infestation (P=.66 and P=.13, respectively)(Table 3).

Sixteen (53.3%) patients (32 eyes) in the Demodex-negative group and 21 (70%) patients (42 eyes) in the Demodex-positive group had TBUT values less than 10 seconds. Eighteen (60%) patients (36 eyes) in the Demodex-negative group and 25 (83.3%) patients (50 eyes) in the Demodex-positive group had Schirmer scores less than 10 mm (Table 3). The 2 groups were not significantly different in dry eye findings (P=.25 and P=.29, respectively).

Comment

Inflammation in Rosacea
It is known that the density of nonfloral bacteria as well as D folliculorum and Demodex brevis increases in skin affected by rosacea compared to normal skin. Vascular dilation associated with rosacea that results from sunlight and heat causes increased capillary permeability and creates the ideal environment for the proliferation of D folliculorum. Demodex is thought to act as a vector for the activity of certain other microorganisms, particularly Bacillus oleronius, and thus initiates the inflammatory response associated with rosacea.⁹

One study reported that the inflammation associated with rosacea that was caused by Demodex and other environmental stimuli occurred through toll-like receptor 2 and various cytokines.¹⁰ It has been reported that the abnormal function of toll-like receptor 2 in the epidermis leads to the increased production of cathelicidin. Cathelicidin is an antimicrobial peptide with both vasoactive and proinflammatory activity and has been used as a basis to explain the pathogenesis of facial erythema, flushing, and telangiectasia in the context of rosacea.^11,12 In addition, it has been reported that the increased secretion of proinflammatory cytokines such as IL-1 and gelatinase B in ocular rosacea leads to tearing film abnormalities that result from increased bacterial flora in the eyelids, which subsequently leads to decreased tear drainage and dry eyes.¹³ In addition, B oleronius isolated from a D folliculorum mite from patients with PPR produced proteins that induced an inflammatory immune response in 73% (16/22) of patients with rosacea.¹⁴

Ocular Findings in Rosacea Patients
In our study, PPR was found to be significantly associated with Demodex positivity compared to ETR and phymatous rosacea (P=.003). However, ocular inflammation findings such as blepharitis and meibomitis were not significantly different between Demodex-positive and Demodex-negative patients. Although the mean Schirmer score of Demodex-positive patients was lower than Demodex-negative patients, this difference was not statistically significant. We evaluated a TBUT of less than 10 seconds and a Schirmer score less than 10 mm as dry eye. Accordingly, the number of patients with dry eye was higher in the Demodex-positive group, but this difference was not statistically significant.

Chronic blepharitis, conjunctival inflammation, and meibomian gland dysfunction are among the most common findings of ocular rosacea.^15,16 Patients with ocular rosacea commonly have dry eye and abnormal TBUT and Schirmer scores.¹⁷ In our study, we found that the fluorescein TBUT and Schirmer scores were more likely to be abnormal in the Demodex-positive group, but the difference between the 2 groups was not statistically significant.

It has been reported that proinflammatory cytokines due to a weakened immune system in rosacea patients were increased. The weakened immune system was further supported by the increased concentrations of proinflammatory cytokines such as IL-1 and matrix metalloproteinase 9 in these patients’ tears and the improvement of symptoms after the inhibition of these cytokines.¹¹ Luo et al¹⁸ reported that Demodex inflammation causes dry eye, particularly with D brevis. Ayyildiz and Sezgin¹⁹ reported that Schirmer scores were significantly lower and that the Ocular Surface Disease Index had significantly increased in the Demodex-positive group compared to the Demodex-negative group (P=.001 for both). A Korean study reported that Demodex density was correlated with age, sex, and TBUT results, but there was no significant relationship between Demodex density and Schirmer scores.¹⁶

Sobolewska et al²⁰ administered ivermectin cream 1% to 10 patients with cutaneous and ocular rosacea, but only to the forehead, chin, nose, cheeks, and regions close to the eyelids, and observed a significant improvement in blepharitis (P=.004). They stated that ivermectin, as applied only to the face, suppressed the proinflammatory cytokines associated with rosacea and showed anti-inflammatory effects by reducing Demodex mites.²⁰Li et al²¹ demonstrated a strong correlation between ocular Demodex inflammation and serum reactivity to these bacterial proteins in patients with ocular rosacea, and they found that eyelid margin inflammation and facial rosacea correlated with reactivity to these proteins. These studies suggest a possible role for Demodex infestation and bacterial proteins in the etiology of rosacea.

Gonzalez-Hinojosa et al²² demonstrated that even though eyelash blepharitis was more common in PPR than ETR, there was no statistically significant association between rosacea and Demodex blepharitis. In our study, we found a significant correlation between PPR and Demodex positivity. Also, meibomian gland dysfunction was more common in the Demodex-positive group; however, this result was not statistically significant. One study compared patients with primary demodicosis and patients with rosacea with Demodex-induced blepharitis to healthy controls and found that patients with primary demodicosis and patients with rosacea did not have significantly different ocular findings.²³ In contrast, Forton and De Maertelaer²⁴ reported that patients with PPR had significantly more severe ocular manifestations compared with patients with demodicosis (P=.004).

Mizuno et al²⁵ compared the normal (nonrosacea) population with and without Demodex-infested eyelashes and found that the 2 groups were not significantly different for meibomian gland dysfunction, fluorescein TBUT, or ocular surface discomfort.

Varying results have been reported regarding the association between Demodex and blepharitis or ocular surface discomfort with or without rosacea. In our study, we found that Demodex did not affect tear function tests or meibomian gland function in patients with rosacea. We believe this study is important because it demonstrates the effects of Demodex on ocular findings in patients with cutaneous rosacea.

Limitations
Our study has some limitations. The number of patients was relatively small, resulting in few significant differences between the comparison groups. A larger prospective research study is required to assess the prevalence of Demodex mites in the ocular rosacea population along with associated symptoms and findings.

Conclusion

Rosacea is a chronic disease associated with skin and ocular manifestations that range from mild to severe, that progresses in the form of attacks, and that requires long-term follow-up and treatment. Rosacea most often presents as a disease that causes ocular surface inflammation of varying degrees. Demodex infestation may increase cutaneous or ocular inflammation in rosacea. Therefore, every patient diagnosed with rosacea should be given a dermatologic examination to determine Demodex positivity and an ophthalmologic examination to determine ocular manifestations.

Acne rosacea is a chronic inflammatory disease that may affect the facial skin, eyes, and eyelids.¹ It is characterized by transient or persistent flushing, facial erythema, and telangiectases, generally located on the central portion of the face, and may progress to papules and pustules.^2,3 At the late stage of the disease, dermal edema or fibroplasia and sebaceous gland hypertrophy may cause phymatous alterations in the skin. In 2004, the National Rosacea Society Expert Committee developed a classification system for rosacea to standardize subtypes and variants that has since been widely accepted and continues to aid in research and epidemiologic studies.⁴ The committee defined 4 subtypes based on clinical characteristics: erythematotelangiectatic (ETR), papulopustular (PPR), phymatous, and ocular rosacea.^2,3

Ocular rosacea may accompany mild, moderate, and severe dermatologic disease or may occur in the absence of diagnostic skin disease.⁵ Ocular signs include eyelid margin telangiectasia, spade-shaped infiltrates in the cornea, scleritis, and sclerokeratitis. Common symptoms include burning, stinging, light sensitivity, and foreign-body sensation. Ocular signs commonly seen in rosacea are meibomian gland dysfunction characterized by inspissation and inflammation of the meibomian glands (chalazia), conjunctivitis, honey crust and cylindrical collarette accumulation at the base of the eyelashes, irregularity of the eyelid margin architecture, and evaporative tear dysfunction.^5,6

The physiopathology of rosacea is still unknown. Potential factors include genetic predisposition, abnormal inflammation, vascular dysfunction, and involvement of several microbial agents, such as commensal Demodex mites. The number of Demodex mites on normal skin flora is less than 5/cm²; however, the increased vascular dilation and capillary permeability associated with rosacea that result from sunlight and heat exposure increase the density of Demodex folliculorum.⁷ Elevated Demodex mite density has been observed in the lumens of the sebaceous follicles in patients with rosacea. However, because the severity of the clinical manifestations of the disease is not directly associated with the density of D folliculorum, it generally is accepted that D folliculorum is not a pathogenetic but rather an exacerbating factor.⁸ It has been reported that this species of mite is mostly found on the face and around the eyelashes and scalp of patients and that it can cause ocular surface inflammation.⁸

Most studies have researched ocular manifestations of rosacea but not ocular involvement in rosacea patients with and without Demodex mite infestation. In our study, we sought to compare the ocular surface, meibomian gland characteristics, and tear film abnormalities among patients with cutaneous rosacea with and without Demodex infestation.

Materials and Methods

We conducted a retrospective study of 60 patients with cutaneous rosacea. This study was approved by the ethics committee of the local hospital (2018/002-003), and all patients provided verbal and written informed consent before participating in the study. The study was carried out according to the guidelines of the Declaration of Helsinki.

Patient Selection and Evaluation
Patients diagnosed with rosacea by a dermatologist within 6 months were included in the study. Diagnosis of the disease was made after a detailed anamnesis and dermatologic examination. Rosacea was diagnosed if patients had an itching sensation, erythema and/or erythema attacks, and papules and pustules, and fulfilled the diagnostic criteria according to the National Rosacea Society. The skin disease was classified according to the subtypes as ETR, PPR, phymatous rosacea, or ocular rosacea.

The standard skin surface biopsy method was used in 60 patients for detecting Demodex density. When more than 5 mites were detected per square centimeter, the result was recorded as positive. Thirty consecutive, newly diagnosed patients with cutaneous acne rosacea with Demodex infestation and 30 consecutive, newly diagnosed sex- and age-matched patients with acne rosacea without Demodex infestation admitted to the dermatology outpatient clinic were included to this study. The patients who did not have any known dermatologic, systemic, or ocular diseases were included in the study. Patients who met any of the following criteria were excluded from the study: prior anti-inflammatory topical and/or systemic treatment for rosacea during the last 3 months, contact lens wear, eyelid surgery, or autoimmune disease requiring treatment.

Microscopic Demodex Examination
Demodex count was determined using a standardized skin surface biopsy, which is a noninvasive method. Every patient gave samples from the cheeks. This biopsy was repeated from the same site. A drop of cyanoacrylate was placed on a clean slide, pressed against a skin lesion, held in place for 1 minute, and removed. The obtained samples were evaluated under a light microscope (Nikon E200) with oil immersion. When more than 5 mites were detected per square centimeter, the result was recorded as positive.

Ophthalmologic Examination
A complete ophthalmologic examination including visual acuity assessment, standardized slit lamp examination, and fundus examination was done for all patients. Ocular rosacea was diagnosed on detection of 1 or more of the following: watery or bloodshot appearance, foreign-body sensation, burning or stinging, dryness, itching, light sensitivity, blurred vision, telangiectases of the conjunctiva and eyelid margin, eyelid lid and periocular erythema, anterior blepharitis, meibomian gland dysfunction, or irregularity of eyelid margins. All patients were screened for the signs and symptoms of ocular rosacea and underwent other ophthalmologic examinations, including tear function tests. Tear functions were evaluated with Schirmer tests without anesthesia and fluorescein tear breakup time (TBUT). Tear film breakup time was assessed after instillation of 2% fluorescein staining under a cobalt blue filter. The time interval between the last complete blink and the appearance of the first dry spot was recorded. The mean of 3 consecutive measurements was obtained. The Schirmer test was performed without topical anesthesia using a standardized filter strip (Bio-Tech Vision Care). The amount of wetting was measured after 5 minutes. Meibomian gland expressibility was assessed by applying digital pressure to the eyelid margin.

Statistical Analysis
Statistical analysis of the study was performed with SPSS Statistics Version 22.0 (SPSS Inc). Continuous variables were reported as mean (SD), and categorical variables were reported as percentages and counts. Descriptive statistics for numerical variables were created. An independent sample t test was used for normally distributed continuous variables. The Kolmogorov-Smirnov test was used to determine normality. The Schirmer test without anesthesia and TBUT values among groups were compared using one-way analysis of variance. The differences were calculated using the multiple comparison Tukey test. P<.05 was considered statistically significant.

Results

Demographic Characteristics of Rosacea Patients
Sixty eyes of 30 newly diagnosed patients with acne rosacea with Demodex infestation and 60 eyes of 30 newly diagnosed patients with acne rosacea without Demodex infestation were enrolled in this study. The mean age (SD) of the 60 patients was 37.63 (10.01) years. The mean TBUT (SD) of the 120 eyes was 6.65 (3.44) seconds, and the mean Schirmer score (SD) was 12.59 (6.71) mm (Table 1).

Meibomian Gland Dysfunction vs Subgroup of Rosacea Patients
Thirty-four (57%) patients had blepharitis, and 18 (30%) patients had meibomitis. Thirty-five (58.3%) patients had ETR, 5 (8.3%) patients had phymatous rosacea, and 20 (33.4%) patients had PPR (Table 2). Of the Demodex-negative patients, 73.3% (22/30) had ETR, 20% (6/30) had PPR, and 6.7% (2/30) had phymatous rosacea. Of the Demodex-positive patients, 43.3% (13/30) had ETR, 46.7% (14/30) had PPR, and 10% (3/30) had phymatous rosacea (Table 3). Papulopustular rosacea was found to be significantly associated with Demodex positivity (P=.003); neither ETR nor phymatous rosacea was found to be significantly associated with Demodex infestation (P=.66 and P=.13, respectively)(Table 3).

Sixteen (53.3%) patients (32 eyes) in the Demodex-negative group and 21 (70%) patients (42 eyes) in the Demodex-positive group had TBUT values less than 10 seconds. Eighteen (60%) patients (36 eyes) in the Demodex-negative group and 25 (83.3%) patients (50 eyes) in the Demodex-positive group had Schirmer scores less than 10 mm (Table 3). The 2 groups were not significantly different in dry eye findings (P=.25 and P=.29, respectively).

Comment

Inflammation in Rosacea
It is known that the density of nonfloral bacteria as well as D folliculorum and Demodex brevis increases in skin affected by rosacea compared to normal skin. Vascular dilation associated with rosacea that results from sunlight and heat causes increased capillary permeability and creates the ideal environment for the proliferation of D folliculorum. Demodex is thought to act as a vector for the activity of certain other microorganisms, particularly Bacillus oleronius, and thus initiates the inflammatory response associated with rosacea.⁹

One study reported that the inflammation associated with rosacea that was caused by Demodex and other environmental stimuli occurred through toll-like receptor 2 and various cytokines.¹⁰ It has been reported that the abnormal function of toll-like receptor 2 in the epidermis leads to the increased production of cathelicidin. Cathelicidin is an antimicrobial peptide with both vasoactive and proinflammatory activity and has been used as a basis to explain the pathogenesis of facial erythema, flushing, and telangiectasia in the context of rosacea.^11,12 In addition, it has been reported that the increased secretion of proinflammatory cytokines such as IL-1 and gelatinase B in ocular rosacea leads to tearing film abnormalities that result from increased bacterial flora in the eyelids, which subsequently leads to decreased tear drainage and dry eyes.¹³ In addition, B oleronius isolated from a D folliculorum mite from patients with PPR produced proteins that induced an inflammatory immune response in 73% (16/22) of patients with rosacea.¹⁴

Ocular Findings in Rosacea Patients
In our study, PPR was found to be significantly associated with Demodex positivity compared to ETR and phymatous rosacea (P=.003). However, ocular inflammation findings such as blepharitis and meibomitis were not significantly different between Demodex-positive and Demodex-negative patients. Although the mean Schirmer score of Demodex-positive patients was lower than Demodex-negative patients, this difference was not statistically significant. We evaluated a TBUT of less than 10 seconds and a Schirmer score less than 10 mm as dry eye. Accordingly, the number of patients with dry eye was higher in the Demodex-positive group, but this difference was not statistically significant.

Chronic blepharitis, conjunctival inflammation, and meibomian gland dysfunction are among the most common findings of ocular rosacea.^15,16 Patients with ocular rosacea commonly have dry eye and abnormal TBUT and Schirmer scores.¹⁷ In our study, we found that the fluorescein TBUT and Schirmer scores were more likely to be abnormal in the Demodex-positive group, but the difference between the 2 groups was not statistically significant.

It has been reported that proinflammatory cytokines due to a weakened immune system in rosacea patients were increased. The weakened immune system was further supported by the increased concentrations of proinflammatory cytokines such as IL-1 and matrix metalloproteinase 9 in these patients’ tears and the improvement of symptoms after the inhibition of these cytokines.¹¹ Luo et al¹⁸ reported that Demodex inflammation causes dry eye, particularly with D brevis. Ayyildiz and Sezgin¹⁹ reported that Schirmer scores were significantly lower and that the Ocular Surface Disease Index had significantly increased in the Demodex-positive group compared to the Demodex-negative group (P=.001 for both). A Korean study reported that Demodex density was correlated with age, sex, and TBUT results, but there was no significant relationship between Demodex density and Schirmer scores.¹⁶

Sobolewska et al²⁰ administered ivermectin cream 1% to 10 patients with cutaneous and ocular rosacea, but only to the forehead, chin, nose, cheeks, and regions close to the eyelids, and observed a significant improvement in blepharitis (P=.004). They stated that ivermectin, as applied only to the face, suppressed the proinflammatory cytokines associated with rosacea and showed anti-inflammatory effects by reducing Demodex mites.²⁰Li et al²¹ demonstrated a strong correlation between ocular Demodex inflammation and serum reactivity to these bacterial proteins in patients with ocular rosacea, and they found that eyelid margin inflammation and facial rosacea correlated with reactivity to these proteins. These studies suggest a possible role for Demodex infestation and bacterial proteins in the etiology of rosacea.

Gonzalez-Hinojosa et al²² demonstrated that even though eyelash blepharitis was more common in PPR than ETR, there was no statistically significant association between rosacea and Demodex blepharitis. In our study, we found a significant correlation between PPR and Demodex positivity. Also, meibomian gland dysfunction was more common in the Demodex-positive group; however, this result was not statistically significant. One study compared patients with primary demodicosis and patients with rosacea with Demodex-induced blepharitis to healthy controls and found that patients with primary demodicosis and patients with rosacea did not have significantly different ocular findings.²³ In contrast, Forton and De Maertelaer²⁴ reported that patients with PPR had significantly more severe ocular manifestations compared with patients with demodicosis (P=.004).

Mizuno et al²⁵ compared the normal (nonrosacea) population with and without Demodex-infested eyelashes and found that the 2 groups were not significantly different for meibomian gland dysfunction, fluorescein TBUT, or ocular surface discomfort.

Varying results have been reported regarding the association between Demodex and blepharitis or ocular surface discomfort with or without rosacea. In our study, we found that Demodex did not affect tear function tests or meibomian gland function in patients with rosacea. We believe this study is important because it demonstrates the effects of Demodex on ocular findings in patients with cutaneous rosacea.

Limitations
Our study has some limitations. The number of patients was relatively small, resulting in few significant differences between the comparison groups. A larger prospective research study is required to assess the prevalence of Demodex mites in the ocular rosacea population along with associated symptoms and findings.

Conclusion

Rosacea is a chronic disease associated with skin and ocular manifestations that range from mild to severe, that progresses in the form of attacks, and that requires long-term follow-up and treatment. Rosacea most often presents as a disease that causes ocular surface inflammation of varying degrees. Demodex infestation may increase cutaneous or ocular inflammation in rosacea. Therefore, every patient diagnosed with rosacea should be given a dermatologic examination to determine Demodex positivity and an ophthalmologic examination to determine ocular manifestations.

Acne rosacea is a chronic inflammatory disease that may affect the facial skin, eyes, and eyelids.¹ It is characterized by transient or persistent flushing, facial erythema, and telangiectases, generally located on the central portion of the face, and may progress to papules and pustules.^2,3 At the late stage of the disease, dermal edema or fibroplasia and sebaceous gland hypertrophy may cause phymatous alterations in the skin. In 2004, the National Rosacea Society Expert Committee developed a classification system for rosacea to standardize subtypes and variants that has since been widely accepted and continues to aid in research and epidemiologic studies.⁴ The committee defined 4 subtypes based on clinical characteristics: erythematotelangiectatic (ETR), papulopustular (PPR), phymatous, and ocular rosacea.^2,3

Ocular rosacea may accompany mild, moderate, and severe dermatologic disease or may occur in the absence of diagnostic skin disease.⁵ Ocular signs include eyelid margin telangiectasia, spade-shaped infiltrates in the cornea, scleritis, and sclerokeratitis. Common symptoms include burning, stinging, light sensitivity, and foreign-body sensation. Ocular signs commonly seen in rosacea are meibomian gland dysfunction characterized by inspissation and inflammation of the meibomian glands (chalazia), conjunctivitis, honey crust and cylindrical collarette accumulation at the base of the eyelashes, irregularity of the eyelid margin architecture, and evaporative tear dysfunction.^5,6

The physiopathology of rosacea is still unknown. Potential factors include genetic predisposition, abnormal inflammation, vascular dysfunction, and involvement of several microbial agents, such as commensal Demodex mites. The number of Demodex mites on normal skin flora is less than 5/cm²; however, the increased vascular dilation and capillary permeability associated with rosacea that result from sunlight and heat exposure increase the density of Demodex folliculorum.⁷ Elevated Demodex mite density has been observed in the lumens of the sebaceous follicles in patients with rosacea. However, because the severity of the clinical manifestations of the disease is not directly associated with the density of D folliculorum, it generally is accepted that D folliculorum is not a pathogenetic but rather an exacerbating factor.⁸ It has been reported that this species of mite is mostly found on the face and around the eyelashes and scalp of patients and that it can cause ocular surface inflammation.⁸

Most studies have researched ocular manifestations of rosacea but not ocular involvement in rosacea patients with and without Demodex mite infestation. In our study, we sought to compare the ocular surface, meibomian gland characteristics, and tear film abnormalities among patients with cutaneous rosacea with and without Demodex infestation.

Materials and Methods

We conducted a retrospective study of 60 patients with cutaneous rosacea. This study was approved by the ethics committee of the local hospital (2018/002-003), and all patients provided verbal and written informed consent before participating in the study. The study was carried out according to the guidelines of the Declaration of Helsinki.

Patient Selection and Evaluation
Patients diagnosed with rosacea by a dermatologist within 6 months were included in the study. Diagnosis of the disease was made after a detailed anamnesis and dermatologic examination. Rosacea was diagnosed if patients had an itching sensation, erythema and/or erythema attacks, and papules and pustules, and fulfilled the diagnostic criteria according to the National Rosacea Society. The skin disease was classified according to the subtypes as ETR, PPR, phymatous rosacea, or ocular rosacea.

The standard skin surface biopsy method was used in 60 patients for detecting Demodex density. When more than 5 mites were detected per square centimeter, the result was recorded as positive. Thirty consecutive, newly diagnosed patients with cutaneous acne rosacea with Demodex infestation and 30 consecutive, newly diagnosed sex- and age-matched patients with acne rosacea without Demodex infestation admitted to the dermatology outpatient clinic were included to this study. The patients who did not have any known dermatologic, systemic, or ocular diseases were included in the study. Patients who met any of the following criteria were excluded from the study: prior anti-inflammatory topical and/or systemic treatment for rosacea during the last 3 months, contact lens wear, eyelid surgery, or autoimmune disease requiring treatment.

Microscopic Demodex Examination
Demodex count was determined using a standardized skin surface biopsy, which is a noninvasive method. Every patient gave samples from the cheeks. This biopsy was repeated from the same site. A drop of cyanoacrylate was placed on a clean slide, pressed against a skin lesion, held in place for 1 minute, and removed. The obtained samples were evaluated under a light microscope (Nikon E200) with oil immersion. When more than 5 mites were detected per square centimeter, the result was recorded as positive.

Ophthalmologic Examination
A complete ophthalmologic examination including visual acuity assessment, standardized slit lamp examination, and fundus examination was done for all patients. Ocular rosacea was diagnosed on detection of 1 or more of the following: watery or bloodshot appearance, foreign-body sensation, burning or stinging, dryness, itching, light sensitivity, blurred vision, telangiectases of the conjunctiva and eyelid margin, eyelid lid and periocular erythema, anterior blepharitis, meibomian gland dysfunction, or irregularity of eyelid margins. All patients were screened for the signs and symptoms of ocular rosacea and underwent other ophthalmologic examinations, including tear function tests. Tear functions were evaluated with Schirmer tests without anesthesia and fluorescein tear breakup time (TBUT). Tear film breakup time was assessed after instillation of 2% fluorescein staining under a cobalt blue filter. The time interval between the last complete blink and the appearance of the first dry spot was recorded. The mean of 3 consecutive measurements was obtained. The Schirmer test was performed without topical anesthesia using a standardized filter strip (Bio-Tech Vision Care). The amount of wetting was measured after 5 minutes. Meibomian gland expressibility was assessed by applying digital pressure to the eyelid margin.

Statistical Analysis
Statistical analysis of the study was performed with SPSS Statistics Version 22.0 (SPSS Inc). Continuous variables were reported as mean (SD), and categorical variables were reported as percentages and counts. Descriptive statistics for numerical variables were created. An independent sample t test was used for normally distributed continuous variables. The Kolmogorov-Smirnov test was used to determine normality. The Schirmer test without anesthesia and TBUT values among groups were compared using one-way analysis of variance. The differences were calculated using the multiple comparison Tukey test. P<.05 was considered statistically significant.

Results

Demographic Characteristics of Rosacea Patients
Sixty eyes of 30 newly diagnosed patients with acne rosacea with Demodex infestation and 60 eyes of 30 newly diagnosed patients with acne rosacea without Demodex infestation were enrolled in this study. The mean age (SD) of the 60 patients was 37.63 (10.01) years. The mean TBUT (SD) of the 120 eyes was 6.65 (3.44) seconds, and the mean Schirmer score (SD) was 12.59 (6.71) mm (Table 1).

Meibomian Gland Dysfunction vs Subgroup of Rosacea Patients
Thirty-four (57%) patients had blepharitis, and 18 (30%) patients had meibomitis. Thirty-five (58.3%) patients had ETR, 5 (8.3%) patients had phymatous rosacea, and 20 (33.4%) patients had PPR (Table 2). Of the Demodex-negative patients, 73.3% (22/30) had ETR, 20% (6/30) had PPR, and 6.7% (2/30) had phymatous rosacea. Of the Demodex-positive patients, 43.3% (13/30) had ETR, 46.7% (14/30) had PPR, and 10% (3/30) had phymatous rosacea (Table 3). Papulopustular rosacea was found to be significantly associated with Demodex positivity (P=.003); neither ETR nor phymatous rosacea was found to be significantly associated with Demodex infestation (P=.66 and P=.13, respectively)(Table 3).

Sixteen (53.3%) patients (32 eyes) in the Demodex-negative group and 21 (70%) patients (42 eyes) in the Demodex-positive group had TBUT values less than 10 seconds. Eighteen (60%) patients (36 eyes) in the Demodex-negative group and 25 (83.3%) patients (50 eyes) in the Demodex-positive group had Schirmer scores less than 10 mm (Table 3). The 2 groups were not significantly different in dry eye findings (P=.25 and P=.29, respectively).

Comment

Inflammation in Rosacea
It is known that the density of nonfloral bacteria as well as D folliculorum and Demodex brevis increases in skin affected by rosacea compared to normal skin. Vascular dilation associated with rosacea that results from sunlight and heat causes increased capillary permeability and creates the ideal environment for the proliferation of D folliculorum. Demodex is thought to act as a vector for the activity of certain other microorganisms, particularly Bacillus oleronius, and thus initiates the inflammatory response associated with rosacea.⁹

One study reported that the inflammation associated with rosacea that was caused by Demodex and other environmental stimuli occurred through toll-like receptor 2 and various cytokines.¹⁰ It has been reported that the abnormal function of toll-like receptor 2 in the epidermis leads to the increased production of cathelicidin. Cathelicidin is an antimicrobial peptide with both vasoactive and proinflammatory activity and has been used as a basis to explain the pathogenesis of facial erythema, flushing, and telangiectasia in the context of rosacea.^11,12 In addition, it has been reported that the increased secretion of proinflammatory cytokines such as IL-1 and gelatinase B in ocular rosacea leads to tearing film abnormalities that result from increased bacterial flora in the eyelids, which subsequently leads to decreased tear drainage and dry eyes.¹³ In addition, B oleronius isolated from a D folliculorum mite from patients with PPR produced proteins that induced an inflammatory immune response in 73% (16/22) of patients with rosacea.¹⁴

Ocular Findings in Rosacea Patients
In our study, PPR was found to be significantly associated with Demodex positivity compared to ETR and phymatous rosacea (P=.003). However, ocular inflammation findings such as blepharitis and meibomitis were not significantly different between Demodex-positive and Demodex-negative patients. Although the mean Schirmer score of Demodex-positive patients was lower than Demodex-negative patients, this difference was not statistically significant. We evaluated a TBUT of less than 10 seconds and a Schirmer score less than 10 mm as dry eye. Accordingly, the number of patients with dry eye was higher in the Demodex-positive group, but this difference was not statistically significant.

Chronic blepharitis, conjunctival inflammation, and meibomian gland dysfunction are among the most common findings of ocular rosacea.^15,16 Patients with ocular rosacea commonly have dry eye and abnormal TBUT and Schirmer scores.¹⁷ In our study, we found that the fluorescein TBUT and Schirmer scores were more likely to be abnormal in the Demodex-positive group, but the difference between the 2 groups was not statistically significant.

It has been reported that proinflammatory cytokines due to a weakened immune system in rosacea patients were increased. The weakened immune system was further supported by the increased concentrations of proinflammatory cytokines such as IL-1 and matrix metalloproteinase 9 in these patients’ tears and the improvement of symptoms after the inhibition of these cytokines.¹¹ Luo et al¹⁸ reported that Demodex inflammation causes dry eye, particularly with D brevis. Ayyildiz and Sezgin¹⁹ reported that Schirmer scores were significantly lower and that the Ocular Surface Disease Index had significantly increased in the Demodex-positive group compared to the Demodex-negative group (P=.001 for both). A Korean study reported that Demodex density was correlated with age, sex, and TBUT results, but there was no significant relationship between Demodex density and Schirmer scores.¹⁶

Sobolewska et al²⁰ administered ivermectin cream 1% to 10 patients with cutaneous and ocular rosacea, but only to the forehead, chin, nose, cheeks, and regions close to the eyelids, and observed a significant improvement in blepharitis (P=.004). They stated that ivermectin, as applied only to the face, suppressed the proinflammatory cytokines associated with rosacea and showed anti-inflammatory effects by reducing Demodex mites.²⁰Li et al²¹ demonstrated a strong correlation between ocular Demodex inflammation and serum reactivity to these bacterial proteins in patients with ocular rosacea, and they found that eyelid margin inflammation and facial rosacea correlated with reactivity to these proteins. These studies suggest a possible role for Demodex infestation and bacterial proteins in the etiology of rosacea.

Gonzalez-Hinojosa et al²² demonstrated that even though eyelash blepharitis was more common in PPR than ETR, there was no statistically significant association between rosacea and Demodex blepharitis. In our study, we found a significant correlation between PPR and Demodex positivity. Also, meibomian gland dysfunction was more common in the Demodex-positive group; however, this result was not statistically significant. One study compared patients with primary demodicosis and patients with rosacea with Demodex-induced blepharitis to healthy controls and found that patients with primary demodicosis and patients with rosacea did not have significantly different ocular findings.²³ In contrast, Forton and De Maertelaer²⁴ reported that patients with PPR had significantly more severe ocular manifestations compared with patients with demodicosis (P=.004).

Mizuno et al²⁵ compared the normal (nonrosacea) population with and without Demodex-infested eyelashes and found that the 2 groups were not significantly different for meibomian gland dysfunction, fluorescein TBUT, or ocular surface discomfort.

Varying results have been reported regarding the association between Demodex and blepharitis or ocular surface discomfort with or without rosacea. In our study, we found that Demodex did not affect tear function tests or meibomian gland function in patients with rosacea. We believe this study is important because it demonstrates the effects of Demodex on ocular findings in patients with cutaneous rosacea.

Limitations
Our study has some limitations. The number of patients was relatively small, resulting in few significant differences between the comparison groups. A larger prospective research study is required to assess the prevalence of Demodex mites in the ocular rosacea population along with associated symptoms and findings.

Conclusion

Rosacea is a chronic disease associated with skin and ocular manifestations that range from mild to severe, that progresses in the form of attacks, and that requires long-term follow-up and treatment. Rosacea most often presents as a disease that causes ocular surface inflammation of varying degrees. Demodex infestation may increase cutaneous or ocular inflammation in rosacea. Therefore, every patient diagnosed with rosacea should be given a dermatologic examination to determine Demodex positivity and an ophthalmologic examination to determine ocular manifestations.

Intraosseous (IO) access can afford a lifesaving means of vascular access in emergency settings, as it allows for the administration of large volumes of fluids, blood products, and medications at high flow rates directly into the highly vascularized osseous medullary cavity.¹ Fortunately, the complication rate with this resuscitative effort is low, with many reports demonstrating complication rates of less than 1%.² The most commonly reported complications include fluid extravasation, osteomyelitis, traumatic bone fracture, and epiphyseal plate damage.^1-3 Although compartment syndrome and skin necrosis have been reported,^4,5 there is no comprehensive list of sequelae resulting from fluid extravasation in the literature, and there are no known studies examining the incidence and types of cutaneous complications. In this study, we sought to evaluate the dermatologic impacts of this procedure.

Methods

We performed a retrospective chart review approved by the institutional review board at a large metropolitan level I trauma center in the Midwestern United States spanning 18 consecutive months to identify all patients who underwent IO line placement, either en route to or upon arrival at the trauma center. The electronic medical records of 113 patients (age range, 10 days–94 years) were identified using either an automated natural language look-up program with keywords including intraosseous access and IO or a Current Procedural Terminology code 36680. Data including patient age, reason for IO insertion, anatomic location of the IO, and complications secondary to IO line placement were recorded.

Results

We identified an overall complication rate of 2.7% (3/113), with only 1 patient showing isolated cutaneous complications from IO line placement. The complications in the first 2 patients included compartment syndrome following IO line placement in the right tibia and needle breakage during IO line placement. The third patient, a 30-year-old heart transplant recipient, developed tense bullae on the left leg 5 days after a resuscitative effort required IO access through the bilateral tibiae. The patient had received vasopressors as well as 750 mL of normal saline through these access points. Two days after resuscitation, she developed an enlarging ecchymosis around the left IO access point. On day 5, cutaneous findings included 2 large firm bullae on a purpuric base overlying the left proximal tibia and patella (Figure 1). After an ultrasound revealed no connection to the underlying joint space, the bullae were incised and drained for patient comfort, as she reported pain and undue pressure at the knee joint on initial dermatologic consultation. These symptoms abated after the procedure. Cultures of bullous fluid were negative for infection. Histopathologic examination revealed a subepidermal split with underlying minimal mixed inflammation, favoring the diagnosis of traumatic bullae.

At a scheduled 7-month dermatology follow-up, the wound bed appeared to be healing well with surrounding scarring with no residual bleeding or drainage (Figure 2) despite the patient reporting a protracted course of wound healing requiring debridement due to eschar formation and multiple follow-up appointments with the wound care service.

Comment

The most commonly reported complications with IO line placement result from fluid infiltration of the subcutaneous tissue secondary to catheter misplacement.^1,3 Extravasated fluid may lead to tissue damage, compartment syndrome, and even tissue necrosis in some cases.^1,4,5 Localized cellulitis and the formation of subcutaneous abscesses also have been reported, albeit rarely.^3,5

In our retrospective cohort review, we identified an additional potential complication of IO line placement that has not been widely reported—development of large traumatic bullae. It is most likely that this patient’s IO catheter became dislodged, resulting in extravasation of fluids into the dermal and subcutaneous tissues.

Our findings support the previously noted complication rate of less than 1% following IO line placement, with an overall complication rate of 2.7% that included only 1 patient with a cutaneous complication.² Given this low incidence, providers may not be used to recognizing such complications, leading to delayed or incorrect diagnosis of these entities. While there are certain conditions in which IO insertion is contraindicated, including severe bone diseases (eg, osteogenesis imperfecta, osteomyelitis), overlying cellulitis, and bone fracture, these conditions are rare and can be avoided in most cases by use of an alternative site for needle insertion.² Due to the widespread utility of this tool and its few contraindications, its use in hospitalized patients is rapidly increasing, necessitating a need for quick recognition of potential complications.

From previous data on the incidence of traumatic blisters with underlying bone fractures, there are several identifiable risk factors that could be extended to patients at high risk for developing cutaneous IO complications secondary to the trauma associated with needle insertion,⁶ including wound-healing impairments in patients with fragile lymphatics, peripheral vascular disease, diabetes, or collagen vascular diseases (eg, lupus, rheumatoid arthritis, Sjögren syndrome). Patients with these conditions should be closely monitored for the development of bullae.⁶ While the patient we highlighted in our study did not have a history of such conditions, her history of cardiac disease, recent resuscitation attempts, and immunosuppression certainly could have contributed to suboptimal tissue agility and repair after IO line placement.

Conclusion

Intraosseous access is a safe, effective, and reliable option for vascular access in both pediatric and adult populations that is widely used in both prehospital (ie, paramedic administered) and hospital settings, including intensive care units, emergency departments, and any acute situation where rapid vascular access is necessary. This retrospective chart review examining the incidence and types of cutaneous complications associated with IO line placement at a level I trauma center revealed a total complication rate similar to those reported in previous studies and also highlighted a unique postprocedural cutaneous finding of traumatic bullae. Although no unified management recommendations currently exist, providers should consider this complication in the differential for hospitalized patients with large, atypical, asymmetric bullae in the absence of an alternative explanation for such skin findings.

Intraosseous (IO) access can afford a lifesaving means of vascular access in emergency settings, as it allows for the administration of large volumes of fluids, blood products, and medications at high flow rates directly into the highly vascularized osseous medullary cavity.¹ Fortunately, the complication rate with this resuscitative effort is low, with many reports demonstrating complication rates of less than 1%.² The most commonly reported complications include fluid extravasation, osteomyelitis, traumatic bone fracture, and epiphyseal plate damage.^1-3 Although compartment syndrome and skin necrosis have been reported,^4,5 there is no comprehensive list of sequelae resulting from fluid extravasation in the literature, and there are no known studies examining the incidence and types of cutaneous complications. In this study, we sought to evaluate the dermatologic impacts of this procedure.

Methods

We performed a retrospective chart review approved by the institutional review board at a large metropolitan level I trauma center in the Midwestern United States spanning 18 consecutive months to identify all patients who underwent IO line placement, either en route to or upon arrival at the trauma center. The electronic medical records of 113 patients (age range, 10 days–94 years) were identified using either an automated natural language look-up program with keywords including intraosseous access and IO or a Current Procedural Terminology code 36680. Data including patient age, reason for IO insertion, anatomic location of the IO, and complications secondary to IO line placement were recorded.

Results

We identified an overall complication rate of 2.7% (3/113), with only 1 patient showing isolated cutaneous complications from IO line placement. The complications in the first 2 patients included compartment syndrome following IO line placement in the right tibia and needle breakage during IO line placement. The third patient, a 30-year-old heart transplant recipient, developed tense bullae on the left leg 5 days after a resuscitative effort required IO access through the bilateral tibiae. The patient had received vasopressors as well as 750 mL of normal saline through these access points. Two days after resuscitation, she developed an enlarging ecchymosis around the left IO access point. On day 5, cutaneous findings included 2 large firm bullae on a purpuric base overlying the left proximal tibia and patella (Figure 1). After an ultrasound revealed no connection to the underlying joint space, the bullae were incised and drained for patient comfort, as she reported pain and undue pressure at the knee joint on initial dermatologic consultation. These symptoms abated after the procedure. Cultures of bullous fluid were negative for infection. Histopathologic examination revealed a subepidermal split with underlying minimal mixed inflammation, favoring the diagnosis of traumatic bullae.

At a scheduled 7-month dermatology follow-up, the wound bed appeared to be healing well with surrounding scarring with no residual bleeding or drainage (Figure 2) despite the patient reporting a protracted course of wound healing requiring debridement due to eschar formation and multiple follow-up appointments with the wound care service.

Comment

The most commonly reported complications with IO line placement result from fluid infiltration of the subcutaneous tissue secondary to catheter misplacement.^1,3 Extravasated fluid may lead to tissue damage, compartment syndrome, and even tissue necrosis in some cases.^1,4,5 Localized cellulitis and the formation of subcutaneous abscesses also have been reported, albeit rarely.^3,5

In our retrospective cohort review, we identified an additional potential complication of IO line placement that has not been widely reported—development of large traumatic bullae. It is most likely that this patient’s IO catheter became dislodged, resulting in extravasation of fluids into the dermal and subcutaneous tissues.

Our findings support the previously noted complication rate of less than 1% following IO line placement, with an overall complication rate of 2.7% that included only 1 patient with a cutaneous complication.² Given this low incidence, providers may not be used to recognizing such complications, leading to delayed or incorrect diagnosis of these entities. While there are certain conditions in which IO insertion is contraindicated, including severe bone diseases (eg, osteogenesis imperfecta, osteomyelitis), overlying cellulitis, and bone fracture, these conditions are rare and can be avoided in most cases by use of an alternative site for needle insertion.² Due to the widespread utility of this tool and its few contraindications, its use in hospitalized patients is rapidly increasing, necessitating a need for quick recognition of potential complications.

From previous data on the incidence of traumatic blisters with underlying bone fractures, there are several identifiable risk factors that could be extended to patients at high risk for developing cutaneous IO complications secondary to the trauma associated with needle insertion,⁶ including wound-healing impairments in patients with fragile lymphatics, peripheral vascular disease, diabetes, or collagen vascular diseases (eg, lupus, rheumatoid arthritis, Sjögren syndrome). Patients with these conditions should be closely monitored for the development of bullae.⁶ While the patient we highlighted in our study did not have a history of such conditions, her history of cardiac disease, recent resuscitation attempts, and immunosuppression certainly could have contributed to suboptimal tissue agility and repair after IO line placement.

Conclusion

Intraosseous access is a safe, effective, and reliable option for vascular access in both pediatric and adult populations that is widely used in both prehospital (ie, paramedic administered) and hospital settings, including intensive care units, emergency departments, and any acute situation where rapid vascular access is necessary. This retrospective chart review examining the incidence and types of cutaneous complications associated with IO line placement at a level I trauma center revealed a total complication rate similar to those reported in previous studies and also highlighted a unique postprocedural cutaneous finding of traumatic bullae. Although no unified management recommendations currently exist, providers should consider this complication in the differential for hospitalized patients with large, atypical, asymmetric bullae in the absence of an alternative explanation for such skin findings.

Intraosseous (IO) access can afford a lifesaving means of vascular access in emergency settings, as it allows for the administration of large volumes of fluids, blood products, and medications at high flow rates directly into the highly vascularized osseous medullary cavity.¹ Fortunately, the complication rate with this resuscitative effort is low, with many reports demonstrating complication rates of less than 1%.² The most commonly reported complications include fluid extravasation, osteomyelitis, traumatic bone fracture, and epiphyseal plate damage.^1-3 Although compartment syndrome and skin necrosis have been reported,^4,5 there is no comprehensive list of sequelae resulting from fluid extravasation in the literature, and there are no known studies examining the incidence and types of cutaneous complications. In this study, we sought to evaluate the dermatologic impacts of this procedure.

Methods

We performed a retrospective chart review approved by the institutional review board at a large metropolitan level I trauma center in the Midwestern United States spanning 18 consecutive months to identify all patients who underwent IO line placement, either en route to or upon arrival at the trauma center. The electronic medical records of 113 patients (age range, 10 days–94 years) were identified using either an automated natural language look-up program with keywords including intraosseous access and IO or a Current Procedural Terminology code 36680. Data including patient age, reason for IO insertion, anatomic location of the IO, and complications secondary to IO line placement were recorded.

Results

We identified an overall complication rate of 2.7% (3/113), with only 1 patient showing isolated cutaneous complications from IO line placement. The complications in the first 2 patients included compartment syndrome following IO line placement in the right tibia and needle breakage during IO line placement. The third patient, a 30-year-old heart transplant recipient, developed tense bullae on the left leg 5 days after a resuscitative effort required IO access through the bilateral tibiae. The patient had received vasopressors as well as 750 mL of normal saline through these access points. Two days after resuscitation, she developed an enlarging ecchymosis around the left IO access point. On day 5, cutaneous findings included 2 large firm bullae on a purpuric base overlying the left proximal tibia and patella (Figure 1). After an ultrasound revealed no connection to the underlying joint space, the bullae were incised and drained for patient comfort, as she reported pain and undue pressure at the knee joint on initial dermatologic consultation. These symptoms abated after the procedure. Cultures of bullous fluid were negative for infection. Histopathologic examination revealed a subepidermal split with underlying minimal mixed inflammation, favoring the diagnosis of traumatic bullae.

At a scheduled 7-month dermatology follow-up, the wound bed appeared to be healing well with surrounding scarring with no residual bleeding or drainage (Figure 2) despite the patient reporting a protracted course of wound healing requiring debridement due to eschar formation and multiple follow-up appointments with the wound care service.

Comment

The most commonly reported complications with IO line placement result from fluid infiltration of the subcutaneous tissue secondary to catheter misplacement.^1,3 Extravasated fluid may lead to tissue damage, compartment syndrome, and even tissue necrosis in some cases.^1,4,5 Localized cellulitis and the formation of subcutaneous abscesses also have been reported, albeit rarely.^3,5

In our retrospective cohort review, we identified an additional potential complication of IO line placement that has not been widely reported—development of large traumatic bullae. It is most likely that this patient’s IO catheter became dislodged, resulting in extravasation of fluids into the dermal and subcutaneous tissues.

Our findings support the previously noted complication rate of less than 1% following IO line placement, with an overall complication rate of 2.7% that included only 1 patient with a cutaneous complication.² Given this low incidence, providers may not be used to recognizing such complications, leading to delayed or incorrect diagnosis of these entities. While there are certain conditions in which IO insertion is contraindicated, including severe bone diseases (eg, osteogenesis imperfecta, osteomyelitis), overlying cellulitis, and bone fracture, these conditions are rare and can be avoided in most cases by use of an alternative site for needle insertion.² Due to the widespread utility of this tool and its few contraindications, its use in hospitalized patients is rapidly increasing, necessitating a need for quick recognition of potential complications.

From previous data on the incidence of traumatic blisters with underlying bone fractures, there are several identifiable risk factors that could be extended to patients at high risk for developing cutaneous IO complications secondary to the trauma associated with needle insertion,⁶ including wound-healing impairments in patients with fragile lymphatics, peripheral vascular disease, diabetes, or collagen vascular diseases (eg, lupus, rheumatoid arthritis, Sjögren syndrome). Patients with these conditions should be closely monitored for the development of bullae.⁶ While the patient we highlighted in our study did not have a history of such conditions, her history of cardiac disease, recent resuscitation attempts, and immunosuppression certainly could have contributed to suboptimal tissue agility and repair after IO line placement.

Conclusion

Intraosseous access is a safe, effective, and reliable option for vascular access in both pediatric and adult populations that is widely used in both prehospital (ie, paramedic administered) and hospital settings, including intensive care units, emergency departments, and any acute situation where rapid vascular access is necessary. This retrospective chart review examining the incidence and types of cutaneous complications associated with IO line placement at a level I trauma center revealed a total complication rate similar to those reported in previous studies and also highlighted a unique postprocedural cutaneous finding of traumatic bullae. Although no unified management recommendations currently exist, providers should consider this complication in the differential for hospitalized patients with large, atypical, asymmetric bullae in the absence of an alternative explanation for such skin findings.

Regarded as dermatologic emergencies, Stevens-Johnson syndrome (SJS) and toxic epidermal necrolysis (TEN) represent a spectrum of blistering skin diseases that have a high mortality rate. Because of a misguided immune response to medications or infections, CD8⁺ T lymphocytes release proinflammatory cytokines, giving rise to the extensive epidermal destruction seen in SJS and TEN. The exact pathogenesis of SJS and TEN is still poorly defined, but studies have proposed that T cells mediate keratinocyte (KC) apoptosis through perforin and granzyme release and activation of the Fas/Fas ligand (FasL). Functioning as a transmembrane death receptor in the tumor necrosis factor (TNF) superfamily, Fas (CD95) activates Fas-associated death domain protein, caspases, and nucleases, resulting in organized cell destruction. Likewise, perforin and granzymes also have been shown to play a similar role in apoptosis via activation of caspases.¹

Evidence for the role of TNF-α in SJS and TEN has been supported by findings of elevated levels of TNF-α within the blister fluid, serum, and KC cell surface. Additionally, TNF-α has been shown to upregulate inducible nitric oxide synthase in KCs, causing an accumulation of nitric oxide and subsequent FasL-mediated cell death.^1-3 Notably, studies have demonstrated a relative lack of lymphocytes in the tissue of TEN patients despite the extensive destruction that is observed, thus emphasizing the importance of amplification and cell signaling via inflammatory mediators such as TNF-α.¹ In this proposed model, T cells release IFN-γ, causing KCs to release TNF-α that subsequently promotes the upregulation of the aforementioned FasL.¹ Tumor necrosis factor α also may promote increased MHC class I complex deposition on KC surfaces that may play a role in perforin and granzyme-mediated apoptosis of KCs.¹

There is still debate on the standard of care for the treatment of SJS and TEN, attributed to the absence of randomized controlled trials and the rarity of the disease as well as the numerous conflicting studies evaluating potential treatments.^1,4 Despite conflicting data to support their use, supportive care and intravenous immunoglobulin (IVIG) continue to be common treatments for SJS and TEN in hospitals worldwide. Elucidation of the role of TNF-α has prompted the use of infliximab and etanercept. In a case series of Italian patients with TEN (average SCORTEN, 3.6) treated with the TNF-α antagonist etanercept, no mortality was observed, which was well below the calculated expected mortality of 46.9%.² Our retrospective study compared the use of a TNF antagonist to other therapies in the treatment of SJS/TEN. Our data suggest that etanercept is a lifesaving and disease-modifying therapy.

Methods

Twenty-two patients with SJS/TEN were included in this analysis. This included all patients who carried a clinical diagnosis of SJS/TEN with a confirmatory biopsy at our 2 university centers—University of California, Los Angeles, and Keck-LA County-Norris Hospital at the University of Southern California, Los Angeles—from 2013 to 2016. The diagnosis was rendered when a clinical diagnosis of SJS/TEN was given by a dermatologist and a confirmatory biopsy was performed. Every patient given a diagnosis of SJS/TEN at either university system from 2015 onward received an injection of etanercept given the positive results reported by Paradisi et al.²

The 9 patients who presented from 2013 to 2014 to our 2 hospital systems and were given a diagnosis of SJS/TEN received either IVIG or supportive care alone and had an average body surface area (BSA) affected of 23%. The 13 patients who presented from 2015 to 2016 were treated with etanercept in the form of a 50-mg subcutaneous injection given once to the right upper arm. Of this group, 4 patients received dual therapy with both IVIG and etanercept. In the etanercept-treated group (etanercept alone and etanercept plus IVIG), the average BSA affected was 30%. At the time of preliminary diagnosis, all patient medications were evaluated for a possible temporal relationship to the onset of rash and were discontinued if felt to be causative. The causative agent and treatment course for each patient is summarized in Table 1.

Results

A total of 13 patients received etanercept. The mean SCORTEN was 2.2. The observed mortality was 0%, which was markedly lower than the predicted mortality of 24.3% (as determined by linear interpolation). Of this cohort, 9 patients received etanercept alone (mean SCORTEN of 2.1, predicted mortality of 22.9%), whereas 4 patients received a combination of etanercept and IVIG (mean SCORTEN of 2.3, predicted mortality of 27.2%).

The 4 patients who received both etanercept and IVIG received dual therapy for varying reasons. In patient 2 (Table 1), the perceived severity of this case ultimately led to the decision to start IVIG in addition to etanercept, resulting in rapid recovery and discharge after only 1 week of hospitalization. Intravenous immunoglobulin also was given in patient 3 (SCORTEN of 4) and patient 6 (SCORTEN of 2) for progression of disease despite administration of etanercept, with subsequent cessation of progression after the addition of the second agent (IVIG). Patient 12 might have done well on etanercept monotherapy but was administered IVIG as a precautionary measure because of hospital treatment algorithms.

Nine patients did not receive etanercept. Of this group, 5 received IVIG and 4 were managed with supportive care alone. The average SCORTEN for this group was 2.4, only slightly higher than the group that received etanercept (Table 2). The mortality rate in this group was 33%, which was higher than the predicted mortality of 28.1%.

Comment

There is no definitive gold standard treatment of SJS, SJS/TEN overlap, or TEN. However, generally agreed upon management includes immediate discontinuation of the offending medication and supportive therapy with aggressive electrolyte replacement and wound care. Management in a burn unit or intensive care unit is recommended in severe cases. Contention over the efficacy of various medications in the treatment of SJS and TEN continues and largely is due to the rarity of SJS and TEN; studies are small and almost all lack randomization. Therapies that have been used include high-dose steroids, IVIG, plasmapheresis, cyclophosphamide, cyclosporine A, and TNF inhibitors (eg, etanercept, infliximab).¹

Evidence for the use of anti–TNF-α antibodies has been limited thus far, with most of the literature focusing on infliximab and etanercept. Adalimumab, a fully humanized clonal antibody, has no reported cases in the dermatologic literature for use in patients with SJS/TEN. Two case reports of adalimumab paradoxically causing SJS have been documented. In both cases, adalimumab was stopped and patients responded to intravenous corticosteroids and infliximab.^7,8 Similarly, thalidomide has not proven to be a promising anti–TNF-α agent for the treatment of SJS/TEN. In the only attempted randomized controlled trial for SJS and TEN, thalidomide appeared to increase mortality, eventuating in this trial being terminated prior to the planned end date.⁹Infliximab and etanercept have several case reports and a few case series highlighting potentially efficacious application of TNF-α inhibitors for the treatment of SJS/TEN.^10-13 In 2002, Fischer et al¹⁰ reported the first case of TEN treated successfully with a single dose of infliximab 5 mg/kg. Kreft et al¹⁴ reported on etoricoxib-induced TEN that was treated with infliximab 5 mg/kg, which led to re-epithelialization within 5 weeks (notably a 5-week re-epithelialization time is not necessarily an improvement).

In 2005, Hunger et al³ demonstrated TNF-α’s release by KCs in the epidermis and by inflammatory cells in the dermis of a TEN patient. Twenty-four hours after the administration of infliximab 5 mg/kg in these patients, TNF-α was found to be below normal and epidermal detachment ceased.³ Wojtkietwicz et al¹³ demonstrated benefit following an infusion of infliximab 5 mg/kg in a patient whose disease continued to progress despite treatment with dexamethasone and 1.8 g/kg of IVIG.

Then 2 subsequent case series added further support for the efficacy of infliximab in the treatment of TEN. Patmanidis et al¹⁵ and Gaitanis et al¹⁶ reported similar results in 4 patients, each treated with infliximab 5 mg/kg immediately followed by initiation of high-dose IVIG (2 g/kg over 5 days). Zárate-Correa et al¹⁷ reported a 0% mortality rate and near-complete re-epithelialization after 5 to 14 days in 4 patients treated with a single 300-mg dose of infliximab.

However, the success of infliximab in the treatment of TEN has been countered by the pilot study by Paquet et al,¹⁸ which compared the efficacy of 150 mg/kg of N-acetylcysteine alone vs adding infliximab 5 mg/kg to treat 10 TEN patients. The study demonstrated no benefit at 48 hours in the group given infliximab, the time frame in which prior case reports touting infliximab’s benefit claimed the benefit was observed. Similarly, there was no effect on mortality for either treatment modality as assessed by illness auxiliary score.¹⁸

Evidence in support of the use of etanercept in the treatment of SJS/TEN is mounting, and some centers have begun to use it as the first-choice therapy for SJS/TEN. The first case was reported by Famularo et al,¹⁹ in which a patient with TEN was given 2 doses of etanercept 25 mg after failure to improve with prednisolone 1 mg/kg. The patient showed near-complete and rapid re-epithelization in 6 days before death due to disseminated intravascular coagulation 10 days after admission.¹⁹ Gubinelli et al²⁰ and Sadighha²¹ independently reported cases of TEN and TEN/acute generalized exanthematous pustulosis overlap treated with a total of 50 mg of etanercept, demonstrating rapid cessation of lesion progression. Didona et al²² found similar benefit using etanercept 50 mg to treat TEN secondary to rituximab after failure to improve with prednisone and cyclophosphamide. Treatment of TEN with etanercept in an HIV-positive patient also has been reported. Lee et al²³ described a patient who was administered 50-mg and 25-mg injections on days 3 and 5 of hospitalization, respectively, with re-epithelialization occurring by day 8. Finally, Owczarczyk-Saczonek et al²⁴ reported a case of SJS in a patient with a 4-year history of etanercept and sulfasalazine treatment of rheumatoid arthritis; sulfasalazine was stopped, but this patient was continued on etanercept until resolution of skin and mucosal symptoms. However, it is important to consider the possibility of publication bias among these cases selected for their positive outcomes.

Perhaps the most compelling literature regarding the use of etanercept for TEN was described in a case series by Paradisi et al.² This study included 10 patients with TEN, all of whom demonstrated complete re-epithelialization shortly after receiving etanercept 50 mg. Average SCORTEN was 3.6 with a range of 2 to 6. Eight patients in this study had severe comorbidities and all 10 patients survived, with a time to re-epithelialization ranging from 7 to 20 days.² Additionally, a randomized controlled trial showed that 38 etanercept-treated patients had improved mortality (P=.266) and re-epithelialization time (P=.01) compared to patients treated with intravenous methylprednisolone.²⁵Limitations to our study are similar to other reports of SJS/TEN and included the small number of cases and lack of randomization. Additionally, we do not have data available for all patients for time between onset of disease and treatment initiation. Because of these challenges, data presented in this case series is observational only. Additionally, the patients treated with etanercept alone had a slightly lower SCORTEN compared to the group that received IVIG or supportive care alone (2.1 and 2.4 respectively). However, the etanercept-only group actually had higher involvement of epidermal detachment (33%) compared to the non-etanercept group (23%).

Conclusion

Although treatment with etanercept lacks the support of a randomized controlled trial, similar to all other treatments currently used for SJS and TEN, preliminary reports highlight a benefit in disease progression and improvement in time to re-epithelialization. In particular, if etanercept 50 mg subcutaneously is given as monotherapy or is given early in the disease course (prior to other therapies being attempted and ideally within 6 hours of presentation), our data suggest an even greater trend toward improved mortality and decreased time to re-epithelialization. Additionally, our findings may suggest that in some patients, etanercept monotherapy is not an adequate intervention but the addition of IVIG may be helpful; however, the senior author (S.W.) notes anecdotally that in his experience with the patients treated at the University of California Los Angeles, the order of administration of combination therapies—etanercept followed by IVIG—was important in addition to the choice of therapy. These findings are promising enough to warrant a multicenter randomized controlled trial comparing the efficacy of etanercept to other more commonly used treatments for this spectrum of disease, including IVIG and/or cyclosporine. Based on the data presented in this case series, including the 13 patients who received etanercept and had a 0% mortality rate, etanercept may be viewed as a targeted therapeutic intervention for patients with SJS and TEN.

Regarded as dermatologic emergencies, Stevens-Johnson syndrome (SJS) and toxic epidermal necrolysis (TEN) represent a spectrum of blistering skin diseases that have a high mortality rate. Because of a misguided immune response to medications or infections, CD8⁺ T lymphocytes release proinflammatory cytokines, giving rise to the extensive epidermal destruction seen in SJS and TEN. The exact pathogenesis of SJS and TEN is still poorly defined, but studies have proposed that T cells mediate keratinocyte (KC) apoptosis through perforin and granzyme release and activation of the Fas/Fas ligand (FasL). Functioning as a transmembrane death receptor in the tumor necrosis factor (TNF) superfamily, Fas (CD95) activates Fas-associated death domain protein, caspases, and nucleases, resulting in organized cell destruction. Likewise, perforin and granzymes also have been shown to play a similar role in apoptosis via activation of caspases.¹

Evidence for the role of TNF-α in SJS and TEN has been supported by findings of elevated levels of TNF-α within the blister fluid, serum, and KC cell surface. Additionally, TNF-α has been shown to upregulate inducible nitric oxide synthase in KCs, causing an accumulation of nitric oxide and subsequent FasL-mediated cell death.^1-3 Notably, studies have demonstrated a relative lack of lymphocytes in the tissue of TEN patients despite the extensive destruction that is observed, thus emphasizing the importance of amplification and cell signaling via inflammatory mediators such as TNF-α.¹ In this proposed model, T cells release IFN-γ, causing KCs to release TNF-α that subsequently promotes the upregulation of the aforementioned FasL.¹ Tumor necrosis factor α also may promote increased MHC class I complex deposition on KC surfaces that may play a role in perforin and granzyme-mediated apoptosis of KCs.¹

There is still debate on the standard of care for the treatment of SJS and TEN, attributed to the absence of randomized controlled trials and the rarity of the disease as well as the numerous conflicting studies evaluating potential treatments.^1,4 Despite conflicting data to support their use, supportive care and intravenous immunoglobulin (IVIG) continue to be common treatments for SJS and TEN in hospitals worldwide. Elucidation of the role of TNF-α has prompted the use of infliximab and etanercept. In a case series of Italian patients with TEN (average SCORTEN, 3.6) treated with the TNF-α antagonist etanercept, no mortality was observed, which was well below the calculated expected mortality of 46.9%.² Our retrospective study compared the use of a TNF antagonist to other therapies in the treatment of SJS/TEN. Our data suggest that etanercept is a lifesaving and disease-modifying therapy.

Methods

Twenty-two patients with SJS/TEN were included in this analysis. This included all patients who carried a clinical diagnosis of SJS/TEN with a confirmatory biopsy at our 2 university centers—University of California, Los Angeles, and Keck-LA County-Norris Hospital at the University of Southern California, Los Angeles—from 2013 to 2016. The diagnosis was rendered when a clinical diagnosis of SJS/TEN was given by a dermatologist and a confirmatory biopsy was performed. Every patient given a diagnosis of SJS/TEN at either university system from 2015 onward received an injection of etanercept given the positive results reported by Paradisi et al.²

The 9 patients who presented from 2013 to 2014 to our 2 hospital systems and were given a diagnosis of SJS/TEN received either IVIG or supportive care alone and had an average body surface area (BSA) affected of 23%. The 13 patients who presented from 2015 to 2016 were treated with etanercept in the form of a 50-mg subcutaneous injection given once to the right upper arm. Of this group, 4 patients received dual therapy with both IVIG and etanercept. In the etanercept-treated group (etanercept alone and etanercept plus IVIG), the average BSA affected was 30%. At the time of preliminary diagnosis, all patient medications were evaluated for a possible temporal relationship to the onset of rash and were discontinued if felt to be causative. The causative agent and treatment course for each patient is summarized in Table 1.

Results

A total of 13 patients received etanercept. The mean SCORTEN was 2.2. The observed mortality was 0%, which was markedly lower than the predicted mortality of 24.3% (as determined by linear interpolation). Of this cohort, 9 patients received etanercept alone (mean SCORTEN of 2.1, predicted mortality of 22.9%), whereas 4 patients received a combination of etanercept and IVIG (mean SCORTEN of 2.3, predicted mortality of 27.2%).

The 4 patients who received both etanercept and IVIG received dual therapy for varying reasons. In patient 2 (Table 1), the perceived severity of this case ultimately led to the decision to start IVIG in addition to etanercept, resulting in rapid recovery and discharge after only 1 week of hospitalization. Intravenous immunoglobulin also was given in patient 3 (SCORTEN of 4) and patient 6 (SCORTEN of 2) for progression of disease despite administration of etanercept, with subsequent cessation of progression after the addition of the second agent (IVIG). Patient 12 might have done well on etanercept monotherapy but was administered IVIG as a precautionary measure because of hospital treatment algorithms.

Nine patients did not receive etanercept. Of this group, 5 received IVIG and 4 were managed with supportive care alone. The average SCORTEN for this group was 2.4, only slightly higher than the group that received etanercept (Table 2). The mortality rate in this group was 33%, which was higher than the predicted mortality of 28.1%.

Comment

There is no definitive gold standard treatment of SJS, SJS/TEN overlap, or TEN. However, generally agreed upon management includes immediate discontinuation of the offending medication and supportive therapy with aggressive electrolyte replacement and wound care. Management in a burn unit or intensive care unit is recommended in severe cases. Contention over the efficacy of various medications in the treatment of SJS and TEN continues and largely is due to the rarity of SJS and TEN; studies are small and almost all lack randomization. Therapies that have been used include high-dose steroids, IVIG, plasmapheresis, cyclophosphamide, cyclosporine A, and TNF inhibitors (eg, etanercept, infliximab).¹

Evidence for the use of anti–TNF-α antibodies has been limited thus far, with most of the literature focusing on infliximab and etanercept. Adalimumab, a fully humanized clonal antibody, has no reported cases in the dermatologic literature for use in patients with SJS/TEN. Two case reports of adalimumab paradoxically causing SJS have been documented. In both cases, adalimumab was stopped and patients responded to intravenous corticosteroids and infliximab.^7,8 Similarly, thalidomide has not proven to be a promising anti–TNF-α agent for the treatment of SJS/TEN. In the only attempted randomized controlled trial for SJS and TEN, thalidomide appeared to increase mortality, eventuating in this trial being terminated prior to the planned end date.⁹Infliximab and etanercept have several case reports and a few case series highlighting potentially efficacious application of TNF-α inhibitors for the treatment of SJS/TEN.^10-13 In 2002, Fischer et al¹⁰ reported the first case of TEN treated successfully with a single dose of infliximab 5 mg/kg. Kreft et al¹⁴ reported on etoricoxib-induced TEN that was treated with infliximab 5 mg/kg, which led to re-epithelialization within 5 weeks (notably a 5-week re-epithelialization time is not necessarily an improvement).

In 2005, Hunger et al³ demonstrated TNF-α’s release by KCs in the epidermis and by inflammatory cells in the dermis of a TEN patient. Twenty-four hours after the administration of infliximab 5 mg/kg in these patients, TNF-α was found to be below normal and epidermal detachment ceased.³ Wojtkietwicz et al¹³ demonstrated benefit following an infusion of infliximab 5 mg/kg in a patient whose disease continued to progress despite treatment with dexamethasone and 1.8 g/kg of IVIG.

Then 2 subsequent case series added further support for the efficacy of infliximab in the treatment of TEN. Patmanidis et al¹⁵ and Gaitanis et al¹⁶ reported similar results in 4 patients, each treated with infliximab 5 mg/kg immediately followed by initiation of high-dose IVIG (2 g/kg over 5 days). Zárate-Correa et al¹⁷ reported a 0% mortality rate and near-complete re-epithelialization after 5 to 14 days in 4 patients treated with a single 300-mg dose of infliximab.

However, the success of infliximab in the treatment of TEN has been countered by the pilot study by Paquet et al,¹⁸ which compared the efficacy of 150 mg/kg of N-acetylcysteine alone vs adding infliximab 5 mg/kg to treat 10 TEN patients. The study demonstrated no benefit at 48 hours in the group given infliximab, the time frame in which prior case reports touting infliximab’s benefit claimed the benefit was observed. Similarly, there was no effect on mortality for either treatment modality as assessed by illness auxiliary score.¹⁸

Evidence in support of the use of etanercept in the treatment of SJS/TEN is mounting, and some centers have begun to use it as the first-choice therapy for SJS/TEN. The first case was reported by Famularo et al,¹⁹ in which a patient with TEN was given 2 doses of etanercept 25 mg after failure to improve with prednisolone 1 mg/kg. The patient showed near-complete and rapid re-epithelization in 6 days before death due to disseminated intravascular coagulation 10 days after admission.¹⁹ Gubinelli et al²⁰ and Sadighha²¹ independently reported cases of TEN and TEN/acute generalized exanthematous pustulosis overlap treated with a total of 50 mg of etanercept, demonstrating rapid cessation of lesion progression. Didona et al²² found similar benefit using etanercept 50 mg to treat TEN secondary to rituximab after failure to improve with prednisone and cyclophosphamide. Treatment of TEN with etanercept in an HIV-positive patient also has been reported. Lee et al²³ described a patient who was administered 50-mg and 25-mg injections on days 3 and 5 of hospitalization, respectively, with re-epithelialization occurring by day 8. Finally, Owczarczyk-Saczonek et al²⁴ reported a case of SJS in a patient with a 4-year history of etanercept and sulfasalazine treatment of rheumatoid arthritis; sulfasalazine was stopped, but this patient was continued on etanercept until resolution of skin and mucosal symptoms. However, it is important to consider the possibility of publication bias among these cases selected for their positive outcomes.

Perhaps the most compelling literature regarding the use of etanercept for TEN was described in a case series by Paradisi et al.² This study included 10 patients with TEN, all of whom demonstrated complete re-epithelialization shortly after receiving etanercept 50 mg. Average SCORTEN was 3.6 with a range of 2 to 6. Eight patients in this study had severe comorbidities and all 10 patients survived, with a time to re-epithelialization ranging from 7 to 20 days.² Additionally, a randomized controlled trial showed that 38 etanercept-treated patients had improved mortality (P=.266) and re-epithelialization time (P=.01) compared to patients treated with intravenous methylprednisolone.²⁵Limitations to our study are similar to other reports of SJS/TEN and included the small number of cases and lack of randomization. Additionally, we do not have data available for all patients for time between onset of disease and treatment initiation. Because of these challenges, data presented in this case series is observational only. Additionally, the patients treated with etanercept alone had a slightly lower SCORTEN compared to the group that received IVIG or supportive care alone (2.1 and 2.4 respectively). However, the etanercept-only group actually had higher involvement of epidermal detachment (33%) compared to the non-etanercept group (23%).

Conclusion

Although treatment with etanercept lacks the support of a randomized controlled trial, similar to all other treatments currently used for SJS and TEN, preliminary reports highlight a benefit in disease progression and improvement in time to re-epithelialization. In particular, if etanercept 50 mg subcutaneously is given as monotherapy or is given early in the disease course (prior to other therapies being attempted and ideally within 6 hours of presentation), our data suggest an even greater trend toward improved mortality and decreased time to re-epithelialization. Additionally, our findings may suggest that in some patients, etanercept monotherapy is not an adequate intervention but the addition of IVIG may be helpful; however, the senior author (S.W.) notes anecdotally that in his experience with the patients treated at the University of California Los Angeles, the order of administration of combination therapies—etanercept followed by IVIG—was important in addition to the choice of therapy. These findings are promising enough to warrant a multicenter randomized controlled trial comparing the efficacy of etanercept to other more commonly used treatments for this spectrum of disease, including IVIG and/or cyclosporine. Based on the data presented in this case series, including the 13 patients who received etanercept and had a 0% mortality rate, etanercept may be viewed as a targeted therapeutic intervention for patients with SJS and TEN.

Regarded as dermatologic emergencies, Stevens-Johnson syndrome (SJS) and toxic epidermal necrolysis (TEN) represent a spectrum of blistering skin diseases that have a high mortality rate. Because of a misguided immune response to medications or infections, CD8⁺ T lymphocytes release proinflammatory cytokines, giving rise to the extensive epidermal destruction seen in SJS and TEN. The exact pathogenesis of SJS and TEN is still poorly defined, but studies have proposed that T cells mediate keratinocyte (KC) apoptosis through perforin and granzyme release and activation of the Fas/Fas ligand (FasL). Functioning as a transmembrane death receptor in the tumor necrosis factor (TNF) superfamily, Fas (CD95) activates Fas-associated death domain protein, caspases, and nucleases, resulting in organized cell destruction. Likewise, perforin and granzymes also have been shown to play a similar role in apoptosis via activation of caspases.¹

Evidence for the role of TNF-α in SJS and TEN has been supported by findings of elevated levels of TNF-α within the blister fluid, serum, and KC cell surface. Additionally, TNF-α has been shown to upregulate inducible nitric oxide synthase in KCs, causing an accumulation of nitric oxide and subsequent FasL-mediated cell death.^1-3 Notably, studies have demonstrated a relative lack of lymphocytes in the tissue of TEN patients despite the extensive destruction that is observed, thus emphasizing the importance of amplification and cell signaling via inflammatory mediators such as TNF-α.¹ In this proposed model, T cells release IFN-γ, causing KCs to release TNF-α that subsequently promotes the upregulation of the aforementioned FasL.¹ Tumor necrosis factor α also may promote increased MHC class I complex deposition on KC surfaces that may play a role in perforin and granzyme-mediated apoptosis of KCs.¹

There is still debate on the standard of care for the treatment of SJS and TEN, attributed to the absence of randomized controlled trials and the rarity of the disease as well as the numerous conflicting studies evaluating potential treatments.^1,4 Despite conflicting data to support their use, supportive care and intravenous immunoglobulin (IVIG) continue to be common treatments for SJS and TEN in hospitals worldwide. Elucidation of the role of TNF-α has prompted the use of infliximab and etanercept. In a case series of Italian patients with TEN (average SCORTEN, 3.6) treated with the TNF-α antagonist etanercept, no mortality was observed, which was well below the calculated expected mortality of 46.9%.² Our retrospective study compared the use of a TNF antagonist to other therapies in the treatment of SJS/TEN. Our data suggest that etanercept is a lifesaving and disease-modifying therapy.

Methods

Twenty-two patients with SJS/TEN were included in this analysis. This included all patients who carried a clinical diagnosis of SJS/TEN with a confirmatory biopsy at our 2 university centers—University of California, Los Angeles, and Keck-LA County-Norris Hospital at the University of Southern California, Los Angeles—from 2013 to 2016. The diagnosis was rendered when a clinical diagnosis of SJS/TEN was given by a dermatologist and a confirmatory biopsy was performed. Every patient given a diagnosis of SJS/TEN at either university system from 2015 onward received an injection of etanercept given the positive results reported by Paradisi et al.²

The 9 patients who presented from 2013 to 2014 to our 2 hospital systems and were given a diagnosis of SJS/TEN received either IVIG or supportive care alone and had an average body surface area (BSA) affected of 23%. The 13 patients who presented from 2015 to 2016 were treated with etanercept in the form of a 50-mg subcutaneous injection given once to the right upper arm. Of this group, 4 patients received dual therapy with both IVIG and etanercept. In the etanercept-treated group (etanercept alone and etanercept plus IVIG), the average BSA affected was 30%. At the time of preliminary diagnosis, all patient medications were evaluated for a possible temporal relationship to the onset of rash and were discontinued if felt to be causative. The causative agent and treatment course for each patient is summarized in Table 1.

Results

A total of 13 patients received etanercept. The mean SCORTEN was 2.2. The observed mortality was 0%, which was markedly lower than the predicted mortality of 24.3% (as determined by linear interpolation). Of this cohort, 9 patients received etanercept alone (mean SCORTEN of 2.1, predicted mortality of 22.9%), whereas 4 patients received a combination of etanercept and IVIG (mean SCORTEN of 2.3, predicted mortality of 27.2%).

The 4 patients who received both etanercept and IVIG received dual therapy for varying reasons. In patient 2 (Table 1), the perceived severity of this case ultimately led to the decision to start IVIG in addition to etanercept, resulting in rapid recovery and discharge after only 1 week of hospitalization. Intravenous immunoglobulin also was given in patient 3 (SCORTEN of 4) and patient 6 (SCORTEN of 2) for progression of disease despite administration of etanercept, with subsequent cessation of progression after the addition of the second agent (IVIG). Patient 12 might have done well on etanercept monotherapy but was administered IVIG as a precautionary measure because of hospital treatment algorithms.

Nine patients did not receive etanercept. Of this group, 5 received IVIG and 4 were managed with supportive care alone. The average SCORTEN for this group was 2.4, only slightly higher than the group that received etanercept (Table 2). The mortality rate in this group was 33%, which was higher than the predicted mortality of 28.1%.

Comment

There is no definitive gold standard treatment of SJS, SJS/TEN overlap, or TEN. However, generally agreed upon management includes immediate discontinuation of the offending medication and supportive therapy with aggressive electrolyte replacement and wound care. Management in a burn unit or intensive care unit is recommended in severe cases. Contention over the efficacy of various medications in the treatment of SJS and TEN continues and largely is due to the rarity of SJS and TEN; studies are small and almost all lack randomization. Therapies that have been used include high-dose steroids, IVIG, plasmapheresis, cyclophosphamide, cyclosporine A, and TNF inhibitors (eg, etanercept, infliximab).¹

Evidence for the use of anti–TNF-α antibodies has been limited thus far, with most of the literature focusing on infliximab and etanercept. Adalimumab, a fully humanized clonal antibody, has no reported cases in the dermatologic literature for use in patients with SJS/TEN. Two case reports of adalimumab paradoxically causing SJS have been documented. In both cases, adalimumab was stopped and patients responded to intravenous corticosteroids and infliximab.^7,8 Similarly, thalidomide has not proven to be a promising anti–TNF-α agent for the treatment of SJS/TEN. In the only attempted randomized controlled trial for SJS and TEN, thalidomide appeared to increase mortality, eventuating in this trial being terminated prior to the planned end date.⁹Infliximab and etanercept have several case reports and a few case series highlighting potentially efficacious application of TNF-α inhibitors for the treatment of SJS/TEN.^10-13 In 2002, Fischer et al¹⁰ reported the first case of TEN treated successfully with a single dose of infliximab 5 mg/kg. Kreft et al¹⁴ reported on etoricoxib-induced TEN that was treated with infliximab 5 mg/kg, which led to re-epithelialization within 5 weeks (notably a 5-week re-epithelialization time is not necessarily an improvement).

In 2005, Hunger et al³ demonstrated TNF-α’s release by KCs in the epidermis and by inflammatory cells in the dermis of a TEN patient. Twenty-four hours after the administration of infliximab 5 mg/kg in these patients, TNF-α was found to be below normal and epidermal detachment ceased.³ Wojtkietwicz et al¹³ demonstrated benefit following an infusion of infliximab 5 mg/kg in a patient whose disease continued to progress despite treatment with dexamethasone and 1.8 g/kg of IVIG.

Then 2 subsequent case series added further support for the efficacy of infliximab in the treatment of TEN. Patmanidis et al¹⁵ and Gaitanis et al¹⁶ reported similar results in 4 patients, each treated with infliximab 5 mg/kg immediately followed by initiation of high-dose IVIG (2 g/kg over 5 days). Zárate-Correa et al¹⁷ reported a 0% mortality rate and near-complete re-epithelialization after 5 to 14 days in 4 patients treated with a single 300-mg dose of infliximab.

However, the success of infliximab in the treatment of TEN has been countered by the pilot study by Paquet et al,¹⁸ which compared the efficacy of 150 mg/kg of N-acetylcysteine alone vs adding infliximab 5 mg/kg to treat 10 TEN patients. The study demonstrated no benefit at 48 hours in the group given infliximab, the time frame in which prior case reports touting infliximab’s benefit claimed the benefit was observed. Similarly, there was no effect on mortality for either treatment modality as assessed by illness auxiliary score.¹⁸

Evidence in support of the use of etanercept in the treatment of SJS/TEN is mounting, and some centers have begun to use it as the first-choice therapy for SJS/TEN. The first case was reported by Famularo et al,¹⁹ in which a patient with TEN was given 2 doses of etanercept 25 mg after failure to improve with prednisolone 1 mg/kg. The patient showed near-complete and rapid re-epithelization in 6 days before death due to disseminated intravascular coagulation 10 days after admission.¹⁹ Gubinelli et al²⁰ and Sadighha²¹ independently reported cases of TEN and TEN/acute generalized exanthematous pustulosis overlap treated with a total of 50 mg of etanercept, demonstrating rapid cessation of lesion progression. Didona et al²² found similar benefit using etanercept 50 mg to treat TEN secondary to rituximab after failure to improve with prednisone and cyclophosphamide. Treatment of TEN with etanercept in an HIV-positive patient also has been reported. Lee et al²³ described a patient who was administered 50-mg and 25-mg injections on days 3 and 5 of hospitalization, respectively, with re-epithelialization occurring by day 8. Finally, Owczarczyk-Saczonek et al²⁴ reported a case of SJS in a patient with a 4-year history of etanercept and sulfasalazine treatment of rheumatoid arthritis; sulfasalazine was stopped, but this patient was continued on etanercept until resolution of skin and mucosal symptoms. However, it is important to consider the possibility of publication bias among these cases selected for their positive outcomes.

Perhaps the most compelling literature regarding the use of etanercept for TEN was described in a case series by Paradisi et al.² This study included 10 patients with TEN, all of whom demonstrated complete re-epithelialization shortly after receiving etanercept 50 mg. Average SCORTEN was 3.6 with a range of 2 to 6. Eight patients in this study had severe comorbidities and all 10 patients survived, with a time to re-epithelialization ranging from 7 to 20 days.² Additionally, a randomized controlled trial showed that 38 etanercept-treated patients had improved mortality (P=.266) and re-epithelialization time (P=.01) compared to patients treated with intravenous methylprednisolone.²⁵Limitations to our study are similar to other reports of SJS/TEN and included the small number of cases and lack of randomization. Additionally, we do not have data available for all patients for time between onset of disease and treatment initiation. Because of these challenges, data presented in this case series is observational only. Additionally, the patients treated with etanercept alone had a slightly lower SCORTEN compared to the group that received IVIG or supportive care alone (2.1 and 2.4 respectively). However, the etanercept-only group actually had higher involvement of epidermal detachment (33%) compared to the non-etanercept group (23%).

Conclusion

Although treatment with etanercept lacks the support of a randomized controlled trial, similar to all other treatments currently used for SJS and TEN, preliminary reports highlight a benefit in disease progression and improvement in time to re-epithelialization. In particular, if etanercept 50 mg subcutaneously is given as monotherapy or is given early in the disease course (prior to other therapies being attempted and ideally within 6 hours of presentation), our data suggest an even greater trend toward improved mortality and decreased time to re-epithelialization. Additionally, our findings may suggest that in some patients, etanercept monotherapy is not an adequate intervention but the addition of IVIG may be helpful; however, the senior author (S.W.) notes anecdotally that in his experience with the patients treated at the University of California Los Angeles, the order of administration of combination therapies—etanercept followed by IVIG—was important in addition to the choice of therapy. These findings are promising enough to warrant a multicenter randomized controlled trial comparing the efficacy of etanercept to other more commonly used treatments for this spectrum of disease, including IVIG and/or cyclosporine. Based on the data presented in this case series, including the 13 patients who received etanercept and had a 0% mortality rate, etanercept may be viewed as a targeted therapeutic intervention for patients with SJS and TEN.

The prospect of facing a medical malpractice claim is a source of apprehension for physicians that affects physician behavior, including leading to defensive medicine.^1-3 Overall, annual defensive medicine costs have been estimated at $45.6 billion,⁴ and surveys of hospitalists indicate that 13.0% to 37.5% of hospitalist healthcare costs involve defensive medicine.^5,6 Despite the impact of malpractice concerns on hospitalist practice and the unprecedented growth of the field of hospital medicine, relatively few studies have examined the liability environment surrounding hospitalist practice.^7,8 The specific issue of malpractice claims rates faced by hospitalists has received even less attention in the medical literature.⁸

A better understanding of the contributing factors and other attributes of malpractice claims can help guide patient safety initiatives and inform hospitalists’ level of concern regarding liability. Although most medical errors do not result in a malpractice claim,^9,10 the majority of malpractice claims in which there is an indemnity payment involve medical injury due to clinician error.¹¹ Even malpractice claims that do not result in an indemnity payment represent opportunities to identify patient safety and risk management vulnerabilities.¹²

We used a national malpractice claims database to analyze the characteristics of claims made against hospitalists, including claims rates. In addition to claims rates, we also analyzed the other types of providers named in hospitalist claims given the importance of interdisciplinary collaboration to hospital medicine.^13,14 To provide context for understanding hospitalist liability data, we present data on other specialties. We also describe a model to predict whether hospitalist malpractice claims will close with an indemnity payment.

Data Sources and Elements

This analysis used a repository of malpractice claims maintained by CRICO, the captive malpractice insurer of the Harvard-affiliated medical institutions. This database, the Comparative Benchmarking System, aggregates malpractice claims from multiple other malpractice carriers (both commercial and captive). This database comprises approximately 31% of all malpractice claims in the United States (both paid and unpaid) according to a comparison with claims in the National Practitioner Data Bank (NPDB), which is a government-run database to which all paid malpractice claims against physicians must be reported.¹⁵ To compile this national database, trained nurse coders, who have access to the medical records and legal documents from each case, abstract information into coded fields, such as the type of allegation, contributing factors, and injury severity. The information in these coded fields is based on a taxonomy that was developed specifically for use in malpractice cases. The nurse coders participate in regular meetings in which cases are discussed to ensure that they are applying the coding and taxonomy uniformly.

Injury severity was based on a widely used scale developed for malpractice claims by the National Association of Insurance Commissioners.¹⁶ Low injury severity included emotional injury and temporary insignificant injury. Medium injury severity included temporary minor, temporary major, and permanent minor injury. High injury severity included permanent significant injury through death. Because this study used a database assembled for operational and patient safety purposes and was not human subjects research, institutional review board approval was not needed.

Study Cohort

Malpractice claims included formal lawsuits or written requests for compensation for negligent medical care. Hospitalist malpractice claims were defined as those in which the nurse coders determined that the internal medicine hospitalist service was the clinical service that was primarily responsible for the care of the patient at the time of the clinical event giving rise to the claim. Malpractice claims closed during the period 2009-2018 were included, regardless of whether an indemnity payment was made. Claims against select other medical specialties (nonhospitalist general internal medicine, internal medicine subspecialists, emergency medicine, neurosurgery, and psychiatry), defined by the specialty of the physician named in the claim, were included for comparison with the characteristics of the claims against hospitalists. Physician specialty was available for all physicians in the study cohort; however, given that malpractice cases were aggregated from multiple different insurers, full demographic information (eg, age and sex) was not available for all physicians.

Statistical Analysis

Malpractice claims rates were treated as Poisson rates and compared using a Z-test. Malpractice claims rates are expressed as claims per 100 physician-years. Each physician-year represents 1 year of coverage of one physician by the medical malpractice carrier whose data were used. Physician-years represent the duration of time physicians practiced during which they were insured by the malpractice carrier and, as such, could have been subject to a malpractice claim that would have been included in our data. Claims rates are based on the subset of the malpractice claims in the study for which the number of physician-years of coverage is available, representing 8.2% of hospitalist claims and 11.6% of all claims.

Comparisons of the percentages of cases closing with an indemnity payment, as well as the percentages of cases in different allegation type and clinical severity categories, were made using the Fisher exact test. Indemnity payment amounts were inflation-adjusted to 2018 dollars using the Consumer Price Index. Comparisons of indemnity payment amounts between physician specialties were carried out using the Wilcoxon rank sum test given that the distribution of the payment amounts appeared nonnormal; this was confirmed with the Shapiro-Wilk test. A multivariable logistic regression model was developed to predict the binary outcome of whether a hospitalist case would close with an indemnity payment (compared with no payment), based on the 1,216 hospitalist claims. The predictors used in this regression model were chosen a priori based on hypotheses about what factors drive the likelihood that a case closes with payment. Both the unadjusted and adjusted odds ratios for the predictors are presented. The adjusted model is adjusted for all the other predictors contained in the model. All reported P values are two-sided. The statistical analysis was carried out using JMP Pro version 15 (SAS Institute Inc) and Minitab version 19 (Minitab LLC).

We identified 1,216 hospital medicine malpractice claims from our database. Claims rates were calculated from the subset of our data for which physician-years were available—including 5,140 physician-years encompassing 100 claims, representing 8.2% of all hospitalist claims studied. An additional 18,644 malpractice claims from five other specialties—nonhospitalist general internal medicine, internal medicine subspecialists, emergency medicine, neurosurgery, and psychiatry—were analyzed to provide context for the hospitalist claims.

The malpractice claims rate for hospitalists was significantly higher than the rate for internal medicine subspecialists (1.95 vs 1.30 claims per 100 physician-years; P < .001), though they were not significantly different from the rate for nonhospitalist general internal medicine physicians (1.95 vs 1.92 claims per 100 physician-years; P = .93) (Table 1). Compared with emergency medicine physicians, with whom hospitalists are sometimes compared due to both specialties being defined by their site of practice and the absence of longitudinal patient relationships, hospitalists had a significantly lower claims rate (1.95 vs 4.07 claims per 100 physician-years; P < .001).

An assessment of the temporal trends in the claims rates, based on a comparison between the two halves of the study period (2014-2018 vs 2009-2013), showed that the claims rate for hospitalists was increasing, but at a rate that did not reach statistical significance (Table 1). In contrast, the claims rates for the five other specialties assessed decreased over time, and the decreases were significant for four of these five other specialties (internal medicine subspecialties, emergency medicine, neurosurgery, and psychiatry).

Multiple claims against a single physician were uncommon in our hospitalist malpractice claims data. Among the 100 claims that were used to calculate the claims rates, one physician was named in 2 claims, and all the other physicians were named in only a single claim. Among all of the 1,216 hospitalist malpractice claims we analyzed, there were eight physicians who were named in more than 1 claim, seven of whom were named in 2 claims, and one of whom was named in 3 claims.

The median indemnity payment for hospitalist claims was $231,454 (interquartile range [IQR], $100,000-$503,015), similar to the median indemnity payment for neurosurgery ($233,723; IQR, $85,292-$697,872), though significantly greater than the median indemnity payment for the other four specialties studied (Table 2). Among the hospitalist claims, 29.9% resulted in an indemnity payment, not significantly different from the rate for nonhospitalist general internal medicine, internal medicine subspecialties, or neurosurgery, but significantly lower than the rate for emergency medicine (33.8%; P = .011). No significant temporal trend in the percentage of hospitalist claims that resulted in an indemnity payment was observed.

We performed a multivariable logistic regression analysis to assess the effect of different factors on the likelihood of a hospitalist case closing with an indemnity payment, compared with no payment (Table 3). In the multivariable model, the presence of an error in clinical judgment had an adjusted odds ratio (AOR) of 5.01 (95% CI, 3.37-7.45; P < .001) for a claim closing with payment, the largest effect found. The presence of problems with communication (AOR, 1.89; 95% CI, 1.42-2.51; P < .001), the clinical environment (eg, weekend/holiday or clinical busyness; AOR, 1.70; 95% CI, 1.20-2.40; P = .0026), and documentation (AOR, 1.65; 95% CI, 1.18-2.31; P = .0038) were also positive predictors of claims closing with payment. Greater patient age (per decade) was a negative predictor of the likelihood of a claim closing with payment (AOR, 0.92; 95% CI, 0.86-0.998), though it was of borderline statistical significance (P = .044).