Polina V. Kukhareva, MS, MPH

Healthcare costs continue to increase and are estimated to be approximately $3.1 trillion per year in the United States.[1] Waste is a major contributor to this cost, accounting for an estimated $910 billion/year.[2] Laboratory tests are well documented to contribute to healthcare waste, with an estimated 30% to 50% of tests for hospitalized patients being unnecessary.[3, 4, 5] This issue has been highlighted by the American Board of Internal Medicine Foundation's Choosing Wisely campaign as an area to reduce waste.[6] Evaluating this concern locally, a University Health Systems Consortium 2011 analysis indicated that the University of Utah general internal medicine hospitalist service had a higher average direct lab cost per discharge compared to top performers, indicating an opportunity for improvement.

Multiple interventions have been described in the literature to address excessive laboratory utilization, including physician education, audit and feedback, cost information display, and administrative rules restricting certain types of ordering.[7, 8, 9, 10, 11] Despite these interventions, barriers remain common and not all interventions are sustained. For example, interventions focused mainly on education see a small improvement initially that is not sustained.[4, 12, 13] Additionally, although most studies focus on individual interventions, those that target multiple factors have been found to be more successful at producing and sustaining change.[14] Therefore, the opportunity existed to incorporate multiple etiologies into a single intervention and apply a checklist to laboratory ordering to see if combined modalities could be effective at reducing laboratory costs in a sustainable manner.

In addition to cost, there is potential patient harm resulting from unnecessary laboratory testing. For prolonged hospitalizations, anemia is a well‐recognized side effect of phlebotomy,[15, 16] and a recent evaluation of cardiac surgery patients found an average cumulative blood loss due to phlebotomy of 454 mL/hospital stay.[17] The sheer number of tests ordered can lead to false positive tests that result in additional testing and monitoring. Furthermore, patients subjected to laboratory blood draws are often awakened early in the morning, which is unpleasant and could adversely affect the patient experience.

Recognizing laboratory cost as a problem, the University of Utah general internal medicine hospitalist service implemented a multifaceted quality‐improvement initiative with a goal to reduce laboratory testing. At the time of this project, University of Utah Health Care (UUHC) developed a Value Driven Outcomes (VDO) tool to give direct data related to costs of care, including the actual cost paid by the hospital to the university‐owned laboratory vendor (ARUP Laboratories, Salt Lake City, UT) for testing.[18] The hospitalist group incorporated VDO into the initiative for routine cost feedback. This study evaluates the impact of this intervention on laboratory costs.

METHODS

RESULTS

We analyzed 48,327 inpatient visits that met inclusion criteria. We excluded 15,659 obstetrics, rehabilitation, and psychiatry visits. Seven hundred seventy‐two (2.4%) of the remaining visits were excluded due to missing data. A total of 31,896 inpatient visits by 22,545 patients were included in the analysis. There were 10,136 visits before the intervention and 21,760 visits after. Characteristics of the study groups for the full study timeframe (July 1, 2012April 30, 2014) are summarized in Table 1.

During the study period, there were 25,586 visits in the control group and 6310 visits in the intervention group. Patients in the intervention group were on average older than patients in the control group. There were more female patients in the intervention group. Mean CCI was 4.2 in the intervention group and 3.6 in the control group. The intervention group had lower LOS and 30‐day readmissions than the control group.

Descriptive statistics and simple comparisons of covariates and outcomes before and after the intervention are shown in Table 2. Age and gender distributions remained unchanged in both groups. CCI increased in the control group by 0.24 (P < 0.001) and remained unchanged in the intervention group. In the intervention group, lab cost per day was reduced from $138 before the intervention to $123 after the intervention (P < 0.001). In contrast, among control patients, cost per day increased nonsignificantly from $130 preintervention to $132 postintervention (P = 0.37). Number of tests per day significantly decreased for all specific tests in the intervention group. Readmission rates decreased significantly from 14% to 11% in the intervention group (P = 0.01). LOS remained constant in both groups.

ITS analysis results are shown in Table 3. After the intervention, the difference in monthly means between the 2 groups dropped by $16 for cost per day (P = 0.034) and by $128 for cost per visit (P = 0.02). The decreased cost in the intervention group amounts to approximately $251,427 (95% confidence interval [CI]: $20,370‐$482,484) savings over the first year. If the intervention was rolled out for the control group and had a similar impact, it could have led to an additional cost savings of $1,321,669 (95% CI: 107,081‐2,536,256). Moreover, the number of basic metabolic panel, comprehensive metabolic panel, and complete blood count test per day were reduced significantly more in the intervention group compared to the control group (<0.001, 0.004, and <0.001).

Figure 1 shows a graphical representation of the biweekly means for the 2 primary outcomeslab cost per day and lab cost per visit. Figure 2 shows all other outcomes. To the right of each figure, P values are provided for the b₂ coefficients from Table 3.

Figure 1

Figure 2

DISCUSSION

Through a multifaceted quality‐improvement initiative, the UUHC hospitalist group was able to reduce lab cost per day and per visit as well as commonly ordered routine labs as compared to an institutional control group. A multifaceted approach was selected given the literature supporting this approach as the most likely method to sustain improvement.[14] At the same time, the use of a multifaceted intervention makes it difficult to rigorously determine the relative impact of different components of the intervention. In discussing this issue, however, the hospitalist group felt that the driving factors for change were those related to process change, specifically, the use of a standardized rounding checklist to discuss lab testing and the routine review of lab costs at group meetings. The ultimate goal was to change the culture of routine test ordering into a thoughtful process of needed tests and thereby reduce costs. Prior to this intervention, the least experienced person on this team (the intern) ordered any test he or she wanted, usually without discussion. The intervention focused on this issue through standardized supervision and explicit discussion of laboratory tests. Importantly, although improvements from education initiatives typically decrease over time, the incorporation of process change in this intervention was felt to likely contribute to the sustained reduction seen at 15 months. Although use of the rounding checklist added another step to daily rounds, the routine cost feedback, including comparisons to peers, helped encourage use of the checklist. Thus, we feel that routine feedback was essential to sustaining the intervention and its impact.

Inappropriate and unnecessary testing has been recognized for decades, and multiple interventions have been attempted, including a recent article that demonstrated a 10% reduction in common laboratory ordering through an initiative mainly focused on education and ordering feedback.[25] Despite reported success of several interventions, none have combined multiple interventions and explicitly required discussion of laboratory tests on rounds. For example, although the UUHC intervention used Attali et al.[21] and Barie and Hydo's[20] work to develop the intervention, neither of these studies described how laboratory testing was discussed with the attending or supervising resident. The UUHC intervention thus builds on the current literature by combining other successful modalities with explicit discussion of laboratory testing via a rounding checklist and feedback with the novel VDO tool to reduce laboratory costs. A major strength of this intervention is the relatively low cost and the generalizability of implementing rounding checklists. Initial support from the hospital was needed to provide accurate VDO information to the hospitalist group. However, ongoing costs were minimal and related to any additional time spent during rounds to discuss laboratory tests. Thus, we feel that this intervention is feasible for wide replication.

Another strength of the study is the use of the VDO tool to measure actual costs. Whereas previous studies have relied on estimated costs with extrapolation to potential cost savings, this study used direct costs to the institution as a more accurate marker of cost savings. Additionally, most studies on lab utilization have used a before/after analysis without a control group. The presence of a control group for this analysis is important to help assess for institutional trends that may not be reflected in a before/after intervention. The reduction in cost in the intervention group despite a trend toward increased cost in the institutional control group supports the impact of this intervention.

Limitations of this study include that it was a single‐center, controlled ITS study and not a randomized controlled trial. Related to this limitation, the control group reflected a different patient population compared to the intervention group, with a longer LOS, lower CCI, and inclusion of nonmedical patients. However, these differences were relatively stable before and after the intervention. Also, ITS is considered one of the most robust research designs outside of randomized controlled trials, and it accounts for baseline differences in both levels and trends.[24] Nevertheless, it remains possible that secular trends existed that we did not capture and that affected the 2 populations differently.

A further limitation is that the baseline period was only 7 months and the intervention was 15 months. As the 7 months started in July, this could have reflected the time when interns were least experienced with ordering. Unfortunately, we did not have VDO availability for a full year prior to the intervention. We believe that any major effect due to this shortened baseline period should have been seen in the control group as well, and therefore accounted for in the analysis. Additionally, it is possible that there was spillover of the intervention to the control group, as internal medicine residents rotated throughout the hospital to other medical services (pulmonary, cardiology, hematology, and oncology). However, any effect of their rotation should have been to lower the control lab cost, thus making differences less profound.

CONCLUSIONS

A multifaceted approach to laboratory reduction through education, process change, cost feedback, and financial incentive resulted in a significant reduction in laboratory cost per day, laboratory cost per visit, and the ordering of common laboratory tests at a major academic medical center.

Healthcare costs continue to increase and are estimated to be approximately $3.1 trillion per year in the United States.[1] Waste is a major contributor to this cost, accounting for an estimated $910 billion/year.[2] Laboratory tests are well documented to contribute to healthcare waste, with an estimated 30% to 50% of tests for hospitalized patients being unnecessary.[3, 4, 5] This issue has been highlighted by the American Board of Internal Medicine Foundation's Choosing Wisely campaign as an area to reduce waste.[6] Evaluating this concern locally, a University Health Systems Consortium 2011 analysis indicated that the University of Utah general internal medicine hospitalist service had a higher average direct lab cost per discharge compared to top performers, indicating an opportunity for improvement.

Multiple interventions have been described in the literature to address excessive laboratory utilization, including physician education, audit and feedback, cost information display, and administrative rules restricting certain types of ordering.[7, 8, 9, 10, 11] Despite these interventions, barriers remain common and not all interventions are sustained. For example, interventions focused mainly on education see a small improvement initially that is not sustained.[4, 12, 13] Additionally, although most studies focus on individual interventions, those that target multiple factors have been found to be more successful at producing and sustaining change.[14] Therefore, the opportunity existed to incorporate multiple etiologies into a single intervention and apply a checklist to laboratory ordering to see if combined modalities could be effective at reducing laboratory costs in a sustainable manner.

In addition to cost, there is potential patient harm resulting from unnecessary laboratory testing. For prolonged hospitalizations, anemia is a well‐recognized side effect of phlebotomy,[15, 16] and a recent evaluation of cardiac surgery patients found an average cumulative blood loss due to phlebotomy of 454 mL/hospital stay.[17] The sheer number of tests ordered can lead to false positive tests that result in additional testing and monitoring. Furthermore, patients subjected to laboratory blood draws are often awakened early in the morning, which is unpleasant and could adversely affect the patient experience.

Recognizing laboratory cost as a problem, the University of Utah general internal medicine hospitalist service implemented a multifaceted quality‐improvement initiative with a goal to reduce laboratory testing. At the time of this project, University of Utah Health Care (UUHC) developed a Value Driven Outcomes (VDO) tool to give direct data related to costs of care, including the actual cost paid by the hospital to the university‐owned laboratory vendor (ARUP Laboratories, Salt Lake City, UT) for testing.[18] The hospitalist group incorporated VDO into the initiative for routine cost feedback. This study evaluates the impact of this intervention on laboratory costs.

METHODS

RESULTS

We analyzed 48,327 inpatient visits that met inclusion criteria. We excluded 15,659 obstetrics, rehabilitation, and psychiatry visits. Seven hundred seventy‐two (2.4%) of the remaining visits were excluded due to missing data. A total of 31,896 inpatient visits by 22,545 patients were included in the analysis. There were 10,136 visits before the intervention and 21,760 visits after. Characteristics of the study groups for the full study timeframe (July 1, 2012April 30, 2014) are summarized in Table 1.

During the study period, there were 25,586 visits in the control group and 6310 visits in the intervention group. Patients in the intervention group were on average older than patients in the control group. There were more female patients in the intervention group. Mean CCI was 4.2 in the intervention group and 3.6 in the control group. The intervention group had lower LOS and 30‐day readmissions than the control group.

Descriptive statistics and simple comparisons of covariates and outcomes before and after the intervention are shown in Table 2. Age and gender distributions remained unchanged in both groups. CCI increased in the control group by 0.24 (P < 0.001) and remained unchanged in the intervention group. In the intervention group, lab cost per day was reduced from $138 before the intervention to $123 after the intervention (P < 0.001). In contrast, among control patients, cost per day increased nonsignificantly from $130 preintervention to $132 postintervention (P = 0.37). Number of tests per day significantly decreased for all specific tests in the intervention group. Readmission rates decreased significantly from 14% to 11% in the intervention group (P = 0.01). LOS remained constant in both groups.

ITS analysis results are shown in Table 3. After the intervention, the difference in monthly means between the 2 groups dropped by $16 for cost per day (P = 0.034) and by $128 for cost per visit (P = 0.02). The decreased cost in the intervention group amounts to approximately $251,427 (95% confidence interval [CI]: $20,370‐$482,484) savings over the first year. If the intervention was rolled out for the control group and had a similar impact, it could have led to an additional cost savings of $1,321,669 (95% CI: 107,081‐2,536,256). Moreover, the number of basic metabolic panel, comprehensive metabolic panel, and complete blood count test per day were reduced significantly more in the intervention group compared to the control group (<0.001, 0.004, and <0.001).

Figure 1 shows a graphical representation of the biweekly means for the 2 primary outcomeslab cost per day and lab cost per visit. Figure 2 shows all other outcomes. To the right of each figure, P values are provided for the b₂ coefficients from Table 3.

Figure 1

Figure 2

DISCUSSION

Through a multifaceted quality‐improvement initiative, the UUHC hospitalist group was able to reduce lab cost per day and per visit as well as commonly ordered routine labs as compared to an institutional control group. A multifaceted approach was selected given the literature supporting this approach as the most likely method to sustain improvement.[14] At the same time, the use of a multifaceted intervention makes it difficult to rigorously determine the relative impact of different components of the intervention. In discussing this issue, however, the hospitalist group felt that the driving factors for change were those related to process change, specifically, the use of a standardized rounding checklist to discuss lab testing and the routine review of lab costs at group meetings. The ultimate goal was to change the culture of routine test ordering into a thoughtful process of needed tests and thereby reduce costs. Prior to this intervention, the least experienced person on this team (the intern) ordered any test he or she wanted, usually without discussion. The intervention focused on this issue through standardized supervision and explicit discussion of laboratory tests. Importantly, although improvements from education initiatives typically decrease over time, the incorporation of process change in this intervention was felt to likely contribute to the sustained reduction seen at 15 months. Although use of the rounding checklist added another step to daily rounds, the routine cost feedback, including comparisons to peers, helped encourage use of the checklist. Thus, we feel that routine feedback was essential to sustaining the intervention and its impact.

Inappropriate and unnecessary testing has been recognized for decades, and multiple interventions have been attempted, including a recent article that demonstrated a 10% reduction in common laboratory ordering through an initiative mainly focused on education and ordering feedback.[25] Despite reported success of several interventions, none have combined multiple interventions and explicitly required discussion of laboratory tests on rounds. For example, although the UUHC intervention used Attali et al.[21] and Barie and Hydo's[20] work to develop the intervention, neither of these studies described how laboratory testing was discussed with the attending or supervising resident. The UUHC intervention thus builds on the current literature by combining other successful modalities with explicit discussion of laboratory testing via a rounding checklist and feedback with the novel VDO tool to reduce laboratory costs. A major strength of this intervention is the relatively low cost and the generalizability of implementing rounding checklists. Initial support from the hospital was needed to provide accurate VDO information to the hospitalist group. However, ongoing costs were minimal and related to any additional time spent during rounds to discuss laboratory tests. Thus, we feel that this intervention is feasible for wide replication.

Another strength of the study is the use of the VDO tool to measure actual costs. Whereas previous studies have relied on estimated costs with extrapolation to potential cost savings, this study used direct costs to the institution as a more accurate marker of cost savings. Additionally, most studies on lab utilization have used a before/after analysis without a control group. The presence of a control group for this analysis is important to help assess for institutional trends that may not be reflected in a before/after intervention. The reduction in cost in the intervention group despite a trend toward increased cost in the institutional control group supports the impact of this intervention.

Limitations of this study include that it was a single‐center, controlled ITS study and not a randomized controlled trial. Related to this limitation, the control group reflected a different patient population compared to the intervention group, with a longer LOS, lower CCI, and inclusion of nonmedical patients. However, these differences were relatively stable before and after the intervention. Also, ITS is considered one of the most robust research designs outside of randomized controlled trials, and it accounts for baseline differences in both levels and trends.[24] Nevertheless, it remains possible that secular trends existed that we did not capture and that affected the 2 populations differently.

A further limitation is that the baseline period was only 7 months and the intervention was 15 months. As the 7 months started in July, this could have reflected the time when interns were least experienced with ordering. Unfortunately, we did not have VDO availability for a full year prior to the intervention. We believe that any major effect due to this shortened baseline period should have been seen in the control group as well, and therefore accounted for in the analysis. Additionally, it is possible that there was spillover of the intervention to the control group, as internal medicine residents rotated throughout the hospital to other medical services (pulmonary, cardiology, hematology, and oncology). However, any effect of their rotation should have been to lower the control lab cost, thus making differences less profound.

CONCLUSIONS

A multifaceted approach to laboratory reduction through education, process change, cost feedback, and financial incentive resulted in a significant reduction in laboratory cost per day, laboratory cost per visit, and the ordering of common laboratory tests at a major academic medical center.

Healthcare costs continue to increase and are estimated to be approximately $3.1 trillion per year in the United States.[1] Waste is a major contributor to this cost, accounting for an estimated $910 billion/year.[2] Laboratory tests are well documented to contribute to healthcare waste, with an estimated 30% to 50% of tests for hospitalized patients being unnecessary.[3, 4, 5] This issue has been highlighted by the American Board of Internal Medicine Foundation's Choosing Wisely campaign as an area to reduce waste.[6] Evaluating this concern locally, a University Health Systems Consortium 2011 analysis indicated that the University of Utah general internal medicine hospitalist service had a higher average direct lab cost per discharge compared to top performers, indicating an opportunity for improvement.

Multiple interventions have been described in the literature to address excessive laboratory utilization, including physician education, audit and feedback, cost information display, and administrative rules restricting certain types of ordering.[7, 8, 9, 10, 11] Despite these interventions, barriers remain common and not all interventions are sustained. For example, interventions focused mainly on education see a small improvement initially that is not sustained.[4, 12, 13] Additionally, although most studies focus on individual interventions, those that target multiple factors have been found to be more successful at producing and sustaining change.[14] Therefore, the opportunity existed to incorporate multiple etiologies into a single intervention and apply a checklist to laboratory ordering to see if combined modalities could be effective at reducing laboratory costs in a sustainable manner.

In addition to cost, there is potential patient harm resulting from unnecessary laboratory testing. For prolonged hospitalizations, anemia is a well‐recognized side effect of phlebotomy,[15, 16] and a recent evaluation of cardiac surgery patients found an average cumulative blood loss due to phlebotomy of 454 mL/hospital stay.[17] The sheer number of tests ordered can lead to false positive tests that result in additional testing and monitoring. Furthermore, patients subjected to laboratory blood draws are often awakened early in the morning, which is unpleasant and could adversely affect the patient experience.

Recognizing laboratory cost as a problem, the University of Utah general internal medicine hospitalist service implemented a multifaceted quality‐improvement initiative with a goal to reduce laboratory testing. At the time of this project, University of Utah Health Care (UUHC) developed a Value Driven Outcomes (VDO) tool to give direct data related to costs of care, including the actual cost paid by the hospital to the university‐owned laboratory vendor (ARUP Laboratories, Salt Lake City, UT) for testing.[18] The hospitalist group incorporated VDO into the initiative for routine cost feedback. This study evaluates the impact of this intervention on laboratory costs.

METHODS

RESULTS

We analyzed 48,327 inpatient visits that met inclusion criteria. We excluded 15,659 obstetrics, rehabilitation, and psychiatry visits. Seven hundred seventy‐two (2.4%) of the remaining visits were excluded due to missing data. A total of 31,896 inpatient visits by 22,545 patients were included in the analysis. There were 10,136 visits before the intervention and 21,760 visits after. Characteristics of the study groups for the full study timeframe (July 1, 2012April 30, 2014) are summarized in Table 1.

During the study period, there were 25,586 visits in the control group and 6310 visits in the intervention group. Patients in the intervention group were on average older than patients in the control group. There were more female patients in the intervention group. Mean CCI was 4.2 in the intervention group and 3.6 in the control group. The intervention group had lower LOS and 30‐day readmissions than the control group.

Descriptive statistics and simple comparisons of covariates and outcomes before and after the intervention are shown in Table 2. Age and gender distributions remained unchanged in both groups. CCI increased in the control group by 0.24 (P < 0.001) and remained unchanged in the intervention group. In the intervention group, lab cost per day was reduced from $138 before the intervention to $123 after the intervention (P < 0.001). In contrast, among control patients, cost per day increased nonsignificantly from $130 preintervention to $132 postintervention (P = 0.37). Number of tests per day significantly decreased for all specific tests in the intervention group. Readmission rates decreased significantly from 14% to 11% in the intervention group (P = 0.01). LOS remained constant in both groups.

ITS analysis results are shown in Table 3. After the intervention, the difference in monthly means between the 2 groups dropped by $16 for cost per day (P = 0.034) and by $128 for cost per visit (P = 0.02). The decreased cost in the intervention group amounts to approximately $251,427 (95% confidence interval [CI]: $20,370‐$482,484) savings over the first year. If the intervention was rolled out for the control group and had a similar impact, it could have led to an additional cost savings of $1,321,669 (95% CI: 107,081‐2,536,256). Moreover, the number of basic metabolic panel, comprehensive metabolic panel, and complete blood count test per day were reduced significantly more in the intervention group compared to the control group (<0.001, 0.004, and <0.001).

Figure 1 shows a graphical representation of the biweekly means for the 2 primary outcomeslab cost per day and lab cost per visit. Figure 2 shows all other outcomes. To the right of each figure, P values are provided for the b₂ coefficients from Table 3.

Figure 1

Figure 2

DISCUSSION

Through a multifaceted quality‐improvement initiative, the UUHC hospitalist group was able to reduce lab cost per day and per visit as well as commonly ordered routine labs as compared to an institutional control group. A multifaceted approach was selected given the literature supporting this approach as the most likely method to sustain improvement.[14] At the same time, the use of a multifaceted intervention makes it difficult to rigorously determine the relative impact of different components of the intervention. In discussing this issue, however, the hospitalist group felt that the driving factors for change were those related to process change, specifically, the use of a standardized rounding checklist to discuss lab testing and the routine review of lab costs at group meetings. The ultimate goal was to change the culture of routine test ordering into a thoughtful process of needed tests and thereby reduce costs. Prior to this intervention, the least experienced person on this team (the intern) ordered any test he or she wanted, usually without discussion. The intervention focused on this issue through standardized supervision and explicit discussion of laboratory tests. Importantly, although improvements from education initiatives typically decrease over time, the incorporation of process change in this intervention was felt to likely contribute to the sustained reduction seen at 15 months. Although use of the rounding checklist added another step to daily rounds, the routine cost feedback, including comparisons to peers, helped encourage use of the checklist. Thus, we feel that routine feedback was essential to sustaining the intervention and its impact.

Inappropriate and unnecessary testing has been recognized for decades, and multiple interventions have been attempted, including a recent article that demonstrated a 10% reduction in common laboratory ordering through an initiative mainly focused on education and ordering feedback.[25] Despite reported success of several interventions, none have combined multiple interventions and explicitly required discussion of laboratory tests on rounds. For example, although the UUHC intervention used Attali et al.[21] and Barie and Hydo's[20] work to develop the intervention, neither of these studies described how laboratory testing was discussed with the attending or supervising resident. The UUHC intervention thus builds on the current literature by combining other successful modalities with explicit discussion of laboratory testing via a rounding checklist and feedback with the novel VDO tool to reduce laboratory costs. A major strength of this intervention is the relatively low cost and the generalizability of implementing rounding checklists. Initial support from the hospital was needed to provide accurate VDO information to the hospitalist group. However, ongoing costs were minimal and related to any additional time spent during rounds to discuss laboratory tests. Thus, we feel that this intervention is feasible for wide replication.

Another strength of the study is the use of the VDO tool to measure actual costs. Whereas previous studies have relied on estimated costs with extrapolation to potential cost savings, this study used direct costs to the institution as a more accurate marker of cost savings. Additionally, most studies on lab utilization have used a before/after analysis without a control group. The presence of a control group for this analysis is important to help assess for institutional trends that may not be reflected in a before/after intervention. The reduction in cost in the intervention group despite a trend toward increased cost in the institutional control group supports the impact of this intervention.

Limitations of this study include that it was a single‐center, controlled ITS study and not a randomized controlled trial. Related to this limitation, the control group reflected a different patient population compared to the intervention group, with a longer LOS, lower CCI, and inclusion of nonmedical patients. However, these differences were relatively stable before and after the intervention. Also, ITS is considered one of the most robust research designs outside of randomized controlled trials, and it accounts for baseline differences in both levels and trends.[24] Nevertheless, it remains possible that secular trends existed that we did not capture and that affected the 2 populations differently.

A further limitation is that the baseline period was only 7 months and the intervention was 15 months. As the 7 months started in July, this could have reflected the time when interns were least experienced with ordering. Unfortunately, we did not have VDO availability for a full year prior to the intervention. We believe that any major effect due to this shortened baseline period should have been seen in the control group as well, and therefore accounted for in the analysis. Additionally, it is possible that there was spillover of the intervention to the control group, as internal medicine residents rotated throughout the hospital to other medical services (pulmonary, cardiology, hematology, and oncology). However, any effect of their rotation should have been to lower the control lab cost, thus making differences less profound.

CONCLUSIONS

A multifaceted approach to laboratory reduction through education, process change, cost feedback, and financial incentive resulted in a significant reduction in laboratory cost per day, laboratory cost per visit, and the ordering of common laboratory tests at a major academic medical center.

Cellulitis is a common infection causing inflammation of the skin and subcutaneous tissues. Cellulitis has been attributed to gram‐positive organisms through historical evaluations including fine‐needle aspirates and punch biopsies of the infected tissue.[1] Neither of these diagnostic tests is currently used due to their invasiveness, poor diagnostic yield, and availability. Similarly, readily available tests such as blood cultures provide an etiology <5% of the time[1] and are not cost‐effective for most patients for diagnosing cellulitis.[2] In addition, the prevalence of methicillin‐resistant Staphylococcus aureus (MRSA) has steadily increased, complicating decisions about antibiotic selection.[3] The result of this uncertainty is a large variation in practice with respect to antibiotic and imaging selection for patients with a diagnosis of cellulitis.

University of Utah Health Care (UUHC) performed benchmarking for the management of cellulitis using the University HealthSystem Consortium (UHC) database and associated CareFx analytics tool. Benchmarking demonstrated that UUHC had a greater percentage of broad‐spectrum antibiotic use (defined as vancomycin, piperacillin/tazobactam, or carbapenems) than the top 5 performing UHC facilities for International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) diagnoses of cellulitis (vancomycin 83% vs 58% and carbapenem or piperacillin/tazobactam 44% vs 16%). Advanced imaging (computed tomography [CT] or magnetic resonance imaging [MRI]) for the diagnosis of cellulitis was also found to be an opportunity for improvement (CT 27% vs 20% and MRI 8% vs 5%). The hospitalist group (most patients admitted with cellulitis were on this service) believed these data reflected current practice, as there was no standard of treatment for cellulitis despite an active order set. Therefore, cellulitis was considered an opportunity to improve value to our patients. A standardized clinical care pathway was created, as such pathways have demonstrated a reduction in variation in practice and improved efficiency and effectiveness of care for multiple disease states including cellulitis.[4, 5] We hypothesized that implementation of an evidence‐based care pathway would decrease broad‐spectrum antibiotic use, cost, and use of advanced imaging without having any adverse effects on clinical outcomes such as length of stay (LOS) or readmission.

METHODS

Intervention

Initial steps involved the formation of a multidisciplinary team including key stakeholders from the hospitalist group, infectious diseases, the emergency department (ED), and nursing. This multidisciplinary team was charged with developing a clinical care pathway appropriate for local implementation. National guidance for the care pathway was mainly obtained from the Infectious Disease Society of America (IDSA) guidelines on skin and soft tissue infections (SSTIs)[6] and MRSA.[7] Specific attention was paid to recommendations on blood cultures (only when systemically ill), imaging (rarely needed), antibiotic selection (rarely gram‐negative coverage and consideration of MRSA coverage), and patient‐care principles that are often overlooked (elevation of the affected extremity). A distinction of purulent versus nonpurulent cellulitis was adopted based on the guidelines and a prospective evaluation of the care of patients with nonpurulent cellulitis.[8] The 2014 IDSA update on SSTIs incorporates this distinction more clearly in hopes of determining staphylococcal versus streptococcal infections.[9] After multiple iterations, an agreed‐upon care pathway was created that excluded patients with neutropenia, osteomyelitis, diabetic foot ulcerations; hand, perineal, periorbital, or surgical site infections; and human or animal bites (Figure 1). After the care pathway was determined, interventions were performed to implement this change.

Figure 1

Education of all providers involved included discussion of cellulitis as a disease process, presentation of benchmarking data, dissemination of the care pathway to hospitalist and ED physicians, teaching conferences for internal medicine residents and ED residents, and reinforcement of these concepts at the beginning of resident rotations.

Incorporation of the care pathway into the existing electronic order sets for cellulitis care in the inpatient and ED settings, with links to the care pathway, links to excluded disease processes (eg, hand cellulitis), preselection of commonly needed items (eg, elevate leg), and recommendations for antibiotic selection based on categories of purulent or nonpurulent cellulitis. The electronic health record (EHR) did not allow for forced order set usage, so the order set required selection by the admitting physician if indicated. Additionally, an embedded 48‐hour order set could be accessed at any time by the ordering physician and included vancomycin dosing. Specific changes to the preexisting order set included the development of sections for purulent and nonpurulent cellulitis as well as recommended antibiotics. Piperacillin/tazobactam and nafcillin were both removed and vancomycin was limited to the purulent subheading. Additionally, elevation of the extremity was preselected, and orderables for imaging (chest x‐ray and duplex ultrasound), antiulcer prophylaxis, telemetry, and electrocardiograph were all removed.

Audit and feedback of cases of cellulitis and broad‐spectrum antibiotic usage was performed by a senior hospitalist.

RESULTS

Order Set Use Groups Analysis

Descriptive statistics and simple comparison before/after the intervention for the 2 study groups are shown in the last 6 columns of Table 1. Among patients for whom order sets were used, broad‐spectrum antibiotic usage significantly decreased from 78% before the intervention to 49% after the intervention (P < 0.001). In contrast, among patients for whom order sets were not used, broad‐spectrum antibiotic usage remained relatively constant71% before the intervention versus 67% after the intervention (P = 0.50). Figure 2 shows semiannual changes in the prescription of broad‐spectrum antibiotics. There is a noticeable drop after the intervention among patients for whom order sets were used.

Figure 2

Analysis of the interaction between time period and order set usage is provided in Table 3. After the intervention, patients for whom the order sets were used had greater improvement in broad‐spectrum antibiotic selection (75% decrease, P < 0.001) and LOS (25% decrease, P = 0.041) than patients for whom order sets were not used. Pharmacy costs also decreased by 13% more among patients for whom the order sets were used, although the interaction was not statistically significant (P = 0.074). Laboratory costs decreased in both groups, but order set use did not demonstrate an interaction (P = 0.5). Similar results were found for the subgroups of admitted patients and patients seen in the EDOU.

Table 3.Differential Impact of the Intervention on Process Metrics, Clinical, and Cost Outcomes in Two Order Set Use Levels

Logistic Regression
Outcome Variables	Selected Predictor Variables	Odds*	Percent Change	P
Gamma Regression
Outcome Variables	Selected Predictor Variables	Fold Change*	Percent Change	P
* Exponentiation of the parameter for the variable represents odds for categorical variables and fold change in amount for continuous variables. Minus sign represents decrease in percent change in odds or fold change. P values are based on generalized linear models including gender, age at admission, Charlson Comorbidity Index, hospitalization status, order set use, time period, and interaction term between time period and order set use as predictor variables.
Broad spectrum antibiotics	Time period	0.84 (0.50, 1.40)	16% (50% to 40%)	0.50
	Time periodorder set	0.25 (0.12, 0.52)	75% (88% to 48%)	<0.001
MRI done	Time period	1.04 (0.49, 2.20)	4% (51% to 120%)	0.92
	Time periodorder set	0.44 (0.14, 1.38)	56% (86% to 38%)	0.16
CT done	Time period	0.94 (0.55, 1.60)	6% (45% to 60%)	0.81
	Time periodorder set	0.96 (0.44, 2.12)	4% (56% to 112%)	0.93
30‐day readmission	Time period	0.91 (0.33, 2.53)	9% (67% to 153%)	0.86
	Time periodorder set	0.88 (0.20, 3.93)	12% (80% to 293%)	0.87
Clinical length of stay	Time period	1.04 (0.95, 1.14)	4% (5% to 14%)	0.41
	Time periodorder set	0.87 (0.77, 0.99)	13% (23% to 1%)	0.041
Pharmacy cost ($)	Time period	0.88 (0.70, 1.12)	12% (30% to 12%)	0.31
	Time periodorder set	0.75 (0.54, 1.03)	25% (46% to 3%)	0.074
Lab cost ($)	Time period	0.53 (0.42, 0.66)	47% (58% to 34%)	<0.001
	Time periodorder set	1.11 (0.82, 1.50)	11% (18% to 50%)	0.50
Imaging cost ($)	Time period	1.00 (0.71, 1.40)	0% (29% to 40%)	0.98
	Time periodorder set	0.82 (0.51, 1.30)	18% (49% to 30%)	0.39
Facility cost ($)	Time period	0.92 (0.80, 1.05)	8% (20% to 5%)	0.22
	Time periodorder set	0.90 (0.75, 1.09)	10% (25% to 9%)	0.29

Audit and feedback was initially performed for cases of cellulitis using broad‐spectrum antibiotics. However, given the complexity of cellulitis as a disease process and the frequency of broad‐spectrum antibiotic usage, in all cases of review, it was deemed reasonable to use broad‐spectrum antibiotics. Therefore, the audit was not continued.

DISCUSSION

Care pathways have demonstrated improvement across multiple different disease states including cellulitis.[4, 5] They have been noted to reduce variation in practice and improve physician agreement about treatment options.[4] The best method for implementation is not clearly understood,[12] and there remains concern about maintaining flexibility for patient care.[13] Additionally, although implementation of pathways is often well described, evaluations of the processes are noted to frequently be weak.[12] UUHC felt that the literature supported implementing a care pathway for the diagnosis of cellulitis, but that a thorough evaluation was also needed to understand any resulting benefits or harms. Through this study, we found that the implementation of this pathway resulted in a significant decrease in broad‐spectrum antibiotic use, pharmacy costs, and total facility costs. There was also a trend to decrease in imaging cost, and there were no adverse effects on LOS or 30‐day readmissions. Our findings demonstrate that care‐pathway implementation accompanied by education, pathway‐compliant electronic order sets, and audit and feedback can help drive improvements in quality while reducing costs. This finding furthers the evidence supporting standard work through the creation of clinical care pathways for cellulitis as an effective intervention.[4] Additionally, although not measured in this study, reduction of antibiotic use is supported as a measure to help reduce Clostridium difficile infections, a further potential benefit.[14]

This study has several important strengths. First, we included accurate cost analyses using the VDO tool. Given the increasing importance of improving care value, we feel the inclusion of such cost analysis is an increasingly important aspect of health service intervention evaluations. Second, we used a formal benchmarking approach to identify a priority care improvement area and to monitor changes in practice following the rollout of the intervention. We feel this approach provides a useful example on how to systematically improve care quality and value in a broader health system context. Third, we evaluated not order set implementation per se, but rather changing an existing order set. Because studies in this area generally focus on initial order set implementation, our study contributes insights on what can be expected through modifications of existing order sets based on care pathways. Fourth, the analysis accounted for a variety of variables including the CCI. Of interest, our study found that the intervention group (patients for whom order sets were used) had a lower CCI, confirming Allen et al.'s findings that diseases with predictable trajectories are the most likely to benefit from care pathways.[4] As a final strength, the narrative‐based order set intervention was relatively simple, and the inclusion criteria were broad, making the process generalizable.

Limitations of this study include that it was a single center pre‐/postintervention study and not a randomized controlled trial. Related to this limitation, the control group for which order sets were not used reflected a different patient population compared to the intervention group for which order sets were used. Specifically, it was more common for order sets to be used in the EDOU than upon admission, resulting in the order set group consisting of patients with less comorbidities than patients in the nonorder set group. Additionally, patients in the order set intervention group were older than in the baseline group (48.0 vs 44.2 years). However, these differences in population remained relatively stable before and after the intervention, and relevant variables including demographic factors and CCI were accounted for in the regression models. Nevertheless, it remains possible that secular trends existed that we did not capture that affected the 2 populations differently. For example, there was a separate project that overlapped with the intervention period to reduce unnecessary laboratory usage at UUHC. This intervention could have influenced the trend to decreased laboratory utilization in the postintervention period. However, there were no concurrent initiatives to reduce antibiotic use during the study period. As a final limitation, the statistical analyses have not corrected for multiple testing for the secondary outcomes.

CONCLUSION

Using benchmark data from UHC, an academic medical center was able to identify an opportunity for improving the care of patients with cellulitis and subsequently develop an evidence‐based care pathway. The implementation of this pathway correlated with a significant reduction of broad‐spectrum antibiotic use, pharmacy costs, and total facility costs without adverse clinical affects. An important factor in the success of the intervention was the use of electronic order sets for cellulitis, which provided support for the implementation of the care pathway. This study demonstrates that the intervention was not only effective overall, but that it was more effective for those patients for whom the order set was used. This study adds to the growing body of literature suggesting that a well‐defined care pathway can improve outcomes and reduce costs for patients and institutions.

Cellulitis is a common infection causing inflammation of the skin and subcutaneous tissues. Cellulitis has been attributed to gram‐positive organisms through historical evaluations including fine‐needle aspirates and punch biopsies of the infected tissue.[1] Neither of these diagnostic tests is currently used due to their invasiveness, poor diagnostic yield, and availability. Similarly, readily available tests such as blood cultures provide an etiology <5% of the time[1] and are not cost‐effective for most patients for diagnosing cellulitis.[2] In addition, the prevalence of methicillin‐resistant Staphylococcus aureus (MRSA) has steadily increased, complicating decisions about antibiotic selection.[3] The result of this uncertainty is a large variation in practice with respect to antibiotic and imaging selection for patients with a diagnosis of cellulitis.

University of Utah Health Care (UUHC) performed benchmarking for the management of cellulitis using the University HealthSystem Consortium (UHC) database and associated CareFx analytics tool. Benchmarking demonstrated that UUHC had a greater percentage of broad‐spectrum antibiotic use (defined as vancomycin, piperacillin/tazobactam, or carbapenems) than the top 5 performing UHC facilities for International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) diagnoses of cellulitis (vancomycin 83% vs 58% and carbapenem or piperacillin/tazobactam 44% vs 16%). Advanced imaging (computed tomography [CT] or magnetic resonance imaging [MRI]) for the diagnosis of cellulitis was also found to be an opportunity for improvement (CT 27% vs 20% and MRI 8% vs 5%). The hospitalist group (most patients admitted with cellulitis were on this service) believed these data reflected current practice, as there was no standard of treatment for cellulitis despite an active order set. Therefore, cellulitis was considered an opportunity to improve value to our patients. A standardized clinical care pathway was created, as such pathways have demonstrated a reduction in variation in practice and improved efficiency and effectiveness of care for multiple disease states including cellulitis.[4, 5] We hypothesized that implementation of an evidence‐based care pathway would decrease broad‐spectrum antibiotic use, cost, and use of advanced imaging without having any adverse effects on clinical outcomes such as length of stay (LOS) or readmission.

METHODS

Intervention

Initial steps involved the formation of a multidisciplinary team including key stakeholders from the hospitalist group, infectious diseases, the emergency department (ED), and nursing. This multidisciplinary team was charged with developing a clinical care pathway appropriate for local implementation. National guidance for the care pathway was mainly obtained from the Infectious Disease Society of America (IDSA) guidelines on skin and soft tissue infections (SSTIs)[6] and MRSA.[7] Specific attention was paid to recommendations on blood cultures (only when systemically ill), imaging (rarely needed), antibiotic selection (rarely gram‐negative coverage and consideration of MRSA coverage), and patient‐care principles that are often overlooked (elevation of the affected extremity). A distinction of purulent versus nonpurulent cellulitis was adopted based on the guidelines and a prospective evaluation of the care of patients with nonpurulent cellulitis.[8] The 2014 IDSA update on SSTIs incorporates this distinction more clearly in hopes of determining staphylococcal versus streptococcal infections.[9] After multiple iterations, an agreed‐upon care pathway was created that excluded patients with neutropenia, osteomyelitis, diabetic foot ulcerations; hand, perineal, periorbital, or surgical site infections; and human or animal bites (Figure 1). After the care pathway was determined, interventions were performed to implement this change.

Figure 1

Education of all providers involved included discussion of cellulitis as a disease process, presentation of benchmarking data, dissemination of the care pathway to hospitalist and ED physicians, teaching conferences for internal medicine residents and ED residents, and reinforcement of these concepts at the beginning of resident rotations.

Incorporation of the care pathway into the existing electronic order sets for cellulitis care in the inpatient and ED settings, with links to the care pathway, links to excluded disease processes (eg, hand cellulitis), preselection of commonly needed items (eg, elevate leg), and recommendations for antibiotic selection based on categories of purulent or nonpurulent cellulitis. The electronic health record (EHR) did not allow for forced order set usage, so the order set required selection by the admitting physician if indicated. Additionally, an embedded 48‐hour order set could be accessed at any time by the ordering physician and included vancomycin dosing. Specific changes to the preexisting order set included the development of sections for purulent and nonpurulent cellulitis as well as recommended antibiotics. Piperacillin/tazobactam and nafcillin were both removed and vancomycin was limited to the purulent subheading. Additionally, elevation of the extremity was preselected, and orderables for imaging (chest x‐ray and duplex ultrasound), antiulcer prophylaxis, telemetry, and electrocardiograph were all removed.

Audit and feedback of cases of cellulitis and broad‐spectrum antibiotic usage was performed by a senior hospitalist.

RESULTS

Order Set Use Groups Analysis

Descriptive statistics and simple comparison before/after the intervention for the 2 study groups are shown in the last 6 columns of Table 1. Among patients for whom order sets were used, broad‐spectrum antibiotic usage significantly decreased from 78% before the intervention to 49% after the intervention (P < 0.001). In contrast, among patients for whom order sets were not used, broad‐spectrum antibiotic usage remained relatively constant71% before the intervention versus 67% after the intervention (P = 0.50). Figure 2 shows semiannual changes in the prescription of broad‐spectrum antibiotics. There is a noticeable drop after the intervention among patients for whom order sets were used.

Figure 2

Analysis of the interaction between time period and order set usage is provided in Table 3. After the intervention, patients for whom the order sets were used had greater improvement in broad‐spectrum antibiotic selection (75% decrease, P < 0.001) and LOS (25% decrease, P = 0.041) than patients for whom order sets were not used. Pharmacy costs also decreased by 13% more among patients for whom the order sets were used, although the interaction was not statistically significant (P = 0.074). Laboratory costs decreased in both groups, but order set use did not demonstrate an interaction (P = 0.5). Similar results were found for the subgroups of admitted patients and patients seen in the EDOU.

Table 3.Differential Impact of the Intervention on Process Metrics, Clinical, and Cost Outcomes in Two Order Set Use Levels

Logistic Regression
Outcome Variables	Selected Predictor Variables	Odds*	Percent Change	P
Gamma Regression
Outcome Variables	Selected Predictor Variables	Fold Change*	Percent Change	P
* Exponentiation of the parameter for the variable represents odds for categorical variables and fold change in amount for continuous variables. Minus sign represents decrease in percent change in odds or fold change. P values are based on generalized linear models including gender, age at admission, Charlson Comorbidity Index, hospitalization status, order set use, time period, and interaction term between time period and order set use as predictor variables.
Broad spectrum antibiotics	Time period	0.84 (0.50, 1.40)	16% (50% to 40%)	0.50
	Time periodorder set	0.25 (0.12, 0.52)	75% (88% to 48%)	<0.001
MRI done	Time period	1.04 (0.49, 2.20)	4% (51% to 120%)	0.92
	Time periodorder set	0.44 (0.14, 1.38)	56% (86% to 38%)	0.16
CT done	Time period	0.94 (0.55, 1.60)	6% (45% to 60%)	0.81
	Time periodorder set	0.96 (0.44, 2.12)	4% (56% to 112%)	0.93
30‐day readmission	Time period	0.91 (0.33, 2.53)	9% (67% to 153%)	0.86
	Time periodorder set	0.88 (0.20, 3.93)	12% (80% to 293%)	0.87
Clinical length of stay	Time period	1.04 (0.95, 1.14)	4% (5% to 14%)	0.41
	Time periodorder set	0.87 (0.77, 0.99)	13% (23% to 1%)	0.041
Pharmacy cost ($)	Time period	0.88 (0.70, 1.12)	12% (30% to 12%)	0.31
	Time periodorder set	0.75 (0.54, 1.03)	25% (46% to 3%)	0.074
Lab cost ($)	Time period	0.53 (0.42, 0.66)	47% (58% to 34%)	<0.001
	Time periodorder set	1.11 (0.82, 1.50)	11% (18% to 50%)	0.50
Imaging cost ($)	Time period	1.00 (0.71, 1.40)	0% (29% to 40%)	0.98
	Time periodorder set	0.82 (0.51, 1.30)	18% (49% to 30%)	0.39
Facility cost ($)	Time period	0.92 (0.80, 1.05)	8% (20% to 5%)	0.22
	Time periodorder set	0.90 (0.75, 1.09)	10% (25% to 9%)	0.29

Audit and feedback was initially performed for cases of cellulitis using broad‐spectrum antibiotics. However, given the complexity of cellulitis as a disease process and the frequency of broad‐spectrum antibiotic usage, in all cases of review, it was deemed reasonable to use broad‐spectrum antibiotics. Therefore, the audit was not continued.

DISCUSSION

Care pathways have demonstrated improvement across multiple different disease states including cellulitis.[4, 5] They have been noted to reduce variation in practice and improve physician agreement about treatment options.[4] The best method for implementation is not clearly understood,[12] and there remains concern about maintaining flexibility for patient care.[13] Additionally, although implementation of pathways is often well described, evaluations of the processes are noted to frequently be weak.[12] UUHC felt that the literature supported implementing a care pathway for the diagnosis of cellulitis, but that a thorough evaluation was also needed to understand any resulting benefits or harms. Through this study, we found that the implementation of this pathway resulted in a significant decrease in broad‐spectrum antibiotic use, pharmacy costs, and total facility costs. There was also a trend to decrease in imaging cost, and there were no adverse effects on LOS or 30‐day readmissions. Our findings demonstrate that care‐pathway implementation accompanied by education, pathway‐compliant electronic order sets, and audit and feedback can help drive improvements in quality while reducing costs. This finding furthers the evidence supporting standard work through the creation of clinical care pathways for cellulitis as an effective intervention.[4] Additionally, although not measured in this study, reduction of antibiotic use is supported as a measure to help reduce Clostridium difficile infections, a further potential benefit.[14]

This study has several important strengths. First, we included accurate cost analyses using the VDO tool. Given the increasing importance of improving care value, we feel the inclusion of such cost analysis is an increasingly important aspect of health service intervention evaluations. Second, we used a formal benchmarking approach to identify a priority care improvement area and to monitor changes in practice following the rollout of the intervention. We feel this approach provides a useful example on how to systematically improve care quality and value in a broader health system context. Third, we evaluated not order set implementation per se, but rather changing an existing order set. Because studies in this area generally focus on initial order set implementation, our study contributes insights on what can be expected through modifications of existing order sets based on care pathways. Fourth, the analysis accounted for a variety of variables including the CCI. Of interest, our study found that the intervention group (patients for whom order sets were used) had a lower CCI, confirming Allen et al.'s findings that diseases with predictable trajectories are the most likely to benefit from care pathways.[4] As a final strength, the narrative‐based order set intervention was relatively simple, and the inclusion criteria were broad, making the process generalizable.

Limitations of this study include that it was a single center pre‐/postintervention study and not a randomized controlled trial. Related to this limitation, the control group for which order sets were not used reflected a different patient population compared to the intervention group for which order sets were used. Specifically, it was more common for order sets to be used in the EDOU than upon admission, resulting in the order set group consisting of patients with less comorbidities than patients in the nonorder set group. Additionally, patients in the order set intervention group were older than in the baseline group (48.0 vs 44.2 years). However, these differences in population remained relatively stable before and after the intervention, and relevant variables including demographic factors and CCI were accounted for in the regression models. Nevertheless, it remains possible that secular trends existed that we did not capture that affected the 2 populations differently. For example, there was a separate project that overlapped with the intervention period to reduce unnecessary laboratory usage at UUHC. This intervention could have influenced the trend to decreased laboratory utilization in the postintervention period. However, there were no concurrent initiatives to reduce antibiotic use during the study period. As a final limitation, the statistical analyses have not corrected for multiple testing for the secondary outcomes.

CONCLUSION

Using benchmark data from UHC, an academic medical center was able to identify an opportunity for improving the care of patients with cellulitis and subsequently develop an evidence‐based care pathway. The implementation of this pathway correlated with a significant reduction of broad‐spectrum antibiotic use, pharmacy costs, and total facility costs without adverse clinical affects. An important factor in the success of the intervention was the use of electronic order sets for cellulitis, which provided support for the implementation of the care pathway. This study demonstrates that the intervention was not only effective overall, but that it was more effective for those patients for whom the order set was used. This study adds to the growing body of literature suggesting that a well‐defined care pathway can improve outcomes and reduce costs for patients and institutions.

Cellulitis is a common infection causing inflammation of the skin and subcutaneous tissues. Cellulitis has been attributed to gram‐positive organisms through historical evaluations including fine‐needle aspirates and punch biopsies of the infected tissue.[1] Neither of these diagnostic tests is currently used due to their invasiveness, poor diagnostic yield, and availability. Similarly, readily available tests such as blood cultures provide an etiology <5% of the time[1] and are not cost‐effective for most patients for diagnosing cellulitis.[2] In addition, the prevalence of methicillin‐resistant Staphylococcus aureus (MRSA) has steadily increased, complicating decisions about antibiotic selection.[3] The result of this uncertainty is a large variation in practice with respect to antibiotic and imaging selection for patients with a diagnosis of cellulitis.

University of Utah Health Care (UUHC) performed benchmarking for the management of cellulitis using the University HealthSystem Consortium (UHC) database and associated CareFx analytics tool. Benchmarking demonstrated that UUHC had a greater percentage of broad‐spectrum antibiotic use (defined as vancomycin, piperacillin/tazobactam, or carbapenems) than the top 5 performing UHC facilities for International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) diagnoses of cellulitis (vancomycin 83% vs 58% and carbapenem or piperacillin/tazobactam 44% vs 16%). Advanced imaging (computed tomography [CT] or magnetic resonance imaging [MRI]) for the diagnosis of cellulitis was also found to be an opportunity for improvement (CT 27% vs 20% and MRI 8% vs 5%). The hospitalist group (most patients admitted with cellulitis were on this service) believed these data reflected current practice, as there was no standard of treatment for cellulitis despite an active order set. Therefore, cellulitis was considered an opportunity to improve value to our patients. A standardized clinical care pathway was created, as such pathways have demonstrated a reduction in variation in practice and improved efficiency and effectiveness of care for multiple disease states including cellulitis.[4, 5] We hypothesized that implementation of an evidence‐based care pathway would decrease broad‐spectrum antibiotic use, cost, and use of advanced imaging without having any adverse effects on clinical outcomes such as length of stay (LOS) or readmission.

METHODS

Intervention

Initial steps involved the formation of a multidisciplinary team including key stakeholders from the hospitalist group, infectious diseases, the emergency department (ED), and nursing. This multidisciplinary team was charged with developing a clinical care pathway appropriate for local implementation. National guidance for the care pathway was mainly obtained from the Infectious Disease Society of America (IDSA) guidelines on skin and soft tissue infections (SSTIs)[6] and MRSA.[7] Specific attention was paid to recommendations on blood cultures (only when systemically ill), imaging (rarely needed), antibiotic selection (rarely gram‐negative coverage and consideration of MRSA coverage), and patient‐care principles that are often overlooked (elevation of the affected extremity). A distinction of purulent versus nonpurulent cellulitis was adopted based on the guidelines and a prospective evaluation of the care of patients with nonpurulent cellulitis.[8] The 2014 IDSA update on SSTIs incorporates this distinction more clearly in hopes of determining staphylococcal versus streptococcal infections.[9] After multiple iterations, an agreed‐upon care pathway was created that excluded patients with neutropenia, osteomyelitis, diabetic foot ulcerations; hand, perineal, periorbital, or surgical site infections; and human or animal bites (Figure 1). After the care pathway was determined, interventions were performed to implement this change.

Figure 1

Education of all providers involved included discussion of cellulitis as a disease process, presentation of benchmarking data, dissemination of the care pathway to hospitalist and ED physicians, teaching conferences for internal medicine residents and ED residents, and reinforcement of these concepts at the beginning of resident rotations.

Incorporation of the care pathway into the existing electronic order sets for cellulitis care in the inpatient and ED settings, with links to the care pathway, links to excluded disease processes (eg, hand cellulitis), preselection of commonly needed items (eg, elevate leg), and recommendations for antibiotic selection based on categories of purulent or nonpurulent cellulitis. The electronic health record (EHR) did not allow for forced order set usage, so the order set required selection by the admitting physician if indicated. Additionally, an embedded 48‐hour order set could be accessed at any time by the ordering physician and included vancomycin dosing. Specific changes to the preexisting order set included the development of sections for purulent and nonpurulent cellulitis as well as recommended antibiotics. Piperacillin/tazobactam and nafcillin were both removed and vancomycin was limited to the purulent subheading. Additionally, elevation of the extremity was preselected, and orderables for imaging (chest x‐ray and duplex ultrasound), antiulcer prophylaxis, telemetry, and electrocardiograph were all removed.

Audit and feedback of cases of cellulitis and broad‐spectrum antibiotic usage was performed by a senior hospitalist.

RESULTS

Order Set Use Groups Analysis

Descriptive statistics and simple comparison before/after the intervention for the 2 study groups are shown in the last 6 columns of Table 1. Among patients for whom order sets were used, broad‐spectrum antibiotic usage significantly decreased from 78% before the intervention to 49% after the intervention (P < 0.001). In contrast, among patients for whom order sets were not used, broad‐spectrum antibiotic usage remained relatively constant71% before the intervention versus 67% after the intervention (P = 0.50). Figure 2 shows semiannual changes in the prescription of broad‐spectrum antibiotics. There is a noticeable drop after the intervention among patients for whom order sets were used.

Figure 2

Analysis of the interaction between time period and order set usage is provided in Table 3. After the intervention, patients for whom the order sets were used had greater improvement in broad‐spectrum antibiotic selection (75% decrease, P < 0.001) and LOS (25% decrease, P = 0.041) than patients for whom order sets were not used. Pharmacy costs also decreased by 13% more among patients for whom the order sets were used, although the interaction was not statistically significant (P = 0.074). Laboratory costs decreased in both groups, but order set use did not demonstrate an interaction (P = 0.5). Similar results were found for the subgroups of admitted patients and patients seen in the EDOU.

Table 3.Differential Impact of the Intervention on Process Metrics, Clinical, and Cost Outcomes in Two Order Set Use Levels

Logistic Regression
Outcome Variables	Selected Predictor Variables	Odds*	Percent Change	P
Gamma Regression
Outcome Variables	Selected Predictor Variables	Fold Change*	Percent Change	P
* Exponentiation of the parameter for the variable represents odds for categorical variables and fold change in amount for continuous variables. Minus sign represents decrease in percent change in odds or fold change. P values are based on generalized linear models including gender, age at admission, Charlson Comorbidity Index, hospitalization status, order set use, time period, and interaction term between time period and order set use as predictor variables.
Broad spectrum antibiotics	Time period	0.84 (0.50, 1.40)	16% (50% to 40%)	0.50
	Time periodorder set	0.25 (0.12, 0.52)	75% (88% to 48%)	<0.001
MRI done	Time period	1.04 (0.49, 2.20)	4% (51% to 120%)	0.92
	Time periodorder set	0.44 (0.14, 1.38)	56% (86% to 38%)	0.16
CT done	Time period	0.94 (0.55, 1.60)	6% (45% to 60%)	0.81
	Time periodorder set	0.96 (0.44, 2.12)	4% (56% to 112%)	0.93
30‐day readmission	Time period	0.91 (0.33, 2.53)	9% (67% to 153%)	0.86
	Time periodorder set	0.88 (0.20, 3.93)	12% (80% to 293%)	0.87
Clinical length of stay	Time period	1.04 (0.95, 1.14)	4% (5% to 14%)	0.41
	Time periodorder set	0.87 (0.77, 0.99)	13% (23% to 1%)	0.041
Pharmacy cost ($)	Time period	0.88 (0.70, 1.12)	12% (30% to 12%)	0.31
	Time periodorder set	0.75 (0.54, 1.03)	25% (46% to 3%)	0.074
Lab cost ($)	Time period	0.53 (0.42, 0.66)	47% (58% to 34%)	<0.001
	Time periodorder set	1.11 (0.82, 1.50)	11% (18% to 50%)	0.50
Imaging cost ($)	Time period	1.00 (0.71, 1.40)	0% (29% to 40%)	0.98
	Time periodorder set	0.82 (0.51, 1.30)	18% (49% to 30%)	0.39
Facility cost ($)	Time period	0.92 (0.80, 1.05)	8% (20% to 5%)	0.22
	Time periodorder set	0.90 (0.75, 1.09)	10% (25% to 9%)	0.29

Audit and feedback was initially performed for cases of cellulitis using broad‐spectrum antibiotics. However, given the complexity of cellulitis as a disease process and the frequency of broad‐spectrum antibiotic usage, in all cases of review, it was deemed reasonable to use broad‐spectrum antibiotics. Therefore, the audit was not continued.

DISCUSSION

Care pathways have demonstrated improvement across multiple different disease states including cellulitis.[4, 5] They have been noted to reduce variation in practice and improve physician agreement about treatment options.[4] The best method for implementation is not clearly understood,[12] and there remains concern about maintaining flexibility for patient care.[13] Additionally, although implementation of pathways is often well described, evaluations of the processes are noted to frequently be weak.[12] UUHC felt that the literature supported implementing a care pathway for the diagnosis of cellulitis, but that a thorough evaluation was also needed to understand any resulting benefits or harms. Through this study, we found that the implementation of this pathway resulted in a significant decrease in broad‐spectrum antibiotic use, pharmacy costs, and total facility costs. There was also a trend to decrease in imaging cost, and there were no adverse effects on LOS or 30‐day readmissions. Our findings demonstrate that care‐pathway implementation accompanied by education, pathway‐compliant electronic order sets, and audit and feedback can help drive improvements in quality while reducing costs. This finding furthers the evidence supporting standard work through the creation of clinical care pathways for cellulitis as an effective intervention.[4] Additionally, although not measured in this study, reduction of antibiotic use is supported as a measure to help reduce Clostridium difficile infections, a further potential benefit.[14]

This study has several important strengths. First, we included accurate cost analyses using the VDO tool. Given the increasing importance of improving care value, we feel the inclusion of such cost analysis is an increasingly important aspect of health service intervention evaluations. Second, we used a formal benchmarking approach to identify a priority care improvement area and to monitor changes in practice following the rollout of the intervention. We feel this approach provides a useful example on how to systematically improve care quality and value in a broader health system context. Third, we evaluated not order set implementation per se, but rather changing an existing order set. Because studies in this area generally focus on initial order set implementation, our study contributes insights on what can be expected through modifications of existing order sets based on care pathways. Fourth, the analysis accounted for a variety of variables including the CCI. Of interest, our study found that the intervention group (patients for whom order sets were used) had a lower CCI, confirming Allen et al.'s findings that diseases with predictable trajectories are the most likely to benefit from care pathways.[4] As a final strength, the narrative‐based order set intervention was relatively simple, and the inclusion criteria were broad, making the process generalizable.

Limitations of this study include that it was a single center pre‐/postintervention study and not a randomized controlled trial. Related to this limitation, the control group for which order sets were not used reflected a different patient population compared to the intervention group for which order sets were used. Specifically, it was more common for order sets to be used in the EDOU than upon admission, resulting in the order set group consisting of patients with less comorbidities than patients in the nonorder set group. Additionally, patients in the order set intervention group were older than in the baseline group (48.0 vs 44.2 years). However, these differences in population remained relatively stable before and after the intervention, and relevant variables including demographic factors and CCI were accounted for in the regression models. Nevertheless, it remains possible that secular trends existed that we did not capture that affected the 2 populations differently. For example, there was a separate project that overlapped with the intervention period to reduce unnecessary laboratory usage at UUHC. This intervention could have influenced the trend to decreased laboratory utilization in the postintervention period. However, there were no concurrent initiatives to reduce antibiotic use during the study period. As a final limitation, the statistical analyses have not corrected for multiple testing for the secondary outcomes.

CONCLUSION

Using benchmark data from UHC, an academic medical center was able to identify an opportunity for improving the care of patients with cellulitis and subsequently develop an evidence‐based care pathway. The implementation of this pathway correlated with a significant reduction of broad‐spectrum antibiotic use, pharmacy costs, and total facility costs without adverse clinical affects. An important factor in the success of the intervention was the use of electronic order sets for cellulitis, which provided support for the implementation of the care pathway. This study demonstrates that the intervention was not only effective overall, but that it was more effective for those patients for whom the order set was used. This study adds to the growing body of literature suggesting that a well‐defined care pathway can improve outcomes and reduce costs for patients and institutions.