Vincent S. Fan, MD, MPH

Waste in US healthcare is a public health threat, with an estimated value of $910 billion per year.[1] It constitutes some of the relatively high per‐discharge healthcare spending seen in the United States when compared to other nations.[2] Waste takes many forms, one of which is excessive use of diagnostic laboratory testing.[1] Many hospital providers obtain common labs, such as complete blood counts (CBCs) and basic metabolic panels (BMPs), in an open‐ended, daily manner for their hospitalized patients, without regard for the patient's clinical condition or despite stability of the previous results. Reasons for ordering these tests in a nonpatient‐centered manner include provider convenience (such as inclusion in an order set), ease of access, habit, or defensive practice.[3, 4, 5] All of these reasons may represent waste.

Although the potential waste of routine daily labs may seem small, the frequency with which they are ordered results in a substantial real and potential cost, both financially and clinically. Multiple studies have shown a link between excessive diagnostic phlebotomy and hospital‐acquired anemia.[6, 7, 8, 9] Hospital‐acquired anemia itself has been associated with increased mortality.[10] In addition to blood loss and financial cost, patient experience and satisfaction are also detrimentally affected by excessive laboratory testing in the form of pain and inconvenience from the act of phlebotomy.[11]

There are many reports of strategies to decrease excessive diagnostic laboratory testing as a means of addressing this waste in the inpatient setting.[12, 13, 14, 15, 16, 17, 18, 19, 20, 21] All of these studies have taken place in a traditional academic setting, and many implemented their intervention through a computer‐based order entry system. Based on the literature search regarding this topic, we found no examples of studies conducted among and within community‐based hospitalist practices. More recently, this issue was highlighted as part of the Choosing Wisely campaign sponsored by the American Board of Internal Medicine Foundation, Consumer Reports, and more than 60 specialty societies. The Society of Hospital Medicine, the professional society for hospitalists, recommended avoidance of repetitive common laboratory testing in the face of clinical stability.[22]

Much has been written about quality improvement (QI) by the Institute for Healthcare Improvement, the Society of Hospitalist Medicine, and others.[23, 24, 25] How best to move from a Choosing Wisely recommendation to highly reliable incorporation in clinical practice in a community setting is not known and likely varies depending upon the care environment. Successful QI interventions are often multifaceted and include academic detailing and provider education, transparent display of data, and regular audit and feedback of performance data.[26, 27, 28, 29] Prior to the publication of the Society of Hospital Medicine's Choosing Wisely recommendations, we chose to implement the recommendation to decrease ordering of daily labs using 3 QI strategies in our community 4‐hospital health system.

METHODS

RESULTS

Primary Endpoint

In the unadjusted comparison, 3 of the 4 common labs showed a similar decrease in the intervention cohort from the baseline (Table 2). For example, the mean number of CBCs ordered per patient‐day decreased by 0.15 labs per patient day (1.06 vs 0.91, P<0.01). The total number of common labs ordered per patient‐day decreased by 0.30 labs per patient‐day (2.06 vs 1.76, P<0.01) in the unadjusted analysis (Figure 1 and Table 2). Part of our hypothesis was that decreasing the number of labs that were ordered as daily, in an open‐ended manner, would likely decrease the number of common labs obtained per day. We found that the number of labs ordered as daily decreased by 0.71 labs per patient‐day (0.872.90 vs 0.161.01, P<0.01), an 81.6% decrease from the preintervention time period.

Table 2.Patient Outcomes by Daily Lab Cohort

	Baseline	Intervention	P Valuea
NOTE: Abbreviations: SD, standard deviation. a P value determined by [2] or Student t test. b Basic metabolic panel plus magnesium and phosphate.
Complete blood count, per patient‐day, mean (SD)	1.06 (0.76)	0.91 (0.75)	<0.01
Basic metabolic panel, per patient‐day, mean (SD)	0.68 (0.71)	0.55 (0.60)	<0.01
Nutrition panel, mean (SD)b	0.06 (0.24)	0.07 (0.32)	0.01
Comprehensive metabolic panel, per patient‐day, mean (SD)	0.27 (0.49)	0.23 (0.46)	<0.01
Total no. of basic labs ordered per patient‐day, mean (SD)	2.06 (1.40)	1.76 (1.37)	<0.01
Transfused, n (%)	414 (5.3)	268 (4.7)	0.1
Transfused volume, mL, mean (SD)	847.3 (644.3)	744.9 (472.0)	0.02
Length of stay, days, mean (SD)	3.79 (4.58)	3.81 (4.50)	0.7
Readmitted, n (%)	1049 (13.3)	733 (12.7)	0.3
Died, n (%)	173 (2.2)	104 (1.8)	0.1

Figure 1

In our multivariable regression model, after adjusting for sex, age, and the primary reason for admission as captured by DRG, the number of common labs ordered per day was reduced by 0.22 (95% CI, 0.34 to 0.11; P<0.01). This represents a 10.7% reduction in common labs ordered per patient day.

DISCUSSION

Ordering common labs daily is a routine practice among providers at many institutions. In fact, at our institution, prior to the intervention, 42% of all common labs were ordered as daily, meaning they were obtained each day without regard to the previous value or the patient's clinical condition. The practice is one of convenience or habit, and many times not clinically indicated.[5, 32]

We observed a significant reduction in the number of common labs ordered as daily, and more importantly, the total number of common labs in the intervention period. The rapid change in provider behavior is notable and likely due to several factors. First, there was a general sentiment among the hospitalists in the merits of the project. Second, there may have been an aversion to the display of lower performance relative to peers in the monthly e‐mails. Third, and perhaps most importantly, our hospitalist team had worked together for many years on projects like this, creating a culture of QI and willingness to change practice patterns in response to data.[33]

Concern about decreasing waste and increasing the value of healthcare abound, particularly in the United States.[1] Decreasing the cost to produce equivalent or improved health outcomes for a given episode of care has been proposed as a way to improve value.[34] This intervention results in modest waste reduction, the benefits of which are readily apparent in a DRG‐based reimbursement model, where the hospital realizes any saving in the cost of producing a hospital stay, as well as in a total cost of care environment, such as could be found in an Accountable Care Organization.

The previous work in the field of lab reduction has all been performed at university‐affiliated academic institutions. We demonstrated that the QI tactics described in the literature can be successfully employed in a community‐based hospitalist practice. This has broad applicability to increasing the value of healthcare and could serve as a model for future community‐based hospitalist QI projects.

The study has several limitations. First, the length of follow‐up is only 7 months, and although there was rapid and effective adoption of the intervention, provider behavior may regress to previous practice patterns over time. Second, the simple before‐after nature of our trial design raises the possibility that environmental influences exist and that changes in ordering behavior may have been the result of something other than the intervention. Most notably, the Choosing Wisely recommendation for hospitalists was published in September of 2013, coinciding with our intervention period.[22] The reduction in number of labs ordered may have been a partial result of these recommendations. Third, the 2 cohorts included different times of the year based on the distribution of DRGs, which likely had a different composition of diagnoses being treated. To address this we adjusted for DRG, but there may have been some residual confounding, as some diagnoses may be managed with more laboratory tests than others in a way that was not fully adjusted for in our model. Fourth, the intervention was made possible because of the substantial and ongoing investments that our health system has made in our electronic medical record and data analytics capability. The variability of these resources across institutions limits generalizability. Fifth, although we used the QI tools that were described, we did not do a formal process map or utilize other Lean or Six Sigma tools. As the healthcare industry continues on its journey to high reliability, these use tools will hopefully become more widespread. We demonstrated that even with these simple tactics, significant progress can be made.

Finally, there exists a concern that decreasing regular laboratory monitoring might be associated with undetected worsening in the patient's clinical status. We did not observe any significant adverse effects on coarse measures of clinical performance, including length of stay, readmission rate, or mortality. However, we did not collect data on all clinical parameters, and it is possible that there could have been an undetected effect on incident renal failure or hemodialysis or intensive care unit transfer. Other studies on this type of intervention have evaluated some of these possible adverse outcomes and have not noted an association.[12, 15, 18, 20, 22] Future studies should evaluate harms associated with implementation of Choosing Wisely and other interventions targeted at waste reduction. Future work is also needed to disseminate more formal and rigorous QI tools and methodologies.

CONCLUSION

We implemented a multifaceted QI intervention including provider education, transparent display of data, and audit and feedback that was associated with a significant reduction in the number of common labs ordered in a large community‐based hospitalist group, without evidence of harm. Further study is needed to understand how hospitalist groups can optimally decrease waste in healthcare.

Waste in US healthcare is a public health threat, with an estimated value of $910 billion per year.[1] It constitutes some of the relatively high per‐discharge healthcare spending seen in the United States when compared to other nations.[2] Waste takes many forms, one of which is excessive use of diagnostic laboratory testing.[1] Many hospital providers obtain common labs, such as complete blood counts (CBCs) and basic metabolic panels (BMPs), in an open‐ended, daily manner for their hospitalized patients, without regard for the patient's clinical condition or despite stability of the previous results. Reasons for ordering these tests in a nonpatient‐centered manner include provider convenience (such as inclusion in an order set), ease of access, habit, or defensive practice.[3, 4, 5] All of these reasons may represent waste.

Although the potential waste of routine daily labs may seem small, the frequency with which they are ordered results in a substantial real and potential cost, both financially and clinically. Multiple studies have shown a link between excessive diagnostic phlebotomy and hospital‐acquired anemia.[6, 7, 8, 9] Hospital‐acquired anemia itself has been associated with increased mortality.[10] In addition to blood loss and financial cost, patient experience and satisfaction are also detrimentally affected by excessive laboratory testing in the form of pain and inconvenience from the act of phlebotomy.[11]

There are many reports of strategies to decrease excessive diagnostic laboratory testing as a means of addressing this waste in the inpatient setting.[12, 13, 14, 15, 16, 17, 18, 19, 20, 21] All of these studies have taken place in a traditional academic setting, and many implemented their intervention through a computer‐based order entry system. Based on the literature search regarding this topic, we found no examples of studies conducted among and within community‐based hospitalist practices. More recently, this issue was highlighted as part of the Choosing Wisely campaign sponsored by the American Board of Internal Medicine Foundation, Consumer Reports, and more than 60 specialty societies. The Society of Hospital Medicine, the professional society for hospitalists, recommended avoidance of repetitive common laboratory testing in the face of clinical stability.[22]

Much has been written about quality improvement (QI) by the Institute for Healthcare Improvement, the Society of Hospitalist Medicine, and others.[23, 24, 25] How best to move from a Choosing Wisely recommendation to highly reliable incorporation in clinical practice in a community setting is not known and likely varies depending upon the care environment. Successful QI interventions are often multifaceted and include academic detailing and provider education, transparent display of data, and regular audit and feedback of performance data.[26, 27, 28, 29] Prior to the publication of the Society of Hospital Medicine's Choosing Wisely recommendations, we chose to implement the recommendation to decrease ordering of daily labs using 3 QI strategies in our community 4‐hospital health system.

METHODS

RESULTS

Primary Endpoint

In the unadjusted comparison, 3 of the 4 common labs showed a similar decrease in the intervention cohort from the baseline (Table 2). For example, the mean number of CBCs ordered per patient‐day decreased by 0.15 labs per patient day (1.06 vs 0.91, P<0.01). The total number of common labs ordered per patient‐day decreased by 0.30 labs per patient‐day (2.06 vs 1.76, P<0.01) in the unadjusted analysis (Figure 1 and Table 2). Part of our hypothesis was that decreasing the number of labs that were ordered as daily, in an open‐ended manner, would likely decrease the number of common labs obtained per day. We found that the number of labs ordered as daily decreased by 0.71 labs per patient‐day (0.872.90 vs 0.161.01, P<0.01), an 81.6% decrease from the preintervention time period.

Table 2.Patient Outcomes by Daily Lab Cohort

	Baseline	Intervention	P Valuea
NOTE: Abbreviations: SD, standard deviation. a P value determined by [2] or Student t test. b Basic metabolic panel plus magnesium and phosphate.
Complete blood count, per patient‐day, mean (SD)	1.06 (0.76)	0.91 (0.75)	<0.01
Basic metabolic panel, per patient‐day, mean (SD)	0.68 (0.71)	0.55 (0.60)	<0.01
Nutrition panel, mean (SD)b	0.06 (0.24)	0.07 (0.32)	0.01
Comprehensive metabolic panel, per patient‐day, mean (SD)	0.27 (0.49)	0.23 (0.46)	<0.01
Total no. of basic labs ordered per patient‐day, mean (SD)	2.06 (1.40)	1.76 (1.37)	<0.01
Transfused, n (%)	414 (5.3)	268 (4.7)	0.1
Transfused volume, mL, mean (SD)	847.3 (644.3)	744.9 (472.0)	0.02
Length of stay, days, mean (SD)	3.79 (4.58)	3.81 (4.50)	0.7
Readmitted, n (%)	1049 (13.3)	733 (12.7)	0.3
Died, n (%)	173 (2.2)	104 (1.8)	0.1

Figure 1

In our multivariable regression model, after adjusting for sex, age, and the primary reason for admission as captured by DRG, the number of common labs ordered per day was reduced by 0.22 (95% CI, 0.34 to 0.11; P<0.01). This represents a 10.7% reduction in common labs ordered per patient day.

DISCUSSION

Ordering common labs daily is a routine practice among providers at many institutions. In fact, at our institution, prior to the intervention, 42% of all common labs were ordered as daily, meaning they were obtained each day without regard to the previous value or the patient's clinical condition. The practice is one of convenience or habit, and many times not clinically indicated.[5, 32]

We observed a significant reduction in the number of common labs ordered as daily, and more importantly, the total number of common labs in the intervention period. The rapid change in provider behavior is notable and likely due to several factors. First, there was a general sentiment among the hospitalists in the merits of the project. Second, there may have been an aversion to the display of lower performance relative to peers in the monthly e‐mails. Third, and perhaps most importantly, our hospitalist team had worked together for many years on projects like this, creating a culture of QI and willingness to change practice patterns in response to data.[33]

Concern about decreasing waste and increasing the value of healthcare abound, particularly in the United States.[1] Decreasing the cost to produce equivalent or improved health outcomes for a given episode of care has been proposed as a way to improve value.[34] This intervention results in modest waste reduction, the benefits of which are readily apparent in a DRG‐based reimbursement model, where the hospital realizes any saving in the cost of producing a hospital stay, as well as in a total cost of care environment, such as could be found in an Accountable Care Organization.

The previous work in the field of lab reduction has all been performed at university‐affiliated academic institutions. We demonstrated that the QI tactics described in the literature can be successfully employed in a community‐based hospitalist practice. This has broad applicability to increasing the value of healthcare and could serve as a model for future community‐based hospitalist QI projects.

The study has several limitations. First, the length of follow‐up is only 7 months, and although there was rapid and effective adoption of the intervention, provider behavior may regress to previous practice patterns over time. Second, the simple before‐after nature of our trial design raises the possibility that environmental influences exist and that changes in ordering behavior may have been the result of something other than the intervention. Most notably, the Choosing Wisely recommendation for hospitalists was published in September of 2013, coinciding with our intervention period.[22] The reduction in number of labs ordered may have been a partial result of these recommendations. Third, the 2 cohorts included different times of the year based on the distribution of DRGs, which likely had a different composition of diagnoses being treated. To address this we adjusted for DRG, but there may have been some residual confounding, as some diagnoses may be managed with more laboratory tests than others in a way that was not fully adjusted for in our model. Fourth, the intervention was made possible because of the substantial and ongoing investments that our health system has made in our electronic medical record and data analytics capability. The variability of these resources across institutions limits generalizability. Fifth, although we used the QI tools that were described, we did not do a formal process map or utilize other Lean or Six Sigma tools. As the healthcare industry continues on its journey to high reliability, these use tools will hopefully become more widespread. We demonstrated that even with these simple tactics, significant progress can be made.

Finally, there exists a concern that decreasing regular laboratory monitoring might be associated with undetected worsening in the patient's clinical status. We did not observe any significant adverse effects on coarse measures of clinical performance, including length of stay, readmission rate, or mortality. However, we did not collect data on all clinical parameters, and it is possible that there could have been an undetected effect on incident renal failure or hemodialysis or intensive care unit transfer. Other studies on this type of intervention have evaluated some of these possible adverse outcomes and have not noted an association.[12, 15, 18, 20, 22] Future studies should evaluate harms associated with implementation of Choosing Wisely and other interventions targeted at waste reduction. Future work is also needed to disseminate more formal and rigorous QI tools and methodologies.

CONCLUSION

We implemented a multifaceted QI intervention including provider education, transparent display of data, and audit and feedback that was associated with a significant reduction in the number of common labs ordered in a large community‐based hospitalist group, without evidence of harm. Further study is needed to understand how hospitalist groups can optimally decrease waste in healthcare.

Waste in US healthcare is a public health threat, with an estimated value of $910 billion per year.[1] It constitutes some of the relatively high per‐discharge healthcare spending seen in the United States when compared to other nations.[2] Waste takes many forms, one of which is excessive use of diagnostic laboratory testing.[1] Many hospital providers obtain common labs, such as complete blood counts (CBCs) and basic metabolic panels (BMPs), in an open‐ended, daily manner for their hospitalized patients, without regard for the patient's clinical condition or despite stability of the previous results. Reasons for ordering these tests in a nonpatient‐centered manner include provider convenience (such as inclusion in an order set), ease of access, habit, or defensive practice.[3, 4, 5] All of these reasons may represent waste.

Although the potential waste of routine daily labs may seem small, the frequency with which they are ordered results in a substantial real and potential cost, both financially and clinically. Multiple studies have shown a link between excessive diagnostic phlebotomy and hospital‐acquired anemia.[6, 7, 8, 9] Hospital‐acquired anemia itself has been associated with increased mortality.[10] In addition to blood loss and financial cost, patient experience and satisfaction are also detrimentally affected by excessive laboratory testing in the form of pain and inconvenience from the act of phlebotomy.[11]

There are many reports of strategies to decrease excessive diagnostic laboratory testing as a means of addressing this waste in the inpatient setting.[12, 13, 14, 15, 16, 17, 18, 19, 20, 21] All of these studies have taken place in a traditional academic setting, and many implemented their intervention through a computer‐based order entry system. Based on the literature search regarding this topic, we found no examples of studies conducted among and within community‐based hospitalist practices. More recently, this issue was highlighted as part of the Choosing Wisely campaign sponsored by the American Board of Internal Medicine Foundation, Consumer Reports, and more than 60 specialty societies. The Society of Hospital Medicine, the professional society for hospitalists, recommended avoidance of repetitive common laboratory testing in the face of clinical stability.[22]

Much has been written about quality improvement (QI) by the Institute for Healthcare Improvement, the Society of Hospitalist Medicine, and others.[23, 24, 25] How best to move from a Choosing Wisely recommendation to highly reliable incorporation in clinical practice in a community setting is not known and likely varies depending upon the care environment. Successful QI interventions are often multifaceted and include academic detailing and provider education, transparent display of data, and regular audit and feedback of performance data.[26, 27, 28, 29] Prior to the publication of the Society of Hospital Medicine's Choosing Wisely recommendations, we chose to implement the recommendation to decrease ordering of daily labs using 3 QI strategies in our community 4‐hospital health system.

METHODS

RESULTS

Primary Endpoint

In the unadjusted comparison, 3 of the 4 common labs showed a similar decrease in the intervention cohort from the baseline (Table 2). For example, the mean number of CBCs ordered per patient‐day decreased by 0.15 labs per patient day (1.06 vs 0.91, P<0.01). The total number of common labs ordered per patient‐day decreased by 0.30 labs per patient‐day (2.06 vs 1.76, P<0.01) in the unadjusted analysis (Figure 1 and Table 2). Part of our hypothesis was that decreasing the number of labs that were ordered as daily, in an open‐ended manner, would likely decrease the number of common labs obtained per day. We found that the number of labs ordered as daily decreased by 0.71 labs per patient‐day (0.872.90 vs 0.161.01, P<0.01), an 81.6% decrease from the preintervention time period.

Table 2.Patient Outcomes by Daily Lab Cohort

	Baseline	Intervention	P Valuea
NOTE: Abbreviations: SD, standard deviation. a P value determined by [2] or Student t test. b Basic metabolic panel plus magnesium and phosphate.
Complete blood count, per patient‐day, mean (SD)	1.06 (0.76)	0.91 (0.75)	<0.01
Basic metabolic panel, per patient‐day, mean (SD)	0.68 (0.71)	0.55 (0.60)	<0.01
Nutrition panel, mean (SD)b	0.06 (0.24)	0.07 (0.32)	0.01
Comprehensive metabolic panel, per patient‐day, mean (SD)	0.27 (0.49)	0.23 (0.46)	<0.01
Total no. of basic labs ordered per patient‐day, mean (SD)	2.06 (1.40)	1.76 (1.37)	<0.01
Transfused, n (%)	414 (5.3)	268 (4.7)	0.1
Transfused volume, mL, mean (SD)	847.3 (644.3)	744.9 (472.0)	0.02
Length of stay, days, mean (SD)	3.79 (4.58)	3.81 (4.50)	0.7
Readmitted, n (%)	1049 (13.3)	733 (12.7)	0.3
Died, n (%)	173 (2.2)	104 (1.8)	0.1

Figure 1

In our multivariable regression model, after adjusting for sex, age, and the primary reason for admission as captured by DRG, the number of common labs ordered per day was reduced by 0.22 (95% CI, 0.34 to 0.11; P<0.01). This represents a 10.7% reduction in common labs ordered per patient day.

DISCUSSION

Ordering common labs daily is a routine practice among providers at many institutions. In fact, at our institution, prior to the intervention, 42% of all common labs were ordered as daily, meaning they were obtained each day without regard to the previous value or the patient's clinical condition. The practice is one of convenience or habit, and many times not clinically indicated.[5, 32]

We observed a significant reduction in the number of common labs ordered as daily, and more importantly, the total number of common labs in the intervention period. The rapid change in provider behavior is notable and likely due to several factors. First, there was a general sentiment among the hospitalists in the merits of the project. Second, there may have been an aversion to the display of lower performance relative to peers in the monthly e‐mails. Third, and perhaps most importantly, our hospitalist team had worked together for many years on projects like this, creating a culture of QI and willingness to change practice patterns in response to data.[33]

Concern about decreasing waste and increasing the value of healthcare abound, particularly in the United States.[1] Decreasing the cost to produce equivalent or improved health outcomes for a given episode of care has been proposed as a way to improve value.[34] This intervention results in modest waste reduction, the benefits of which are readily apparent in a DRG‐based reimbursement model, where the hospital realizes any saving in the cost of producing a hospital stay, as well as in a total cost of care environment, such as could be found in an Accountable Care Organization.

The previous work in the field of lab reduction has all been performed at university‐affiliated academic institutions. We demonstrated that the QI tactics described in the literature can be successfully employed in a community‐based hospitalist practice. This has broad applicability to increasing the value of healthcare and could serve as a model for future community‐based hospitalist QI projects.

The study has several limitations. First, the length of follow‐up is only 7 months, and although there was rapid and effective adoption of the intervention, provider behavior may regress to previous practice patterns over time. Second, the simple before‐after nature of our trial design raises the possibility that environmental influences exist and that changes in ordering behavior may have been the result of something other than the intervention. Most notably, the Choosing Wisely recommendation for hospitalists was published in September of 2013, coinciding with our intervention period.[22] The reduction in number of labs ordered may have been a partial result of these recommendations. Third, the 2 cohorts included different times of the year based on the distribution of DRGs, which likely had a different composition of diagnoses being treated. To address this we adjusted for DRG, but there may have been some residual confounding, as some diagnoses may be managed with more laboratory tests than others in a way that was not fully adjusted for in our model. Fourth, the intervention was made possible because of the substantial and ongoing investments that our health system has made in our electronic medical record and data analytics capability. The variability of these resources across institutions limits generalizability. Fifth, although we used the QI tools that were described, we did not do a formal process map or utilize other Lean or Six Sigma tools. As the healthcare industry continues on its journey to high reliability, these use tools will hopefully become more widespread. We demonstrated that even with these simple tactics, significant progress can be made.

Finally, there exists a concern that decreasing regular laboratory monitoring might be associated with undetected worsening in the patient's clinical status. We did not observe any significant adverse effects on coarse measures of clinical performance, including length of stay, readmission rate, or mortality. However, we did not collect data on all clinical parameters, and it is possible that there could have been an undetected effect on incident renal failure or hemodialysis or intensive care unit transfer. Other studies on this type of intervention have evaluated some of these possible adverse outcomes and have not noted an association.[12, 15, 18, 20, 22] Future studies should evaluate harms associated with implementation of Choosing Wisely and other interventions targeted at waste reduction. Future work is also needed to disseminate more formal and rigorous QI tools and methodologies.

CONCLUSION

We implemented a multifaceted QI intervention including provider education, transparent display of data, and audit and feedback that was associated with a significant reduction in the number of common labs ordered in a large community‐based hospitalist group, without evidence of harm. Further study is needed to understand how hospitalist groups can optimally decrease waste in healthcare.

Healthcare associated pneumonia (HCAP) is defined as pneumonia that is present upon admission, and occurs in patients that have recently been hospitalized, reside in a nursing home, or have had other recent healthcare exposures. Practice guidelines developed by the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA), recommend strategies for the diagnosis and treatment of patients with HCAP.1 A premise of the guidelines is that recent healthcare exposure places patients at risk for infection due to multi‐drug resistant (MDR) pathogens such as methicillin‐resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa. In addition to criteria utilized to define HCAP, the guidelines state that recent immunosuppression and antibiotic exposure are risk factors for pneumonia due to MDR pathogens. In contrast to the treatment of community‐acquired pneumonia (CAP), the guidelines recommend empirical administration of antibiotics with activity against MRSA and Pseudomonas aeruginosa for all patients with HCAP.

We recently reported that antimicrobial resistance to CAP antibiotics (CAP‐resistance) was identified in one‐third of culture‐positive patients with HCAP.2 Data regarding the predictive ability of the guideline‐defined criteria specific to HCAP are limited.3 Evaluation and potential refinement of the criteria to identify patients at risk for MDR pathogens can aid in making antibiotic‐related treatment decisions.

The purposes of this study are to: 1) develop and validate a model to predict CAP‐resistance among patients with HCAP, and to compare the model's predictive performance to a model that includes traditional guideline‐defined risk factors; and 2) develop models to predict recovery of pathogen‐specific etiology (MRSA and Pseudomonas aeruginosa), and to compare the predictive performance of the pathogen‐specific and CAP‐resistance models.

METHODS

Patients with HCAP who were admitted to 6 Veterans Affairs Medical Centers (VAMC) in the northwestern United States between January 1, 2003 and December 31, 2008 were included in the retrospective cohort study. The cohort was identified utilizing medical records data extracted from the Veterans Integrated Service Network (VISN20) Data Warehouse. The Data Warehouse is a centralized open architecture relational database that houses medical and administrative records data for VISN20 patients. This research complies with all federal guidelines and VAMC policies relative to human subjects and clinical research.

Subjects were identified by the following pneumonia‐related discharge International Classification of Diseases (ICD‐9 CM) codes: 1) a primary diagnosis of 480‐483; 485‐487.0 (pneumonia); or 2) a primary diagnosis of 507.0 (pneumonitis), 518.8 (respiratory failure), or 0.38 (septicemia), and a secondary diagnosis of 480‐483; 485‐487.0.4 Eligibility required that patients received antibiotic therapy for pneumonia within 24 hours of admission, continue inpatient treatment for >24 hours, and meet any of the following guideline‐defined criteria: 1) hospitalization during the preceding 90 days; 2) admission from a nursing home; 3) outpatient or home wound care, outpatient or home infusion therapy, or chronic hemodialysis.1 In addition, patients not meeting guideline‐defined criteria, who had frequent healthcare system exposure, defined as 12 Emergency Department, Medicine, or Surgery clinic visits within 90 days of admission, were also included. Patients were excluded if they were directly transferred from another hospital, or had pneumonia‐related ICD‐9 codes but received inpatient care for pneumonia in a non‐VA hospital.

Study data included medical records for the year prior to admission for HCAP through 30 days afterwards. Data included: demographics; domicile preceding admission; healthcare utilization including diagnosis and procedure codes; inpatient medications administered, and outpatient prescription fills; vital signs; and laboratory test results, including cultures and susceptibilities.

Guideline‐defined criteria for predicting CAP‐resistance were similar to those used to identify the study cohort. Nursing home admission included patients who were directly admitted from a nursing home, skilled nursing facility, or domiciliary. Prior hospitalization 2 days within 90 days was calculated by summing the length of stay for all admissions during the preceding 90 days. Outpatient intravenous therapy, chronic hemodialysis, and wound care therapy was determined from medication administration records and relevant Current Procedural Terminology (CPT) or ICD‐9 procedure codes for care administered within 30 days. Antibiotic exposure was defined as administration of 1 dose of antibiotic during inpatient care, or fill of an outpatient prescription for 1 antibiotic dose within 90 days preceding admission. Immunosuppression was defined as: human immunodeficiency virus (HIV) diagnosis; white blood cell (WBC) count of 2500 cells/mm³ within 30 days of admission; corticosteroid ingestion during prior admission, or outpatient prescription fills for a corticosteroid with quantity sufficient to last 14 days preceding admission; or inpatient ingestion of, or outpatient prescription fills for, transplant or rheumatologic‐related immunosuppressants within 90 days preceding admission.

Additional variables assessed to predict CAP‐resistance were obtained as follows. First, modifications of guideline‐defined criteria were constructed. These included: direct nursing home admission, or recent nursing home stay preceding admission; total days of hospitalization within 90 days preceding admission; specific antibiotic exposures, including dates since last exposure preceding admission; and individual components of the immunosuppression criterion. Other cohort‐developed variables included: demographics; substance use history; chronic comorbidity determined by individual and composite measures of Charlson score; pulmonary disease history (eg, bronchiectasis); type and frequency of outpatient visits; consecutive (2) prescription fills for chronic medications of interest; clinical and surveillance culture results preceding admission; admitting ward; vital signs; and relevant hematology and chemistry labs.5

Sputum, blood, and bronchoscopy‐collected cultures obtained within 48 hours after admission were assessed to determine specimen acceptability. Poor sputum specimens were defined by Gram stain quantitative results indicating >10 epithelial cells (EPI) per low power field (LPF), or in the absence of quantitative results, semi‐quantitative results indicating 2‐4+EPI. Single positive blood cultures with results indicating likely contaminants were considered poor specimens. All bronchoscopy‐obtained specimens were considered acceptable. All cultures classified as poor specimens were excluded, and microbiology results were evaluated for the remaining specimens.2, 6 Organisms thought to represent colonization or contamination were excluded: coagulase‐negative (CN) Staphylococcus, Enterococcus sp, Bacillus sp, Proprionibacterium sp, and Candida sp. Recovery of a potential pneumonia pathogen from 1 acceptable culture constituted a culture‐positive admission.

CAP‐resistance was determined for each isolate. CAP‐resistance was defined as non‐susceptibility to non‐pseudomonal third generation cephalosporins (ceftriaxone or cefotaxime) or non‐pseudomonal 8‐methoxy fluoroquinolones (moxifloxacin, gatifloxacin), the VA preferred agents for treatment of CAP.7 There were differences between facilities in susceptibility reporting criteria; therefore, the following approach was used to determine CAP‐resistance. First, MRSA and Pseudomonas aeruginosa isolates were classified as CAP‐resistant. Second, susceptibility results were directly utilized to determine CAP‐resistance if both antibiotic results were available. Third, if only a surrogate antibiotic from a class was reported, a representative antibiotic consistent with Clinical Laboratory Standards Institute reporting criteria was utilized.8 Finally, expert rules determined CAP‐resistance for select potential pneumonia pathogens (eg, Haemophilus sp) if antibiotic susceptibility results for both cephalosporin and fluoroquinolone classes were not reported.815 Presence of 1 CAP‐resistant isolate resulted in a CAP‐resistant classification for an admission. MRSA and Pseudomonas aeruginosa endpoints were defined in a similar manner. Only the first admission for each patient was utilized in the analysis.

The probability of CAP‐resistance was predicted from guideline‐defined criteria (guideline‐defined model) with logistic regression. Next, non‐guideline variables were classified as high, medium, or low interest for association with CAP‐resistance. Variables were assessed for collinearity. A model of CAP‐resistance was developed from variables of high interest. Guideline‐defined criteria were omitted to allow consideration of more specific measures (eg, specific antibiotic exposures as opposed to receipt of antibiotics within the preceding 90 days) during this stage. Next, guideline‐defined criteria, and subsequently variables of lesser interest, were added in an attempt to improve the model. Annual trends and plausible interactions were considered. Model selection was by Akaike's Information Criterion (AIC).16 To promote model reliability, the final model was required to lack evidence of over‐fitting in bootstrapped internal validation.17 The guideline‐defined and cohort‐developed models were compared by difference in area under receiver operating characteristic (aROC) curves. The model development process was repeated for MRSA and Pseudomonas aeruginosa endpoints. Finally, to determine if the CAP‐resistance model sufficiently predicted pathogen‐specific MDR, the CAP‐resistance model was re‐estimated for MRSA and Pseudomonas aeruginosa endpoints. Statistical analysis was performed with R version 2.10.0 (The R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

The cohort was comprised of 1300 patients with HCAP. Of these, 375 (28.8% [26.4‐31.4]) met culture‐positive criteria for potential pneumonia pathogens. CAP‐resistant organisms were identified in 118 (31.5% [26.8‐36.4]) patients within 48 hours of admission. CAP‐resistant organisms included: MRSA (49.2% [40.4‐58.1]), Pseudomonas aeruginosa (29.5% [21.9‐38.1]), Enterobacteriaceae (11.4% [6.5‐18.0]), Gram‐negative non‐enterics (8.3% [4.2‐14.4]), Streptococcus pneumoniae (1.5% [0.2‐5.4]), and opportunistic organisms (eg, Mycobacterium spp) (8.3% [4.2‐14.4]). Differences in select characteristics and exposures between culture‐positive and culture‐negative admissions, as well as CAP‐resistant and CAP‐sensitive admissions, were evident (Table 1).

Of the guideline‐defined criteria, direct admission from a nursing home, prior hospitalization, and recent antibiotic exposure were associated with CAP‐resistance (Table 2). The cohort‐derived CAP‐resistance model included 6 variables. Prior MRSA colonization or infection within 90 days preceding admission was strongly predictive of CAP‐resistance. A composite variable consisting of direct admission from a nursing home or admission from the community after recent discharge from a nursing home was more predictive than direct admission from a nursing home alone. Exposure to cephalosporin antibiotics within the prior year was also predictive of CAP‐resistance. Subcategorizing cephalosporins by class or by most recent exposure in 90‐day increments did not improve the model. The remaining predictors in the model were guideline‐defined infusion therapy criterion, diabetes, and intensive care unit (ICU) admission.

Of the guideline‐defined criteria, direct admission from a nursing home was most predictive of MRSA HCAP (n = 57), followed by prior hospitalization and recent antibiotic exposure (Table 3). The cohort‐developed model of MRSA HCAP included predictors common to the CAP‐resistance model: direct admission from a nursing home or patients who were recently discharged from a nursing home, history of prior MRSA, and diabetes. Positive MRSA status within 90 days preceding admission exhibited the strongest prediction of MRSA HCAP. Exposure to anti‐pseudomonal fluoroquinolones (ciprofloxacin and levofloxacin) within the prior year was also predictive of MRSA HCAP, however, exposure to 8‐methoxy fluoroquinolone was not (crude odds ratio (OR) = 0.7 [0.3‐1.4]; final model adjusted OR = 0.6 [0.2‐1.2]). Exposure to third generation cephalosporins within the previous year was more predictive than other cephalosporin exposures, and more predictive than exposure times categorized in 90‐day increments.

Of the guideline‐defined criteria, only prior hospitalization within 90 days and admission from a nursing home were predictive of Pseudomonas aeruginosa HCAP (n = 36) (Table 4). In the cohort‐developed model of Pseudomonas aeruginosa HCAP, Pseudomonas aeruginosa was predicted by prior cephalosporin exposure within the preceding year, prior culture of Pseudomonas aeruginosa from any anatomical source within the preceding year, and chronic steroid use of 10 mg/day prednisone equivalents. Again, the model was not improved by subcategorizing cephalosporin by class or by most recent exposure time. Finally, a negative annual trend in Pseudomonas aeruginosa HCAP was evident.

The cohort‐developed model of CAP‐resistance was re‐estimated for MRSA and Pseudomonas aeruginosa endpoints. Only positive MRSA status within 90 days preceding admission was associated with both endpoints (OR = 8.7 [3.5‐22.1] for MRSA; OR = 4.3 [1.4‐12.2] for Pseudomonas aeruginosa). Direct or recent nursing home residence (OR = 2.4 [1.3‐4.6]) and diabetes (OR = 2.4 [1.3‐4.5]) were highly predictive of MRSA, but not Pseudomonas aeruginosa (OR = 1.8 [0.8‐3.9] for nursing home residence; OR = 1.3 [0.6‐2.7] for diabetes), respectively. Cephalosporin exposure preceding admission was highly predictive of Pseudomonas aeruginosa (OR = 4.0 [1.9‐9.3]), but not with MRSA (OR = 1.1 [0.6‐2.1]). In these models, all estimated odds ratios were >1.0, consistent with the cohort‐developed model of CAP‐resistance.

For each endpoint, the cohort‐developed model was more predictive than the guideline‐defined model (Table 5) (to view ROC curves see Supporting Figures 1 to 3 in the online version of the article.). The cohort‐developed model of CAP‐resistance re‐estimated for pathogen‐specific endpoints resulted in similar predictive performance. To assess performance of the cohort developed models by facility, aROC was calculated for each of the 3 larger sites separately and for the 3 smaller facilities combined due to limited counts. Site specific aROC ranged from 0.652 to 0.762 for CAP‐resistance, 0.725 to 0.815 for MRSA, and 0.719 to 0.801 for Pseudomonas aeruginosa. The cohort‐developed model of CAP‐resistance re‐estimated for pathogen‐specific endpoints resulted in similar predictive performance.

A nomogram for the cohort‐developed model of CAP‐resistance can provide the predicted probability of culturing a CAP‐resistant organism for an individual patient (Table 6). Point scores assigned to levels of variables, are summed to obtain a total score, and the total score corresponds to a predicted probability of CAP‐resistance. The prevalence of CAP‐resistance (%) from highest to lowest quartile of predicted probability was 92.9, 58.8, 32.9, and 18.5, respectively.

DISCUSSION

In this study, select ATS/IDSA guideline‐defined criteria predicted identification of CAP‐resistant organisms in patients with HCAP. Admission from a nursing home was most predictive of CAP‐resistant organisms, whereas recent hospitalization and antibiotic exposure were predictive to a lesser extent. There was weak evidence of associations between recent infusion and chronic hemodialysis criteria with MDR endpoints. Recent wound care and a composite definition of immunosuppression were not predictive of these endpoints.

The cohort‐developed model resulted in improved prediction of CAP‐resistance endpoints. Culture history, particularly history of MRSA within 90 days preceding admission, was a strong predictor of MDR endpoints. The MRSA history variable definition included cultures from all anatomical sources and nares polymerase chain reaction surveillance results, the latter increasing in 2007‐2008 due to the implementation of the VA MRSA initiative.18 This finding suggests that prior culture results should be considered when selecting empirical antimicrobial therapy, and the rapid proliferation of electronic medical records increases potential to utilize this information routinely. While the guideline‐defined nursing home admission criterion was a strong predictor of CAP‐resistance, admission from the community after recent discharge from a nursing home, in addition to direct admission from a nursing home, was also important.

Similarities in variables included in the pathogen‐specific and CAP‐resistance models reflect the importance of MRSA in defining the CAP‐resistance endpoint. Both CAP‐resistance and MRSA models included prior MRSA status, diabetes, and ICU admission, whereas cephalosporin exposure was common to the Pseudomonas aeruginosa and CAP‐resistance models. Annual trends in CAP‐resistance and MRSA recovery were not identified. The negative annual trend in Pseudomonas aeruginosa HCAP is unexplained and beyond the scope of this study. The percentage of culture‐positive admissions with Pseudomonas aeruginosa HCAP averaged 12% in 2003‐2006, but dropped to <5% in 2007‐2008. A potential explanation is that identification and isolation of patients with MRSA, as a result of the VA‐wide MRSA initiative, may have impacted Pseudomonas aeruginosa colonization by isolating patients co‐colonized with these pathogens during prior healthcare exposures. This is consistent with the observation that when the cohort‐derived CAP‐resistance model was refit with the Pseudomonas aeruginosa endpoint, recent MRSA colonization was strongly predictive of Pseudomonas aeruginosa. Despite differences between variables in pathogen‐specific and CAP‐resistant models, the CAP‐resistance model provided a similar degree of MRSA and Pseudomonas aeruginosa prediction. Finally, as a study purpose included developing best predictive models for each endpoint, and not merely identifying associations, there were other plausible models not reported.

Study strengths included use of the VISN20 Data Warehouse, which provided an integrated outpatient and inpatient medical record. This facilitated analysis of prior healthcare exposures and inpatient study endpoints. In addition, poor blood and sputum specimens and unlikely pneumonia pathogens were not included in establishing MDR endpoints. The variable set explored in regression modeling was extensive and detailed, and analysis included time and intensity‐based components of the variables. Importantly, a standardized approach to regression modeling was specified in advance, which included identification of variables with high potential for association with MDR endpoints, model selection by AIC, re‐evaluation of guideline‐defined criteria and variables of lower interest, and bootstrapped internal model validation.19

Study limitations included the use of ICD‐9 codes to establish a pneumonia diagnosis, which may lack sensitivity and specificity. However, an enhanced ICD‐9based algorithm superior to other claims‐based definitions of pneumonia was utilized.4, 20 Veterans may have received care at non‐VA facilities impacting identification of all healthcare system exposures preceding admission. Data for microbial endpoints were obtained from sterile and non‐sterile site cultures, and it was not possible to determine if the cultured organisms were truly pathogenic. While pathogen‐specific endpoints were not affected, the use of expert rules in select cases to establish CAP‐resistance may have impacted precision for this endpoint. It is also possible that refitting the cohort‐developed CAP‐resistance model for pathogen‐specific endpoints resulted in optimistic aROC due to model over‐fitting. Finally, the cohort was comprised of elderly males, and caution is warranted in extrapolating the results to other populations.

The predictive ability of the guideline‐defined criteria to identify patients with MDR pathogens has been studied. A prospective observational cohort study of 625 consecutive ICU admissions determined that the guideline‐defined criteriaprior antimicrobial treatment, nursing home residence, and prior hospitalizationwere associated with recovery of MDR colonization.21 Shorr et al., investigating a retrospective cohort of 619 patients with HCAP, reported that recent hospitalization, nursing home residence, hemodialysis, and ICU admission were associated with infections caused by CAP‐resistant organisms.22 This study did not report antimicrobial exposures. Our study complements these studies by evaluating existing HCAP guideline criteria, and identifying specific antibiotic exposure, prior culture data, comorbid illness, and immunosuppressive medications that are predictive of MDR infection.

Studies comparing the bacterial etiology of patients with pneumonia in nursing homes relative to CAP, have demonstrated mixed results in recovery of Gram‐negative MDR pathogens, but generally increased MRSA pneumonia.3 Our study suggests that a nursing home stay in the last 6 months is associated with an increased risk for MRSA, but not Pseudomonas aeruginosa, although this was limited by small sample size. Recent infusion therapy has not been previously reported to be associated with MDR pathogens in an HCAP population. In our study, this criterion was predictive of CAP‐resistance in the cohort‐developed model, but not in conjunction with other variables in the guideline‐defined model. Predictors of pathogen‐specific HCAP are limited to an aforementioned single prior study, which identified recent hospitalization, nursing home residence, and ICU admission as risk factors for MRSA HCAP.22

Many studies have investigated risks for infection with MRSA and Pseudomonas aeruginosa outside of the context of HCAP. Predictor variables in cohort‐developed pathogen‐specific models in our study are known risk factors for colonization or infection with these pathogens. For example, antecedent MRSA colonization has been noted as a strong risk factor for MRSA infection, particularly pneumonia.23, 24 Further, patients with diabetes and inhaled corticosteroid exposure are immunosuppressed and at increased risk for colonization with MRSA.25, 26 Likewise, bronchiolar colonization and corticosteroid exposures are known risk factors for pneumonia due to Pseudomonas aeruginosa.27

Many studies have identified prior antibiotic use as a risk factor for infections caused by MRSA and Pseudomonas aeruginosa. However, this criterion is excessively broad and specific antimicrobial exposures carry different magnitudes of risk. Third generation cephalosporins and anti‐pseudomonal fluoroquinolones are commonly reported antibiotics associated with risk for MRSA infection, whereas 8‐methoxy fluoroquinolones appear not to possess the same effect.2831 Likewise, cephalosporins have been reported as risk factors for MDR Pseudomonas aeruginosa infections.32

Several areas of research involving HCAP MDR risk should be investigated. First, the predictive models developed in our and other studies should be evaluated in larger, more diverse populations to establish generalizability. Second, empirical broad‐spectrum antibiotic therapy in all patients with HCAP results in overtreatment of many patients. To date, no reported models provided optimal performance for selecting empirical therapy for unstable ICU patients with HCAP, and many patients do not receive de‐escalation therapy. Thus, models to identify patients with low probability of MDR pathogens upon admission and to aid in de‐escalation are warranted. Finally, the negative trend in Pseudomonas aeruginosa HCAP requires confirmation and further study.

In conclusion, of the ATS/IDSA guideline‐defined criteria for MDR, nursing home admission, recent hospitalization, and antibiotic exposure were predictive of the recovery of CAP‐resistant organisms. Alternative models primarily based on prior culture data, specific antibiotic exposures, and immunosuppression‐related variables improved predictive performance of HCAP associated with MDR.

Healthcare associated pneumonia (HCAP) is defined as pneumonia that is present upon admission, and occurs in patients that have recently been hospitalized, reside in a nursing home, or have had other recent healthcare exposures. Practice guidelines developed by the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA), recommend strategies for the diagnosis and treatment of patients with HCAP.1 A premise of the guidelines is that recent healthcare exposure places patients at risk for infection due to multi‐drug resistant (MDR) pathogens such as methicillin‐resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa. In addition to criteria utilized to define HCAP, the guidelines state that recent immunosuppression and antibiotic exposure are risk factors for pneumonia due to MDR pathogens. In contrast to the treatment of community‐acquired pneumonia (CAP), the guidelines recommend empirical administration of antibiotics with activity against MRSA and Pseudomonas aeruginosa for all patients with HCAP.

We recently reported that antimicrobial resistance to CAP antibiotics (CAP‐resistance) was identified in one‐third of culture‐positive patients with HCAP.2 Data regarding the predictive ability of the guideline‐defined criteria specific to HCAP are limited.3 Evaluation and potential refinement of the criteria to identify patients at risk for MDR pathogens can aid in making antibiotic‐related treatment decisions.

The purposes of this study are to: 1) develop and validate a model to predict CAP‐resistance among patients with HCAP, and to compare the model's predictive performance to a model that includes traditional guideline‐defined risk factors; and 2) develop models to predict recovery of pathogen‐specific etiology (MRSA and Pseudomonas aeruginosa), and to compare the predictive performance of the pathogen‐specific and CAP‐resistance models.

METHODS

Patients with HCAP who were admitted to 6 Veterans Affairs Medical Centers (VAMC) in the northwestern United States between January 1, 2003 and December 31, 2008 were included in the retrospective cohort study. The cohort was identified utilizing medical records data extracted from the Veterans Integrated Service Network (VISN20) Data Warehouse. The Data Warehouse is a centralized open architecture relational database that houses medical and administrative records data for VISN20 patients. This research complies with all federal guidelines and VAMC policies relative to human subjects and clinical research.

Subjects were identified by the following pneumonia‐related discharge International Classification of Diseases (ICD‐9 CM) codes: 1) a primary diagnosis of 480‐483; 485‐487.0 (pneumonia); or 2) a primary diagnosis of 507.0 (pneumonitis), 518.8 (respiratory failure), or 0.38 (septicemia), and a secondary diagnosis of 480‐483; 485‐487.0.4 Eligibility required that patients received antibiotic therapy for pneumonia within 24 hours of admission, continue inpatient treatment for >24 hours, and meet any of the following guideline‐defined criteria: 1) hospitalization during the preceding 90 days; 2) admission from a nursing home; 3) outpatient or home wound care, outpatient or home infusion therapy, or chronic hemodialysis.1 In addition, patients not meeting guideline‐defined criteria, who had frequent healthcare system exposure, defined as 12 Emergency Department, Medicine, or Surgery clinic visits within 90 days of admission, were also included. Patients were excluded if they were directly transferred from another hospital, or had pneumonia‐related ICD‐9 codes but received inpatient care for pneumonia in a non‐VA hospital.

Study data included medical records for the year prior to admission for HCAP through 30 days afterwards. Data included: demographics; domicile preceding admission; healthcare utilization including diagnosis and procedure codes; inpatient medications administered, and outpatient prescription fills; vital signs; and laboratory test results, including cultures and susceptibilities.

Guideline‐defined criteria for predicting CAP‐resistance were similar to those used to identify the study cohort. Nursing home admission included patients who were directly admitted from a nursing home, skilled nursing facility, or domiciliary. Prior hospitalization 2 days within 90 days was calculated by summing the length of stay for all admissions during the preceding 90 days. Outpatient intravenous therapy, chronic hemodialysis, and wound care therapy was determined from medication administration records and relevant Current Procedural Terminology (CPT) or ICD‐9 procedure codes for care administered within 30 days. Antibiotic exposure was defined as administration of 1 dose of antibiotic during inpatient care, or fill of an outpatient prescription for 1 antibiotic dose within 90 days preceding admission. Immunosuppression was defined as: human immunodeficiency virus (HIV) diagnosis; white blood cell (WBC) count of 2500 cells/mm³ within 30 days of admission; corticosteroid ingestion during prior admission, or outpatient prescription fills for a corticosteroid with quantity sufficient to last 14 days preceding admission; or inpatient ingestion of, or outpatient prescription fills for, transplant or rheumatologic‐related immunosuppressants within 90 days preceding admission.

Additional variables assessed to predict CAP‐resistance were obtained as follows. First, modifications of guideline‐defined criteria were constructed. These included: direct nursing home admission, or recent nursing home stay preceding admission; total days of hospitalization within 90 days preceding admission; specific antibiotic exposures, including dates since last exposure preceding admission; and individual components of the immunosuppression criterion. Other cohort‐developed variables included: demographics; substance use history; chronic comorbidity determined by individual and composite measures of Charlson score; pulmonary disease history (eg, bronchiectasis); type and frequency of outpatient visits; consecutive (2) prescription fills for chronic medications of interest; clinical and surveillance culture results preceding admission; admitting ward; vital signs; and relevant hematology and chemistry labs.5

Sputum, blood, and bronchoscopy‐collected cultures obtained within 48 hours after admission were assessed to determine specimen acceptability. Poor sputum specimens were defined by Gram stain quantitative results indicating >10 epithelial cells (EPI) per low power field (LPF), or in the absence of quantitative results, semi‐quantitative results indicating 2‐4+EPI. Single positive blood cultures with results indicating likely contaminants were considered poor specimens. All bronchoscopy‐obtained specimens were considered acceptable. All cultures classified as poor specimens were excluded, and microbiology results were evaluated for the remaining specimens.2, 6 Organisms thought to represent colonization or contamination were excluded: coagulase‐negative (CN) Staphylococcus, Enterococcus sp, Bacillus sp, Proprionibacterium sp, and Candida sp. Recovery of a potential pneumonia pathogen from 1 acceptable culture constituted a culture‐positive admission.

CAP‐resistance was determined for each isolate. CAP‐resistance was defined as non‐susceptibility to non‐pseudomonal third generation cephalosporins (ceftriaxone or cefotaxime) or non‐pseudomonal 8‐methoxy fluoroquinolones (moxifloxacin, gatifloxacin), the VA preferred agents for treatment of CAP.7 There were differences between facilities in susceptibility reporting criteria; therefore, the following approach was used to determine CAP‐resistance. First, MRSA and Pseudomonas aeruginosa isolates were classified as CAP‐resistant. Second, susceptibility results were directly utilized to determine CAP‐resistance if both antibiotic results were available. Third, if only a surrogate antibiotic from a class was reported, a representative antibiotic consistent with Clinical Laboratory Standards Institute reporting criteria was utilized.8 Finally, expert rules determined CAP‐resistance for select potential pneumonia pathogens (eg, Haemophilus sp) if antibiotic susceptibility results for both cephalosporin and fluoroquinolone classes were not reported.815 Presence of 1 CAP‐resistant isolate resulted in a CAP‐resistant classification for an admission. MRSA and Pseudomonas aeruginosa endpoints were defined in a similar manner. Only the first admission for each patient was utilized in the analysis.

The probability of CAP‐resistance was predicted from guideline‐defined criteria (guideline‐defined model) with logistic regression. Next, non‐guideline variables were classified as high, medium, or low interest for association with CAP‐resistance. Variables were assessed for collinearity. A model of CAP‐resistance was developed from variables of high interest. Guideline‐defined criteria were omitted to allow consideration of more specific measures (eg, specific antibiotic exposures as opposed to receipt of antibiotics within the preceding 90 days) during this stage. Next, guideline‐defined criteria, and subsequently variables of lesser interest, were added in an attempt to improve the model. Annual trends and plausible interactions were considered. Model selection was by Akaike's Information Criterion (AIC).16 To promote model reliability, the final model was required to lack evidence of over‐fitting in bootstrapped internal validation.17 The guideline‐defined and cohort‐developed models were compared by difference in area under receiver operating characteristic (aROC) curves. The model development process was repeated for MRSA and Pseudomonas aeruginosa endpoints. Finally, to determine if the CAP‐resistance model sufficiently predicted pathogen‐specific MDR, the CAP‐resistance model was re‐estimated for MRSA and Pseudomonas aeruginosa endpoints. Statistical analysis was performed with R version 2.10.0 (The R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

The cohort was comprised of 1300 patients with HCAP. Of these, 375 (28.8% [26.4‐31.4]) met culture‐positive criteria for potential pneumonia pathogens. CAP‐resistant organisms were identified in 118 (31.5% [26.8‐36.4]) patients within 48 hours of admission. CAP‐resistant organisms included: MRSA (49.2% [40.4‐58.1]), Pseudomonas aeruginosa (29.5% [21.9‐38.1]), Enterobacteriaceae (11.4% [6.5‐18.0]), Gram‐negative non‐enterics (8.3% [4.2‐14.4]), Streptococcus pneumoniae (1.5% [0.2‐5.4]), and opportunistic organisms (eg, Mycobacterium spp) (8.3% [4.2‐14.4]). Differences in select characteristics and exposures between culture‐positive and culture‐negative admissions, as well as CAP‐resistant and CAP‐sensitive admissions, were evident (Table 1).

Of the guideline‐defined criteria, direct admission from a nursing home, prior hospitalization, and recent antibiotic exposure were associated with CAP‐resistance (Table 2). The cohort‐derived CAP‐resistance model included 6 variables. Prior MRSA colonization or infection within 90 days preceding admission was strongly predictive of CAP‐resistance. A composite variable consisting of direct admission from a nursing home or admission from the community after recent discharge from a nursing home was more predictive than direct admission from a nursing home alone. Exposure to cephalosporin antibiotics within the prior year was also predictive of CAP‐resistance. Subcategorizing cephalosporins by class or by most recent exposure in 90‐day increments did not improve the model. The remaining predictors in the model were guideline‐defined infusion therapy criterion, diabetes, and intensive care unit (ICU) admission.

Of the guideline‐defined criteria, direct admission from a nursing home was most predictive of MRSA HCAP (n = 57), followed by prior hospitalization and recent antibiotic exposure (Table 3). The cohort‐developed model of MRSA HCAP included predictors common to the CAP‐resistance model: direct admission from a nursing home or patients who were recently discharged from a nursing home, history of prior MRSA, and diabetes. Positive MRSA status within 90 days preceding admission exhibited the strongest prediction of MRSA HCAP. Exposure to anti‐pseudomonal fluoroquinolones (ciprofloxacin and levofloxacin) within the prior year was also predictive of MRSA HCAP, however, exposure to 8‐methoxy fluoroquinolone was not (crude odds ratio (OR) = 0.7 [0.3‐1.4]; final model adjusted OR = 0.6 [0.2‐1.2]). Exposure to third generation cephalosporins within the previous year was more predictive than other cephalosporin exposures, and more predictive than exposure times categorized in 90‐day increments.

Of the guideline‐defined criteria, only prior hospitalization within 90 days and admission from a nursing home were predictive of Pseudomonas aeruginosa HCAP (n = 36) (Table 4). In the cohort‐developed model of Pseudomonas aeruginosa HCAP, Pseudomonas aeruginosa was predicted by prior cephalosporin exposure within the preceding year, prior culture of Pseudomonas aeruginosa from any anatomical source within the preceding year, and chronic steroid use of 10 mg/day prednisone equivalents. Again, the model was not improved by subcategorizing cephalosporin by class or by most recent exposure time. Finally, a negative annual trend in Pseudomonas aeruginosa HCAP was evident.

The cohort‐developed model of CAP‐resistance was re‐estimated for MRSA and Pseudomonas aeruginosa endpoints. Only positive MRSA status within 90 days preceding admission was associated with both endpoints (OR = 8.7 [3.5‐22.1] for MRSA; OR = 4.3 [1.4‐12.2] for Pseudomonas aeruginosa). Direct or recent nursing home residence (OR = 2.4 [1.3‐4.6]) and diabetes (OR = 2.4 [1.3‐4.5]) were highly predictive of MRSA, but not Pseudomonas aeruginosa (OR = 1.8 [0.8‐3.9] for nursing home residence; OR = 1.3 [0.6‐2.7] for diabetes), respectively. Cephalosporin exposure preceding admission was highly predictive of Pseudomonas aeruginosa (OR = 4.0 [1.9‐9.3]), but not with MRSA (OR = 1.1 [0.6‐2.1]). In these models, all estimated odds ratios were >1.0, consistent with the cohort‐developed model of CAP‐resistance.

For each endpoint, the cohort‐developed model was more predictive than the guideline‐defined model (Table 5) (to view ROC curves see Supporting Figures 1 to 3 in the online version of the article.). The cohort‐developed model of CAP‐resistance re‐estimated for pathogen‐specific endpoints resulted in similar predictive performance. To assess performance of the cohort developed models by facility, aROC was calculated for each of the 3 larger sites separately and for the 3 smaller facilities combined due to limited counts. Site specific aROC ranged from 0.652 to 0.762 for CAP‐resistance, 0.725 to 0.815 for MRSA, and 0.719 to 0.801 for Pseudomonas aeruginosa. The cohort‐developed model of CAP‐resistance re‐estimated for pathogen‐specific endpoints resulted in similar predictive performance.

A nomogram for the cohort‐developed model of CAP‐resistance can provide the predicted probability of culturing a CAP‐resistant organism for an individual patient (Table 6). Point scores assigned to levels of variables, are summed to obtain a total score, and the total score corresponds to a predicted probability of CAP‐resistance. The prevalence of CAP‐resistance (%) from highest to lowest quartile of predicted probability was 92.9, 58.8, 32.9, and 18.5, respectively.

DISCUSSION

In this study, select ATS/IDSA guideline‐defined criteria predicted identification of CAP‐resistant organisms in patients with HCAP. Admission from a nursing home was most predictive of CAP‐resistant organisms, whereas recent hospitalization and antibiotic exposure were predictive to a lesser extent. There was weak evidence of associations between recent infusion and chronic hemodialysis criteria with MDR endpoints. Recent wound care and a composite definition of immunosuppression were not predictive of these endpoints.

The cohort‐developed model resulted in improved prediction of CAP‐resistance endpoints. Culture history, particularly history of MRSA within 90 days preceding admission, was a strong predictor of MDR endpoints. The MRSA history variable definition included cultures from all anatomical sources and nares polymerase chain reaction surveillance results, the latter increasing in 2007‐2008 due to the implementation of the VA MRSA initiative.18 This finding suggests that prior culture results should be considered when selecting empirical antimicrobial therapy, and the rapid proliferation of electronic medical records increases potential to utilize this information routinely. While the guideline‐defined nursing home admission criterion was a strong predictor of CAP‐resistance, admission from the community after recent discharge from a nursing home, in addition to direct admission from a nursing home, was also important.

Similarities in variables included in the pathogen‐specific and CAP‐resistance models reflect the importance of MRSA in defining the CAP‐resistance endpoint. Both CAP‐resistance and MRSA models included prior MRSA status, diabetes, and ICU admission, whereas cephalosporin exposure was common to the Pseudomonas aeruginosa and CAP‐resistance models. Annual trends in CAP‐resistance and MRSA recovery were not identified. The negative annual trend in Pseudomonas aeruginosa HCAP is unexplained and beyond the scope of this study. The percentage of culture‐positive admissions with Pseudomonas aeruginosa HCAP averaged 12% in 2003‐2006, but dropped to <5% in 2007‐2008. A potential explanation is that identification and isolation of patients with MRSA, as a result of the VA‐wide MRSA initiative, may have impacted Pseudomonas aeruginosa colonization by isolating patients co‐colonized with these pathogens during prior healthcare exposures. This is consistent with the observation that when the cohort‐derived CAP‐resistance model was refit with the Pseudomonas aeruginosa endpoint, recent MRSA colonization was strongly predictive of Pseudomonas aeruginosa. Despite differences between variables in pathogen‐specific and CAP‐resistant models, the CAP‐resistance model provided a similar degree of MRSA and Pseudomonas aeruginosa prediction. Finally, as a study purpose included developing best predictive models for each endpoint, and not merely identifying associations, there were other plausible models not reported.

Study strengths included use of the VISN20 Data Warehouse, which provided an integrated outpatient and inpatient medical record. This facilitated analysis of prior healthcare exposures and inpatient study endpoints. In addition, poor blood and sputum specimens and unlikely pneumonia pathogens were not included in establishing MDR endpoints. The variable set explored in regression modeling was extensive and detailed, and analysis included time and intensity‐based components of the variables. Importantly, a standardized approach to regression modeling was specified in advance, which included identification of variables with high potential for association with MDR endpoints, model selection by AIC, re‐evaluation of guideline‐defined criteria and variables of lower interest, and bootstrapped internal model validation.19

Study limitations included the use of ICD‐9 codes to establish a pneumonia diagnosis, which may lack sensitivity and specificity. However, an enhanced ICD‐9based algorithm superior to other claims‐based definitions of pneumonia was utilized.4, 20 Veterans may have received care at non‐VA facilities impacting identification of all healthcare system exposures preceding admission. Data for microbial endpoints were obtained from sterile and non‐sterile site cultures, and it was not possible to determine if the cultured organisms were truly pathogenic. While pathogen‐specific endpoints were not affected, the use of expert rules in select cases to establish CAP‐resistance may have impacted precision for this endpoint. It is also possible that refitting the cohort‐developed CAP‐resistance model for pathogen‐specific endpoints resulted in optimistic aROC due to model over‐fitting. Finally, the cohort was comprised of elderly males, and caution is warranted in extrapolating the results to other populations.

The predictive ability of the guideline‐defined criteria to identify patients with MDR pathogens has been studied. A prospective observational cohort study of 625 consecutive ICU admissions determined that the guideline‐defined criteriaprior antimicrobial treatment, nursing home residence, and prior hospitalizationwere associated with recovery of MDR colonization.21 Shorr et al., investigating a retrospective cohort of 619 patients with HCAP, reported that recent hospitalization, nursing home residence, hemodialysis, and ICU admission were associated with infections caused by CAP‐resistant organisms.22 This study did not report antimicrobial exposures. Our study complements these studies by evaluating existing HCAP guideline criteria, and identifying specific antibiotic exposure, prior culture data, comorbid illness, and immunosuppressive medications that are predictive of MDR infection.

Studies comparing the bacterial etiology of patients with pneumonia in nursing homes relative to CAP, have demonstrated mixed results in recovery of Gram‐negative MDR pathogens, but generally increased MRSA pneumonia.3 Our study suggests that a nursing home stay in the last 6 months is associated with an increased risk for MRSA, but not Pseudomonas aeruginosa, although this was limited by small sample size. Recent infusion therapy has not been previously reported to be associated with MDR pathogens in an HCAP population. In our study, this criterion was predictive of CAP‐resistance in the cohort‐developed model, but not in conjunction with other variables in the guideline‐defined model. Predictors of pathogen‐specific HCAP are limited to an aforementioned single prior study, which identified recent hospitalization, nursing home residence, and ICU admission as risk factors for MRSA HCAP.22

Many studies have investigated risks for infection with MRSA and Pseudomonas aeruginosa outside of the context of HCAP. Predictor variables in cohort‐developed pathogen‐specific models in our study are known risk factors for colonization or infection with these pathogens. For example, antecedent MRSA colonization has been noted as a strong risk factor for MRSA infection, particularly pneumonia.23, 24 Further, patients with diabetes and inhaled corticosteroid exposure are immunosuppressed and at increased risk for colonization with MRSA.25, 26 Likewise, bronchiolar colonization and corticosteroid exposures are known risk factors for pneumonia due to Pseudomonas aeruginosa.27

Many studies have identified prior antibiotic use as a risk factor for infections caused by MRSA and Pseudomonas aeruginosa. However, this criterion is excessively broad and specific antimicrobial exposures carry different magnitudes of risk. Third generation cephalosporins and anti‐pseudomonal fluoroquinolones are commonly reported antibiotics associated with risk for MRSA infection, whereas 8‐methoxy fluoroquinolones appear not to possess the same effect.2831 Likewise, cephalosporins have been reported as risk factors for MDR Pseudomonas aeruginosa infections.32

Several areas of research involving HCAP MDR risk should be investigated. First, the predictive models developed in our and other studies should be evaluated in larger, more diverse populations to establish generalizability. Second, empirical broad‐spectrum antibiotic therapy in all patients with HCAP results in overtreatment of many patients. To date, no reported models provided optimal performance for selecting empirical therapy for unstable ICU patients with HCAP, and many patients do not receive de‐escalation therapy. Thus, models to identify patients with low probability of MDR pathogens upon admission and to aid in de‐escalation are warranted. Finally, the negative trend in Pseudomonas aeruginosa HCAP requires confirmation and further study.

In conclusion, of the ATS/IDSA guideline‐defined criteria for MDR, nursing home admission, recent hospitalization, and antibiotic exposure were predictive of the recovery of CAP‐resistant organisms. Alternative models primarily based on prior culture data, specific antibiotic exposures, and immunosuppression‐related variables improved predictive performance of HCAP associated with MDR.

Healthcare associated pneumonia (HCAP) is defined as pneumonia that is present upon admission, and occurs in patients that have recently been hospitalized, reside in a nursing home, or have had other recent healthcare exposures. Practice guidelines developed by the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA), recommend strategies for the diagnosis and treatment of patients with HCAP.1 A premise of the guidelines is that recent healthcare exposure places patients at risk for infection due to multi‐drug resistant (MDR) pathogens such as methicillin‐resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa. In addition to criteria utilized to define HCAP, the guidelines state that recent immunosuppression and antibiotic exposure are risk factors for pneumonia due to MDR pathogens. In contrast to the treatment of community‐acquired pneumonia (CAP), the guidelines recommend empirical administration of antibiotics with activity against MRSA and Pseudomonas aeruginosa for all patients with HCAP.

We recently reported that antimicrobial resistance to CAP antibiotics (CAP‐resistance) was identified in one‐third of culture‐positive patients with HCAP.2 Data regarding the predictive ability of the guideline‐defined criteria specific to HCAP are limited.3 Evaluation and potential refinement of the criteria to identify patients at risk for MDR pathogens can aid in making antibiotic‐related treatment decisions.

The purposes of this study are to: 1) develop and validate a model to predict CAP‐resistance among patients with HCAP, and to compare the model's predictive performance to a model that includes traditional guideline‐defined risk factors; and 2) develop models to predict recovery of pathogen‐specific etiology (MRSA and Pseudomonas aeruginosa), and to compare the predictive performance of the pathogen‐specific and CAP‐resistance models.

METHODS

Patients with HCAP who were admitted to 6 Veterans Affairs Medical Centers (VAMC) in the northwestern United States between January 1, 2003 and December 31, 2008 were included in the retrospective cohort study. The cohort was identified utilizing medical records data extracted from the Veterans Integrated Service Network (VISN20) Data Warehouse. The Data Warehouse is a centralized open architecture relational database that houses medical and administrative records data for VISN20 patients. This research complies with all federal guidelines and VAMC policies relative to human subjects and clinical research.

Subjects were identified by the following pneumonia‐related discharge International Classification of Diseases (ICD‐9 CM) codes: 1) a primary diagnosis of 480‐483; 485‐487.0 (pneumonia); or 2) a primary diagnosis of 507.0 (pneumonitis), 518.8 (respiratory failure), or 0.38 (septicemia), and a secondary diagnosis of 480‐483; 485‐487.0.4 Eligibility required that patients received antibiotic therapy for pneumonia within 24 hours of admission, continue inpatient treatment for >24 hours, and meet any of the following guideline‐defined criteria: 1) hospitalization during the preceding 90 days; 2) admission from a nursing home; 3) outpatient or home wound care, outpatient or home infusion therapy, or chronic hemodialysis.1 In addition, patients not meeting guideline‐defined criteria, who had frequent healthcare system exposure, defined as 12 Emergency Department, Medicine, or Surgery clinic visits within 90 days of admission, were also included. Patients were excluded if they were directly transferred from another hospital, or had pneumonia‐related ICD‐9 codes but received inpatient care for pneumonia in a non‐VA hospital.

Study data included medical records for the year prior to admission for HCAP through 30 days afterwards. Data included: demographics; domicile preceding admission; healthcare utilization including diagnosis and procedure codes; inpatient medications administered, and outpatient prescription fills; vital signs; and laboratory test results, including cultures and susceptibilities.

Guideline‐defined criteria for predicting CAP‐resistance were similar to those used to identify the study cohort. Nursing home admission included patients who were directly admitted from a nursing home, skilled nursing facility, or domiciliary. Prior hospitalization 2 days within 90 days was calculated by summing the length of stay for all admissions during the preceding 90 days. Outpatient intravenous therapy, chronic hemodialysis, and wound care therapy was determined from medication administration records and relevant Current Procedural Terminology (CPT) or ICD‐9 procedure codes for care administered within 30 days. Antibiotic exposure was defined as administration of 1 dose of antibiotic during inpatient care, or fill of an outpatient prescription for 1 antibiotic dose within 90 days preceding admission. Immunosuppression was defined as: human immunodeficiency virus (HIV) diagnosis; white blood cell (WBC) count of 2500 cells/mm³ within 30 days of admission; corticosteroid ingestion during prior admission, or outpatient prescription fills for a corticosteroid with quantity sufficient to last 14 days preceding admission; or inpatient ingestion of, or outpatient prescription fills for, transplant or rheumatologic‐related immunosuppressants within 90 days preceding admission.

Additional variables assessed to predict CAP‐resistance were obtained as follows. First, modifications of guideline‐defined criteria were constructed. These included: direct nursing home admission, or recent nursing home stay preceding admission; total days of hospitalization within 90 days preceding admission; specific antibiotic exposures, including dates since last exposure preceding admission; and individual components of the immunosuppression criterion. Other cohort‐developed variables included: demographics; substance use history; chronic comorbidity determined by individual and composite measures of Charlson score; pulmonary disease history (eg, bronchiectasis); type and frequency of outpatient visits; consecutive (2) prescription fills for chronic medications of interest; clinical and surveillance culture results preceding admission; admitting ward; vital signs; and relevant hematology and chemistry labs.5

Sputum, blood, and bronchoscopy‐collected cultures obtained within 48 hours after admission were assessed to determine specimen acceptability. Poor sputum specimens were defined by Gram stain quantitative results indicating >10 epithelial cells (EPI) per low power field (LPF), or in the absence of quantitative results, semi‐quantitative results indicating 2‐4+EPI. Single positive blood cultures with results indicating likely contaminants were considered poor specimens. All bronchoscopy‐obtained specimens were considered acceptable. All cultures classified as poor specimens were excluded, and microbiology results were evaluated for the remaining specimens.2, 6 Organisms thought to represent colonization or contamination were excluded: coagulase‐negative (CN) Staphylococcus, Enterococcus sp, Bacillus sp, Proprionibacterium sp, and Candida sp. Recovery of a potential pneumonia pathogen from 1 acceptable culture constituted a culture‐positive admission.

CAP‐resistance was determined for each isolate. CAP‐resistance was defined as non‐susceptibility to non‐pseudomonal third generation cephalosporins (ceftriaxone or cefotaxime) or non‐pseudomonal 8‐methoxy fluoroquinolones (moxifloxacin, gatifloxacin), the VA preferred agents for treatment of CAP.7 There were differences between facilities in susceptibility reporting criteria; therefore, the following approach was used to determine CAP‐resistance. First, MRSA and Pseudomonas aeruginosa isolates were classified as CAP‐resistant. Second, susceptibility results were directly utilized to determine CAP‐resistance if both antibiotic results were available. Third, if only a surrogate antibiotic from a class was reported, a representative antibiotic consistent with Clinical Laboratory Standards Institute reporting criteria was utilized.8 Finally, expert rules determined CAP‐resistance for select potential pneumonia pathogens (eg, Haemophilus sp) if antibiotic susceptibility results for both cephalosporin and fluoroquinolone classes were not reported.815 Presence of 1 CAP‐resistant isolate resulted in a CAP‐resistant classification for an admission. MRSA and Pseudomonas aeruginosa endpoints were defined in a similar manner. Only the first admission for each patient was utilized in the analysis.

The probability of CAP‐resistance was predicted from guideline‐defined criteria (guideline‐defined model) with logistic regression. Next, non‐guideline variables were classified as high, medium, or low interest for association with CAP‐resistance. Variables were assessed for collinearity. A model of CAP‐resistance was developed from variables of high interest. Guideline‐defined criteria were omitted to allow consideration of more specific measures (eg, specific antibiotic exposures as opposed to receipt of antibiotics within the preceding 90 days) during this stage. Next, guideline‐defined criteria, and subsequently variables of lesser interest, were added in an attempt to improve the model. Annual trends and plausible interactions were considered. Model selection was by Akaike's Information Criterion (AIC).16 To promote model reliability, the final model was required to lack evidence of over‐fitting in bootstrapped internal validation.17 The guideline‐defined and cohort‐developed models were compared by difference in area under receiver operating characteristic (aROC) curves. The model development process was repeated for MRSA and Pseudomonas aeruginosa endpoints. Finally, to determine if the CAP‐resistance model sufficiently predicted pathogen‐specific MDR, the CAP‐resistance model was re‐estimated for MRSA and Pseudomonas aeruginosa endpoints. Statistical analysis was performed with R version 2.10.0 (The R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

The cohort was comprised of 1300 patients with HCAP. Of these, 375 (28.8% [26.4‐31.4]) met culture‐positive criteria for potential pneumonia pathogens. CAP‐resistant organisms were identified in 118 (31.5% [26.8‐36.4]) patients within 48 hours of admission. CAP‐resistant organisms included: MRSA (49.2% [40.4‐58.1]), Pseudomonas aeruginosa (29.5% [21.9‐38.1]), Enterobacteriaceae (11.4% [6.5‐18.0]), Gram‐negative non‐enterics (8.3% [4.2‐14.4]), Streptococcus pneumoniae (1.5% [0.2‐5.4]), and opportunistic organisms (eg, Mycobacterium spp) (8.3% [4.2‐14.4]). Differences in select characteristics and exposures between culture‐positive and culture‐negative admissions, as well as CAP‐resistant and CAP‐sensitive admissions, were evident (Table 1).

Of the guideline‐defined criteria, direct admission from a nursing home, prior hospitalization, and recent antibiotic exposure were associated with CAP‐resistance (Table 2). The cohort‐derived CAP‐resistance model included 6 variables. Prior MRSA colonization or infection within 90 days preceding admission was strongly predictive of CAP‐resistance. A composite variable consisting of direct admission from a nursing home or admission from the community after recent discharge from a nursing home was more predictive than direct admission from a nursing home alone. Exposure to cephalosporin antibiotics within the prior year was also predictive of CAP‐resistance. Subcategorizing cephalosporins by class or by most recent exposure in 90‐day increments did not improve the model. The remaining predictors in the model were guideline‐defined infusion therapy criterion, diabetes, and intensive care unit (ICU) admission.

Of the guideline‐defined criteria, direct admission from a nursing home was most predictive of MRSA HCAP (n = 57), followed by prior hospitalization and recent antibiotic exposure (Table 3). The cohort‐developed model of MRSA HCAP included predictors common to the CAP‐resistance model: direct admission from a nursing home or patients who were recently discharged from a nursing home, history of prior MRSA, and diabetes. Positive MRSA status within 90 days preceding admission exhibited the strongest prediction of MRSA HCAP. Exposure to anti‐pseudomonal fluoroquinolones (ciprofloxacin and levofloxacin) within the prior year was also predictive of MRSA HCAP, however, exposure to 8‐methoxy fluoroquinolone was not (crude odds ratio (OR) = 0.7 [0.3‐1.4]; final model adjusted OR = 0.6 [0.2‐1.2]). Exposure to third generation cephalosporins within the previous year was more predictive than other cephalosporin exposures, and more predictive than exposure times categorized in 90‐day increments.

Of the guideline‐defined criteria, only prior hospitalization within 90 days and admission from a nursing home were predictive of Pseudomonas aeruginosa HCAP (n = 36) (Table 4). In the cohort‐developed model of Pseudomonas aeruginosa HCAP, Pseudomonas aeruginosa was predicted by prior cephalosporin exposure within the preceding year, prior culture of Pseudomonas aeruginosa from any anatomical source within the preceding year, and chronic steroid use of 10 mg/day prednisone equivalents. Again, the model was not improved by subcategorizing cephalosporin by class or by most recent exposure time. Finally, a negative annual trend in Pseudomonas aeruginosa HCAP was evident.

The cohort‐developed model of CAP‐resistance was re‐estimated for MRSA and Pseudomonas aeruginosa endpoints. Only positive MRSA status within 90 days preceding admission was associated with both endpoints (OR = 8.7 [3.5‐22.1] for MRSA; OR = 4.3 [1.4‐12.2] for Pseudomonas aeruginosa). Direct or recent nursing home residence (OR = 2.4 [1.3‐4.6]) and diabetes (OR = 2.4 [1.3‐4.5]) were highly predictive of MRSA, but not Pseudomonas aeruginosa (OR = 1.8 [0.8‐3.9] for nursing home residence; OR = 1.3 [0.6‐2.7] for diabetes), respectively. Cephalosporin exposure preceding admission was highly predictive of Pseudomonas aeruginosa (OR = 4.0 [1.9‐9.3]), but not with MRSA (OR = 1.1 [0.6‐2.1]). In these models, all estimated odds ratios were >1.0, consistent with the cohort‐developed model of CAP‐resistance.

For each endpoint, the cohort‐developed model was more predictive than the guideline‐defined model (Table 5) (to view ROC curves see Supporting Figures 1 to 3 in the online version of the article.). The cohort‐developed model of CAP‐resistance re‐estimated for pathogen‐specific endpoints resulted in similar predictive performance. To assess performance of the cohort developed models by facility, aROC was calculated for each of the 3 larger sites separately and for the 3 smaller facilities combined due to limited counts. Site specific aROC ranged from 0.652 to 0.762 for CAP‐resistance, 0.725 to 0.815 for MRSA, and 0.719 to 0.801 for Pseudomonas aeruginosa. The cohort‐developed model of CAP‐resistance re‐estimated for pathogen‐specific endpoints resulted in similar predictive performance.

A nomogram for the cohort‐developed model of CAP‐resistance can provide the predicted probability of culturing a CAP‐resistant organism for an individual patient (Table 6). Point scores assigned to levels of variables, are summed to obtain a total score, and the total score corresponds to a predicted probability of CAP‐resistance. The prevalence of CAP‐resistance (%) from highest to lowest quartile of predicted probability was 92.9, 58.8, 32.9, and 18.5, respectively.

DISCUSSION

In this study, select ATS/IDSA guideline‐defined criteria predicted identification of CAP‐resistant organisms in patients with HCAP. Admission from a nursing home was most predictive of CAP‐resistant organisms, whereas recent hospitalization and antibiotic exposure were predictive to a lesser extent. There was weak evidence of associations between recent infusion and chronic hemodialysis criteria with MDR endpoints. Recent wound care and a composite definition of immunosuppression were not predictive of these endpoints.

The cohort‐developed model resulted in improved prediction of CAP‐resistance endpoints. Culture history, particularly history of MRSA within 90 days preceding admission, was a strong predictor of MDR endpoints. The MRSA history variable definition included cultures from all anatomical sources and nares polymerase chain reaction surveillance results, the latter increasing in 2007‐2008 due to the implementation of the VA MRSA initiative.18 This finding suggests that prior culture results should be considered when selecting empirical antimicrobial therapy, and the rapid proliferation of electronic medical records increases potential to utilize this information routinely. While the guideline‐defined nursing home admission criterion was a strong predictor of CAP‐resistance, admission from the community after recent discharge from a nursing home, in addition to direct admission from a nursing home, was also important.

Similarities in variables included in the pathogen‐specific and CAP‐resistance models reflect the importance of MRSA in defining the CAP‐resistance endpoint. Both CAP‐resistance and MRSA models included prior MRSA status, diabetes, and ICU admission, whereas cephalosporin exposure was common to the Pseudomonas aeruginosa and CAP‐resistance models. Annual trends in CAP‐resistance and MRSA recovery were not identified. The negative annual trend in Pseudomonas aeruginosa HCAP is unexplained and beyond the scope of this study. The percentage of culture‐positive admissions with Pseudomonas aeruginosa HCAP averaged 12% in 2003‐2006, but dropped to <5% in 2007‐2008. A potential explanation is that identification and isolation of patients with MRSA, as a result of the VA‐wide MRSA initiative, may have impacted Pseudomonas aeruginosa colonization by isolating patients co‐colonized with these pathogens during prior healthcare exposures. This is consistent with the observation that when the cohort‐derived CAP‐resistance model was refit with the Pseudomonas aeruginosa endpoint, recent MRSA colonization was strongly predictive of Pseudomonas aeruginosa. Despite differences between variables in pathogen‐specific and CAP‐resistant models, the CAP‐resistance model provided a similar degree of MRSA and Pseudomonas aeruginosa prediction. Finally, as a study purpose included developing best predictive models for each endpoint, and not merely identifying associations, there were other plausible models not reported.

Study strengths included use of the VISN20 Data Warehouse, which provided an integrated outpatient and inpatient medical record. This facilitated analysis of prior healthcare exposures and inpatient study endpoints. In addition, poor blood and sputum specimens and unlikely pneumonia pathogens were not included in establishing MDR endpoints. The variable set explored in regression modeling was extensive and detailed, and analysis included time and intensity‐based components of the variables. Importantly, a standardized approach to regression modeling was specified in advance, which included identification of variables with high potential for association with MDR endpoints, model selection by AIC, re‐evaluation of guideline‐defined criteria and variables of lower interest, and bootstrapped internal model validation.19

Study limitations included the use of ICD‐9 codes to establish a pneumonia diagnosis, which may lack sensitivity and specificity. However, an enhanced ICD‐9based algorithm superior to other claims‐based definitions of pneumonia was utilized.4, 20 Veterans may have received care at non‐VA facilities impacting identification of all healthcare system exposures preceding admission. Data for microbial endpoints were obtained from sterile and non‐sterile site cultures, and it was not possible to determine if the cultured organisms were truly pathogenic. While pathogen‐specific endpoints were not affected, the use of expert rules in select cases to establish CAP‐resistance may have impacted precision for this endpoint. It is also possible that refitting the cohort‐developed CAP‐resistance model for pathogen‐specific endpoints resulted in optimistic aROC due to model over‐fitting. Finally, the cohort was comprised of elderly males, and caution is warranted in extrapolating the results to other populations.

The predictive ability of the guideline‐defined criteria to identify patients with MDR pathogens has been studied. A prospective observational cohort study of 625 consecutive ICU admissions determined that the guideline‐defined criteriaprior antimicrobial treatment, nursing home residence, and prior hospitalizationwere associated with recovery of MDR colonization.21 Shorr et al., investigating a retrospective cohort of 619 patients with HCAP, reported that recent hospitalization, nursing home residence, hemodialysis, and ICU admission were associated with infections caused by CAP‐resistant organisms.22 This study did not report antimicrobial exposures. Our study complements these studies by evaluating existing HCAP guideline criteria, and identifying specific antibiotic exposure, prior culture data, comorbid illness, and immunosuppressive medications that are predictive of MDR infection.

Studies comparing the bacterial etiology of patients with pneumonia in nursing homes relative to CAP, have demonstrated mixed results in recovery of Gram‐negative MDR pathogens, but generally increased MRSA pneumonia.3 Our study suggests that a nursing home stay in the last 6 months is associated with an increased risk for MRSA, but not Pseudomonas aeruginosa, although this was limited by small sample size. Recent infusion therapy has not been previously reported to be associated with MDR pathogens in an HCAP population. In our study, this criterion was predictive of CAP‐resistance in the cohort‐developed model, but not in conjunction with other variables in the guideline‐defined model. Predictors of pathogen‐specific HCAP are limited to an aforementioned single prior study, which identified recent hospitalization, nursing home residence, and ICU admission as risk factors for MRSA HCAP.22

Many studies have investigated risks for infection with MRSA and Pseudomonas aeruginosa outside of the context of HCAP. Predictor variables in cohort‐developed pathogen‐specific models in our study are known risk factors for colonization or infection with these pathogens. For example, antecedent MRSA colonization has been noted as a strong risk factor for MRSA infection, particularly pneumonia.23, 24 Further, patients with diabetes and inhaled corticosteroid exposure are immunosuppressed and at increased risk for colonization with MRSA.25, 26 Likewise, bronchiolar colonization and corticosteroid exposures are known risk factors for pneumonia due to Pseudomonas aeruginosa.27

Many studies have identified prior antibiotic use as a risk factor for infections caused by MRSA and Pseudomonas aeruginosa. However, this criterion is excessively broad and specific antimicrobial exposures carry different magnitudes of risk. Third generation cephalosporins and anti‐pseudomonal fluoroquinolones are commonly reported antibiotics associated with risk for MRSA infection, whereas 8‐methoxy fluoroquinolones appear not to possess the same effect.2831 Likewise, cephalosporins have been reported as risk factors for MDR Pseudomonas aeruginosa infections.32

Several areas of research involving HCAP MDR risk should be investigated. First, the predictive models developed in our and other studies should be evaluated in larger, more diverse populations to establish generalizability. Second, empirical broad‐spectrum antibiotic therapy in all patients with HCAP results in overtreatment of many patients. To date, no reported models provided optimal performance for selecting empirical therapy for unstable ICU patients with HCAP, and many patients do not receive de‐escalation therapy. Thus, models to identify patients with low probability of MDR pathogens upon admission and to aid in de‐escalation are warranted. Finally, the negative trend in Pseudomonas aeruginosa HCAP requires confirmation and further study.

In conclusion, of the ATS/IDSA guideline‐defined criteria for MDR, nursing home admission, recent hospitalization, and antibiotic exposure were predictive of the recovery of CAP‐resistant organisms. Alternative models primarily based on prior culture data, specific antibiotic exposures, and immunosuppression‐related variables improved predictive performance of HCAP associated with MDR.