Internal medicine residency training continues to evolve as competency‐based and with education organized around patient care.13 Making the patient the center of resident education provides an opportunity for experiential learning in which learning can be organized around the clinical conditions that residents encounter. Despite the renewed emphasis on using patient experience as the basis for residency education, little is known regarding what specific diagnostic conditions are seen by internal medicine residents throughout their training. Attempts have been made to quantify resident clinical experience in various fields, using approaches such as review of medical records, case logs, and prescription profiles, but to date, we lack systematic methods to obtain clinical experience data for internal medicine residents.47

While residency curricula in internal medicine typically outlines specific rotations in various clinical areas such as general medical wards, cardiology services, and intensive care units, time spent on such rotations does not necessarily provide quantitative data on the actual clinical conditions that residents encounter, nor does it ensure consistent clinical experience between residents. It is plausible that there may be substantial variability in clinical experience between residents within the same program, and that the overall spectrum of clinical disorders seen by residents in a program may or may not be consistent with a desired optimum, though this is yet to be defined.

If residency education in internal medicine is to progressively incorporate more experiential learning, detailed knowledge of the clinical conditions seen by residents should be useful, not only for overall curriculum design, but this might also allow for various educational interventions to be made when there are variations in clinical experience between residents. Our program has been interested in the application of electronic resources for the improvement of patient care, such as through the handoff process and the use of personal digital assistants.8 We previously did a small analysis of clinical conditions seen by residents through non‐International Classification of Diseases, Ninth Revision (ICD‐9)‐based data they entered onto personal digital assistants. This suggested to us that electronic resources used by residents might serve as a venue by which they could enter diagnostic information which we could use to generate a more detailed analysis of the clinical conditions that they see. Here we describe a method by which we have attempted to quantify resident clinical experience in internal medicine using a modification of an electronic handoff system.

METHODS

The study was conducted within the Internal Medicine Residency Program at the Long Island Jewish Medical Center in New Hyde Park, New York, part of the North ShoreLong Island Jewish Health System, and was approved by the Institutional Review Board. This work was carried out as part of our participation in the Educational Innovation Project of the Residency Review Committee for Internal Medicine. A central objective of our proposal was to develop a method to assess residents' clinical experience on an individual and an aggregate basis. A group of faculty and residents in our residency program developed an electronic handoff tool which residents use for rapid access to key clinical data for their patients and for the handoff of clinical information for on call coverage. This handoff tool was developed with the technical assistance of MedTech Notes LLC which owns Patient Data Transfer System (PDTS) HandOff Note. We modified the handoff tool to include a section in which residents were required to enter a primary diagnosis for each of their patients (a hard stop design). We chose to use the ICD‐9 system for standardization and created two methods to select the code: 1) an organ system‐based dropdown list containing frequently used codes and 2) a search box allowing for searching of the complete ICD‐9 database. For the organ‐based dropdown list, selection of that organ system would reveal a brief list of frequently used codes to make it easier for residents to find them. Prior to using the handoff tool with the ICD‐9based primary diagnosis coding system, training sessions with the residents were conducted by 3 of the investigators along with 3 chief medical residents. These sessions included training not only in technical aspects of how to find diagnosis codes, but also how to make decisions regarding what the primary diagnosis should be. We also instructed our postgraduate year (PGY)‐1s to update their diagnostic selections during the course of the hospital stay.

Each data point represents a resident caring for a patient with a specific diagnostic entity, and is counted once for that resident's period of taking care of that patient. Thirty‐three PGY‐1s were studied and, on the internal medicine service, they were supervised by either hospitalist faculty or voluntary faculty in comparable proportions. If the patient's care is taken over by another resident, that second resident was also recorded as having had a diagnostic encounter with that patient, hence 1 patient could provide experience with the same diagnostic entity for 1 or more residents. Using this method, the denominator is not patients seen, but residentpatient diagnostic encounters that have taken place. The ICD‐9 diagnostic conditions entered by the residents were grouped using the ICD‐9 system. Individual diagnostic profiles for each resident, as well as an aggregate profile for all residents to reflect the residency program as a whole, were generated. We also carried out an analysis of the ICD‐9 codes entered by 6 consecutive PGY‐1s to assess how the diagnostic spectrum might vary among a small sampling of PGY‐1s. In order to evaluate the accuracy of the residents' diagnostic selections, we carried out a validation assessment using a tool used by the residents' supervising hospitalists (who were the attendings of record for those patients). This was carried out on a subset of patients and could be done at any time during the hospital stay. The hospitalists were asked to review their residents' ICD‐9 codes and indicate whether they agreed or disagreed.

RESULTS

A total of 7562 residentpatient diagnostic encounters were studied from July 1, 2007 through June 1, 2008. Mean patient age was 66 19.4 years. The age distribution is given in Table 1 and reveals that 65% of diagnostic encounters were with patients age 60 years or greater. Twelve housestaff teams were studied, each consisting of 2 PGY‐1s and a supervising PGY‐2 or PGY‐3 resident. All ICD‐9 codes were selected by categorical and preliminary internal medicine PGY‐1s on medical ward and intensive care unit rotations. Residents from other departments doing rotations on the medical service were excluded. A validation assessment of 341 patients indicated 83.3% agreement by the supervising hospitalist with the primary ICD‐9 code selected. ICD‐9 codes were then grouped and categorized using ICD‐9 nomenclature with the distribution provided in Table 2. A wide spectrum of clinical conditions is apparent including symptoms and ill‐defined conditions, circulatory disorders, respiratory disorders, neoplasms, genitourinary disorders, digestive disorders, diseases of the blood/blood forming organs, endocrinologic/nutritional/metabolic/emmmune disorders, and disorders of the skin and subcutaneous tissue, overall accounting for about 86% of resident clinical experience.

We also examined the most common diagnostic conditions within each of these categories. The 3 most common ICD‐9 codes entered by residents within each category are provided in Table 3. Symptoms and ill‐defined conditions represent a sizable portion of resident clinical experience (19.51%). Within this category, the most common conditions were fever; abdominal pain (unspecified site); and chest pain, unspecified. Disorders of the circulatory and respiratory systems were the next most common categories of conditions seen by residents, comprising 18.26% and 12.42%, respectively, of resident clinical experience. Within the category of circulatory disorders, congestive heart failure and acute myocardial infarction were the most common conditions seen; for respiratory disorders, pneumonia, chronic airway obstruction, and asthma were most commonly encountered. In aggregate, symptoms and ill‐defined conditions, and disorders of the circulatory and respiratory systems accounted for 50% of resident clinical experience.

Individual resident clinical experience varied as well. As shown in Table 4, for a group of 6 PGY‐1s, there was substantial variability in the ICD‐9 diagnostic categories. For example, the percentages of codes falling into the cardiovascular disease category ranged from 15.27% to 27.91%, and for respiratory disease ranged from 8.22% to 18.55%. These data suggest that there may be sizable differences in the proportions of various clinical conditions seen by residents over a year of training.

DISCUSSION

Years ago, residency training transitioned from a predominantly bedside experience to a curriculum with a large didactic, non‐bedside component, following parameters defined by organizations such as the Accreditation Council for Graduate Medical Education. Residency training is undergoing substantial change to become competency‐based and to organize learning around patient care experiences.2, 3, 9 The Educational Innovation Project of the Residency Review Committee for Internal Medicine is one such endeavor to help develop new methods by which to accomplish this.1 Effective incorporation of innovative experiential learning methods, based on the core competencies, will require a detailed knowledge of resident clinical experience during the course of their training, yet such data have been sparse in internal medicine. Sequist et al. analyzed data from an electronic medical record to assess resident clinical experience in the outpatient setting.4 Bachur and Nagler have used an electronic patient tracking system to assess the clinical experience of pediatric emergency medicine fellows.5, 6 Most attempts to describe resident clinical experience have relied upon extracting diagnostic information from medical records, case logs, etc, though in another approach, Rohrbaugh et al. reviewed psychiatric resident prescription profiles,7 which might provide some indirect data on clinical experience if applied to internal medicine.

In this study, we attempted to quantify resident clinical experience using resident‐selected ICD‐9 codes, in contrast to other methods that have relied upon medical record review and other resident‐independent approaches. There are various strengths and limitations to this approach. Using the ICD‐9 system provides a number of strengths, a major one being standardization, allowing comparisons between different programs and perhaps even facilitating the development of guidelines for resident clinical experience. In addition, this approach using the ICD‐9 system could be readily implemented at any institution and does not require any specific technology. While we chose to do this through our handoff system, an institution could use any of a variety of other systems to accomplish this. For example, resident‐entered ICD‐9 coding systems could be incorporated into electronic discharge summaries, history and physicals, or progress notes. There may also be some practical benefits to having residents learn how to use the ICD‐9 system at this stage of their careers.

There are limitations to this approach as well. The ICD‐9 system was not intended to be used for medical education purposes. There are features of it that can make finding the best diagnosis difficult, and routes to it may at times seem counterintuitive. While we did not carry out resident surveys, a number of residents anecdotally mentioned that it took time to become comfortable using the system, and it could be challenging at times to find a diagnosis description that best fit what they were looking for. To make diagnosis selection easier, we created an organ system‐based dropdown list in the handoff tool so that when residents select an organ system, another list opens up containing commonly used ICD‐9 codes. This grouping is based on organ system alone and does not necessarily follow the ICD‐9 grouping (in contrast, our reported data in this article are all based on ICD‐9 grouping). A search tool to allow searching the entire ICD‐9 database was also made available on the handoff tool. Other factors that could limit diagnosis code accuracy could be lack of clinical knowledge, and error as a result of pressure to come up with a diagnosis because of the hard stop design of our system, in which residents were required to enter a primary diagnosis, potentially causing alert fatigue. A validation assessment that we carried out revealed fairly good agreement with the specific ICD‐9 codes chosen by the resident, but greater accuracy would be desirable. Further education on diagnosis selection and refinements to the handoff tool should help facilitate this. We are currently addressing this by ongoing education on diagnosis selection and by having the hospitalists share the handoff tool with the residents, allowing them to provide direct feedback on diagnostic selections.

More than 19% of the diagnoses selected by residents fell into the category of symptoms and ill‐defined conditions. This raises a number of potential educational issues. One of those is that if residents do, in fact, encounter such entities at such a high frequency, then the internal medicine curriculum must be structured in such a way as to complement this clinical experience with a comprehensive learning program. However, we must also consider the possibility that, in many such instances, a more definitive diagnosis became evident by the time of discharge and this may not have been reflected in the ICD‐9 code that the resident chose. Hence, the category of symptoms and ill‐defined conditions may actually be somewhat smaller than our findings would suggest.

Many issues will need to be addressed as programs obtain more data on their residents' clinical experience. While there may be many reasons to use the ICD‐9 system for selecting diagnoses including those listed above, the system by which ICD‐9 groups diagnoses might not provide ideal educational information, again as the ICD‐9 system was not designed for this purpose. While in this article we have reported the residents' diagnostic encounters grouped according to the ICD‐9 grouping system to provide an initial standardized description, grouping according to another diagnostic system that is felt to be more educationally meaningful may be preferred.

While one might assume that a higher frequency of exposure to certain clinical conditions should enhance competency, that relationship may not be straightforward in internal medicine. For surgical procedures, there are, in fact, data to show improved outcomes for surgeons with higher operative volumes for those procedures,10 but in internal medicine, we do not have data to demonstrate that competence of a resident caring for a particular condition is enhanced by experience alone. Therefore, as programs obtain more data on clinical experience, it will be important that the focus be kept on quality as opposed to quantity.

Obtaining data on resident clinical experience might greatly facilitate experiential learning approaches. For example, as residents go through training and encounter specific diagnostic conditions, those experiences could be supplemented by various learning innovations to make those experiences more meaningful and, hopefully, more likely to result in the development of competence, though that will require measurement. In our program, for example, we have incorporated an approach using illness scenarios, in that when residents have had a certain level of clinical experience with a given clinical condition, they are assembled in small groups and competency‐based case discussions are carried out with a preceptor. In addition, for those instances in which an individual resident may lack direct clinical experience in a certain area, this might be addressed by interventions to increase their contact with those conditions and/or targeted learning interventions to help develop competence. A resident found to be lacking in clinical experience in a certain area could be assigned to the care of more patients with that condition, or to spending more time in a venue in which that condition is more likely to be encountered. Various learning activities including didactics, case discussions, simulation, self‐directed learning, and others could also be used to compensate for such variability. Furthermore, if a residency program's aggregate clinical experience is divergent from some desirable standard yet to be determined, a detailed knowledge of this could help guide that program's curriculum revision. For example, for residents in a program in which there is relatively low exposure to patients with oncological issues, this could be compensated for by external rotations to achieve more clinical experience in oncology, as well as supplementation of the curriculum with additional learning activities in oncology, which could include small group discussions, self‐directed learning activities, case discussions, and others. While at present there are no defined standards for clinical experience and it remains to be seen if there would be a correlation with development of competence, no such standard would serve a purpose if programs did not have reliable and practical means of clinical experience assessment.

In summary, resident‐selected ICD‐9 codes may be a useful means to obtain data regarding resident clinical experience in internal medicine. Such data may be useful to residency training programs in developing new curricula based on experiential learning.

Internal medicine residency training continues to evolve as competency‐based and with education organized around patient care.13 Making the patient the center of resident education provides an opportunity for experiential learning in which learning can be organized around the clinical conditions that residents encounter. Despite the renewed emphasis on using patient experience as the basis for residency education, little is known regarding what specific diagnostic conditions are seen by internal medicine residents throughout their training. Attempts have been made to quantify resident clinical experience in various fields, using approaches such as review of medical records, case logs, and prescription profiles, but to date, we lack systematic methods to obtain clinical experience data for internal medicine residents.47

While residency curricula in internal medicine typically outlines specific rotations in various clinical areas such as general medical wards, cardiology services, and intensive care units, time spent on such rotations does not necessarily provide quantitative data on the actual clinical conditions that residents encounter, nor does it ensure consistent clinical experience between residents. It is plausible that there may be substantial variability in clinical experience between residents within the same program, and that the overall spectrum of clinical disorders seen by residents in a program may or may not be consistent with a desired optimum, though this is yet to be defined.

If residency education in internal medicine is to progressively incorporate more experiential learning, detailed knowledge of the clinical conditions seen by residents should be useful, not only for overall curriculum design, but this might also allow for various educational interventions to be made when there are variations in clinical experience between residents. Our program has been interested in the application of electronic resources for the improvement of patient care, such as through the handoff process and the use of personal digital assistants.8 We previously did a small analysis of clinical conditions seen by residents through non‐International Classification of Diseases, Ninth Revision (ICD‐9)‐based data they entered onto personal digital assistants. This suggested to us that electronic resources used by residents might serve as a venue by which they could enter diagnostic information which we could use to generate a more detailed analysis of the clinical conditions that they see. Here we describe a method by which we have attempted to quantify resident clinical experience in internal medicine using a modification of an electronic handoff system.

METHODS

The study was conducted within the Internal Medicine Residency Program at the Long Island Jewish Medical Center in New Hyde Park, New York, part of the North ShoreLong Island Jewish Health System, and was approved by the Institutional Review Board. This work was carried out as part of our participation in the Educational Innovation Project of the Residency Review Committee for Internal Medicine. A central objective of our proposal was to develop a method to assess residents' clinical experience on an individual and an aggregate basis. A group of faculty and residents in our residency program developed an electronic handoff tool which residents use for rapid access to key clinical data for their patients and for the handoff of clinical information for on call coverage. This handoff tool was developed with the technical assistance of MedTech Notes LLC which owns Patient Data Transfer System (PDTS) HandOff Note. We modified the handoff tool to include a section in which residents were required to enter a primary diagnosis for each of their patients (a hard stop design). We chose to use the ICD‐9 system for standardization and created two methods to select the code: 1) an organ system‐based dropdown list containing frequently used codes and 2) a search box allowing for searching of the complete ICD‐9 database. For the organ‐based dropdown list, selection of that organ system would reveal a brief list of frequently used codes to make it easier for residents to find them. Prior to using the handoff tool with the ICD‐9based primary diagnosis coding system, training sessions with the residents were conducted by 3 of the investigators along with 3 chief medical residents. These sessions included training not only in technical aspects of how to find diagnosis codes, but also how to make decisions regarding what the primary diagnosis should be. We also instructed our postgraduate year (PGY)‐1s to update their diagnostic selections during the course of the hospital stay.

Each data point represents a resident caring for a patient with a specific diagnostic entity, and is counted once for that resident's period of taking care of that patient. Thirty‐three PGY‐1s were studied and, on the internal medicine service, they were supervised by either hospitalist faculty or voluntary faculty in comparable proportions. If the patient's care is taken over by another resident, that second resident was also recorded as having had a diagnostic encounter with that patient, hence 1 patient could provide experience with the same diagnostic entity for 1 or more residents. Using this method, the denominator is not patients seen, but residentpatient diagnostic encounters that have taken place. The ICD‐9 diagnostic conditions entered by the residents were grouped using the ICD‐9 system. Individual diagnostic profiles for each resident, as well as an aggregate profile for all residents to reflect the residency program as a whole, were generated. We also carried out an analysis of the ICD‐9 codes entered by 6 consecutive PGY‐1s to assess how the diagnostic spectrum might vary among a small sampling of PGY‐1s. In order to evaluate the accuracy of the residents' diagnostic selections, we carried out a validation assessment using a tool used by the residents' supervising hospitalists (who were the attendings of record for those patients). This was carried out on a subset of patients and could be done at any time during the hospital stay. The hospitalists were asked to review their residents' ICD‐9 codes and indicate whether they agreed or disagreed.

RESULTS

A total of 7562 residentpatient diagnostic encounters were studied from July 1, 2007 through June 1, 2008. Mean patient age was 66 19.4 years. The age distribution is given in Table 1 and reveals that 65% of diagnostic encounters were with patients age 60 years or greater. Twelve housestaff teams were studied, each consisting of 2 PGY‐1s and a supervising PGY‐2 or PGY‐3 resident. All ICD‐9 codes were selected by categorical and preliminary internal medicine PGY‐1s on medical ward and intensive care unit rotations. Residents from other departments doing rotations on the medical service were excluded. A validation assessment of 341 patients indicated 83.3% agreement by the supervising hospitalist with the primary ICD‐9 code selected. ICD‐9 codes were then grouped and categorized using ICD‐9 nomenclature with the distribution provided in Table 2. A wide spectrum of clinical conditions is apparent including symptoms and ill‐defined conditions, circulatory disorders, respiratory disorders, neoplasms, genitourinary disorders, digestive disorders, diseases of the blood/blood forming organs, endocrinologic/nutritional/metabolic/emmmune disorders, and disorders of the skin and subcutaneous tissue, overall accounting for about 86% of resident clinical experience.

We also examined the most common diagnostic conditions within each of these categories. The 3 most common ICD‐9 codes entered by residents within each category are provided in Table 3. Symptoms and ill‐defined conditions represent a sizable portion of resident clinical experience (19.51%). Within this category, the most common conditions were fever; abdominal pain (unspecified site); and chest pain, unspecified. Disorders of the circulatory and respiratory systems were the next most common categories of conditions seen by residents, comprising 18.26% and 12.42%, respectively, of resident clinical experience. Within the category of circulatory disorders, congestive heart failure and acute myocardial infarction were the most common conditions seen; for respiratory disorders, pneumonia, chronic airway obstruction, and asthma were most commonly encountered. In aggregate, symptoms and ill‐defined conditions, and disorders of the circulatory and respiratory systems accounted for 50% of resident clinical experience.

Individual resident clinical experience varied as well. As shown in Table 4, for a group of 6 PGY‐1s, there was substantial variability in the ICD‐9 diagnostic categories. For example, the percentages of codes falling into the cardiovascular disease category ranged from 15.27% to 27.91%, and for respiratory disease ranged from 8.22% to 18.55%. These data suggest that there may be sizable differences in the proportions of various clinical conditions seen by residents over a year of training.

DISCUSSION

Years ago, residency training transitioned from a predominantly bedside experience to a curriculum with a large didactic, non‐bedside component, following parameters defined by organizations such as the Accreditation Council for Graduate Medical Education. Residency training is undergoing substantial change to become competency‐based and to organize learning around patient care experiences.2, 3, 9 The Educational Innovation Project of the Residency Review Committee for Internal Medicine is one such endeavor to help develop new methods by which to accomplish this.1 Effective incorporation of innovative experiential learning methods, based on the core competencies, will require a detailed knowledge of resident clinical experience during the course of their training, yet such data have been sparse in internal medicine. Sequist et al. analyzed data from an electronic medical record to assess resident clinical experience in the outpatient setting.4 Bachur and Nagler have used an electronic patient tracking system to assess the clinical experience of pediatric emergency medicine fellows.5, 6 Most attempts to describe resident clinical experience have relied upon extracting diagnostic information from medical records, case logs, etc, though in another approach, Rohrbaugh et al. reviewed psychiatric resident prescription profiles,7 which might provide some indirect data on clinical experience if applied to internal medicine.

In this study, we attempted to quantify resident clinical experience using resident‐selected ICD‐9 codes, in contrast to other methods that have relied upon medical record review and other resident‐independent approaches. There are various strengths and limitations to this approach. Using the ICD‐9 system provides a number of strengths, a major one being standardization, allowing comparisons between different programs and perhaps even facilitating the development of guidelines for resident clinical experience. In addition, this approach using the ICD‐9 system could be readily implemented at any institution and does not require any specific technology. While we chose to do this through our handoff system, an institution could use any of a variety of other systems to accomplish this. For example, resident‐entered ICD‐9 coding systems could be incorporated into electronic discharge summaries, history and physicals, or progress notes. There may also be some practical benefits to having residents learn how to use the ICD‐9 system at this stage of their careers.

There are limitations to this approach as well. The ICD‐9 system was not intended to be used for medical education purposes. There are features of it that can make finding the best diagnosis difficult, and routes to it may at times seem counterintuitive. While we did not carry out resident surveys, a number of residents anecdotally mentioned that it took time to become comfortable using the system, and it could be challenging at times to find a diagnosis description that best fit what they were looking for. To make diagnosis selection easier, we created an organ system‐based dropdown list in the handoff tool so that when residents select an organ system, another list opens up containing commonly used ICD‐9 codes. This grouping is based on organ system alone and does not necessarily follow the ICD‐9 grouping (in contrast, our reported data in this article are all based on ICD‐9 grouping). A search tool to allow searching the entire ICD‐9 database was also made available on the handoff tool. Other factors that could limit diagnosis code accuracy could be lack of clinical knowledge, and error as a result of pressure to come up with a diagnosis because of the hard stop design of our system, in which residents were required to enter a primary diagnosis, potentially causing alert fatigue. A validation assessment that we carried out revealed fairly good agreement with the specific ICD‐9 codes chosen by the resident, but greater accuracy would be desirable. Further education on diagnosis selection and refinements to the handoff tool should help facilitate this. We are currently addressing this by ongoing education on diagnosis selection and by having the hospitalists share the handoff tool with the residents, allowing them to provide direct feedback on diagnostic selections.

More than 19% of the diagnoses selected by residents fell into the category of symptoms and ill‐defined conditions. This raises a number of potential educational issues. One of those is that if residents do, in fact, encounter such entities at such a high frequency, then the internal medicine curriculum must be structured in such a way as to complement this clinical experience with a comprehensive learning program. However, we must also consider the possibility that, in many such instances, a more definitive diagnosis became evident by the time of discharge and this may not have been reflected in the ICD‐9 code that the resident chose. Hence, the category of symptoms and ill‐defined conditions may actually be somewhat smaller than our findings would suggest.

Many issues will need to be addressed as programs obtain more data on their residents' clinical experience. While there may be many reasons to use the ICD‐9 system for selecting diagnoses including those listed above, the system by which ICD‐9 groups diagnoses might not provide ideal educational information, again as the ICD‐9 system was not designed for this purpose. While in this article we have reported the residents' diagnostic encounters grouped according to the ICD‐9 grouping system to provide an initial standardized description, grouping according to another diagnostic system that is felt to be more educationally meaningful may be preferred.

While one might assume that a higher frequency of exposure to certain clinical conditions should enhance competency, that relationship may not be straightforward in internal medicine. For surgical procedures, there are, in fact, data to show improved outcomes for surgeons with higher operative volumes for those procedures,10 but in internal medicine, we do not have data to demonstrate that competence of a resident caring for a particular condition is enhanced by experience alone. Therefore, as programs obtain more data on clinical experience, it will be important that the focus be kept on quality as opposed to quantity.

Obtaining data on resident clinical experience might greatly facilitate experiential learning approaches. For example, as residents go through training and encounter specific diagnostic conditions, those experiences could be supplemented by various learning innovations to make those experiences more meaningful and, hopefully, more likely to result in the development of competence, though that will require measurement. In our program, for example, we have incorporated an approach using illness scenarios, in that when residents have had a certain level of clinical experience with a given clinical condition, they are assembled in small groups and competency‐based case discussions are carried out with a preceptor. In addition, for those instances in which an individual resident may lack direct clinical experience in a certain area, this might be addressed by interventions to increase their contact with those conditions and/or targeted learning interventions to help develop competence. A resident found to be lacking in clinical experience in a certain area could be assigned to the care of more patients with that condition, or to spending more time in a venue in which that condition is more likely to be encountered. Various learning activities including didactics, case discussions, simulation, self‐directed learning, and others could also be used to compensate for such variability. Furthermore, if a residency program's aggregate clinical experience is divergent from some desirable standard yet to be determined, a detailed knowledge of this could help guide that program's curriculum revision. For example, for residents in a program in which there is relatively low exposure to patients with oncological issues, this could be compensated for by external rotations to achieve more clinical experience in oncology, as well as supplementation of the curriculum with additional learning activities in oncology, which could include small group discussions, self‐directed learning activities, case discussions, and others. While at present there are no defined standards for clinical experience and it remains to be seen if there would be a correlation with development of competence, no such standard would serve a purpose if programs did not have reliable and practical means of clinical experience assessment.

In summary, resident‐selected ICD‐9 codes may be a useful means to obtain data regarding resident clinical experience in internal medicine. Such data may be useful to residency training programs in developing new curricula based on experiential learning.

Internal medicine residency training continues to evolve as competency‐based and with education organized around patient care.13 Making the patient the center of resident education provides an opportunity for experiential learning in which learning can be organized around the clinical conditions that residents encounter. Despite the renewed emphasis on using patient experience as the basis for residency education, little is known regarding what specific diagnostic conditions are seen by internal medicine residents throughout their training. Attempts have been made to quantify resident clinical experience in various fields, using approaches such as review of medical records, case logs, and prescription profiles, but to date, we lack systematic methods to obtain clinical experience data for internal medicine residents.47

While residency curricula in internal medicine typically outlines specific rotations in various clinical areas such as general medical wards, cardiology services, and intensive care units, time spent on such rotations does not necessarily provide quantitative data on the actual clinical conditions that residents encounter, nor does it ensure consistent clinical experience between residents. It is plausible that there may be substantial variability in clinical experience between residents within the same program, and that the overall spectrum of clinical disorders seen by residents in a program may or may not be consistent with a desired optimum, though this is yet to be defined.

If residency education in internal medicine is to progressively incorporate more experiential learning, detailed knowledge of the clinical conditions seen by residents should be useful, not only for overall curriculum design, but this might also allow for various educational interventions to be made when there are variations in clinical experience between residents. Our program has been interested in the application of electronic resources for the improvement of patient care, such as through the handoff process and the use of personal digital assistants.8 We previously did a small analysis of clinical conditions seen by residents through non‐International Classification of Diseases, Ninth Revision (ICD‐9)‐based data they entered onto personal digital assistants. This suggested to us that electronic resources used by residents might serve as a venue by which they could enter diagnostic information which we could use to generate a more detailed analysis of the clinical conditions that they see. Here we describe a method by which we have attempted to quantify resident clinical experience in internal medicine using a modification of an electronic handoff system.

METHODS

The study was conducted within the Internal Medicine Residency Program at the Long Island Jewish Medical Center in New Hyde Park, New York, part of the North ShoreLong Island Jewish Health System, and was approved by the Institutional Review Board. This work was carried out as part of our participation in the Educational Innovation Project of the Residency Review Committee for Internal Medicine. A central objective of our proposal was to develop a method to assess residents' clinical experience on an individual and an aggregate basis. A group of faculty and residents in our residency program developed an electronic handoff tool which residents use for rapid access to key clinical data for their patients and for the handoff of clinical information for on call coverage. This handoff tool was developed with the technical assistance of MedTech Notes LLC which owns Patient Data Transfer System (PDTS) HandOff Note. We modified the handoff tool to include a section in which residents were required to enter a primary diagnosis for each of their patients (a hard stop design). We chose to use the ICD‐9 system for standardization and created two methods to select the code: 1) an organ system‐based dropdown list containing frequently used codes and 2) a search box allowing for searching of the complete ICD‐9 database. For the organ‐based dropdown list, selection of that organ system would reveal a brief list of frequently used codes to make it easier for residents to find them. Prior to using the handoff tool with the ICD‐9based primary diagnosis coding system, training sessions with the residents were conducted by 3 of the investigators along with 3 chief medical residents. These sessions included training not only in technical aspects of how to find diagnosis codes, but also how to make decisions regarding what the primary diagnosis should be. We also instructed our postgraduate year (PGY)‐1s to update their diagnostic selections during the course of the hospital stay.

Each data point represents a resident caring for a patient with a specific diagnostic entity, and is counted once for that resident's period of taking care of that patient. Thirty‐three PGY‐1s were studied and, on the internal medicine service, they were supervised by either hospitalist faculty or voluntary faculty in comparable proportions. If the patient's care is taken over by another resident, that second resident was also recorded as having had a diagnostic encounter with that patient, hence 1 patient could provide experience with the same diagnostic entity for 1 or more residents. Using this method, the denominator is not patients seen, but residentpatient diagnostic encounters that have taken place. The ICD‐9 diagnostic conditions entered by the residents were grouped using the ICD‐9 system. Individual diagnostic profiles for each resident, as well as an aggregate profile for all residents to reflect the residency program as a whole, were generated. We also carried out an analysis of the ICD‐9 codes entered by 6 consecutive PGY‐1s to assess how the diagnostic spectrum might vary among a small sampling of PGY‐1s. In order to evaluate the accuracy of the residents' diagnostic selections, we carried out a validation assessment using a tool used by the residents' supervising hospitalists (who were the attendings of record for those patients). This was carried out on a subset of patients and could be done at any time during the hospital stay. The hospitalists were asked to review their residents' ICD‐9 codes and indicate whether they agreed or disagreed.

RESULTS

A total of 7562 residentpatient diagnostic encounters were studied from July 1, 2007 through June 1, 2008. Mean patient age was 66 19.4 years. The age distribution is given in Table 1 and reveals that 65% of diagnostic encounters were with patients age 60 years or greater. Twelve housestaff teams were studied, each consisting of 2 PGY‐1s and a supervising PGY‐2 or PGY‐3 resident. All ICD‐9 codes were selected by categorical and preliminary internal medicine PGY‐1s on medical ward and intensive care unit rotations. Residents from other departments doing rotations on the medical service were excluded. A validation assessment of 341 patients indicated 83.3% agreement by the supervising hospitalist with the primary ICD‐9 code selected. ICD‐9 codes were then grouped and categorized using ICD‐9 nomenclature with the distribution provided in Table 2. A wide spectrum of clinical conditions is apparent including symptoms and ill‐defined conditions, circulatory disorders, respiratory disorders, neoplasms, genitourinary disorders, digestive disorders, diseases of the blood/blood forming organs, endocrinologic/nutritional/metabolic/emmmune disorders, and disorders of the skin and subcutaneous tissue, overall accounting for about 86% of resident clinical experience.

We also examined the most common diagnostic conditions within each of these categories. The 3 most common ICD‐9 codes entered by residents within each category are provided in Table 3. Symptoms and ill‐defined conditions represent a sizable portion of resident clinical experience (19.51%). Within this category, the most common conditions were fever; abdominal pain (unspecified site); and chest pain, unspecified. Disorders of the circulatory and respiratory systems were the next most common categories of conditions seen by residents, comprising 18.26% and 12.42%, respectively, of resident clinical experience. Within the category of circulatory disorders, congestive heart failure and acute myocardial infarction were the most common conditions seen; for respiratory disorders, pneumonia, chronic airway obstruction, and asthma were most commonly encountered. In aggregate, symptoms and ill‐defined conditions, and disorders of the circulatory and respiratory systems accounted for 50% of resident clinical experience.

Individual resident clinical experience varied as well. As shown in Table 4, for a group of 6 PGY‐1s, there was substantial variability in the ICD‐9 diagnostic categories. For example, the percentages of codes falling into the cardiovascular disease category ranged from 15.27% to 27.91%, and for respiratory disease ranged from 8.22% to 18.55%. These data suggest that there may be sizable differences in the proportions of various clinical conditions seen by residents over a year of training.

DISCUSSION

Years ago, residency training transitioned from a predominantly bedside experience to a curriculum with a large didactic, non‐bedside component, following parameters defined by organizations such as the Accreditation Council for Graduate Medical Education. Residency training is undergoing substantial change to become competency‐based and to organize learning around patient care experiences.2, 3, 9 The Educational Innovation Project of the Residency Review Committee for Internal Medicine is one such endeavor to help develop new methods by which to accomplish this.1 Effective incorporation of innovative experiential learning methods, based on the core competencies, will require a detailed knowledge of resident clinical experience during the course of their training, yet such data have been sparse in internal medicine. Sequist et al. analyzed data from an electronic medical record to assess resident clinical experience in the outpatient setting.4 Bachur and Nagler have used an electronic patient tracking system to assess the clinical experience of pediatric emergency medicine fellows.5, 6 Most attempts to describe resident clinical experience have relied upon extracting diagnostic information from medical records, case logs, etc, though in another approach, Rohrbaugh et al. reviewed psychiatric resident prescription profiles,7 which might provide some indirect data on clinical experience if applied to internal medicine.

In this study, we attempted to quantify resident clinical experience using resident‐selected ICD‐9 codes, in contrast to other methods that have relied upon medical record review and other resident‐independent approaches. There are various strengths and limitations to this approach. Using the ICD‐9 system provides a number of strengths, a major one being standardization, allowing comparisons between different programs and perhaps even facilitating the development of guidelines for resident clinical experience. In addition, this approach using the ICD‐9 system could be readily implemented at any institution and does not require any specific technology. While we chose to do this through our handoff system, an institution could use any of a variety of other systems to accomplish this. For example, resident‐entered ICD‐9 coding systems could be incorporated into electronic discharge summaries, history and physicals, or progress notes. There may also be some practical benefits to having residents learn how to use the ICD‐9 system at this stage of their careers.

There are limitations to this approach as well. The ICD‐9 system was not intended to be used for medical education purposes. There are features of it that can make finding the best diagnosis difficult, and routes to it may at times seem counterintuitive. While we did not carry out resident surveys, a number of residents anecdotally mentioned that it took time to become comfortable using the system, and it could be challenging at times to find a diagnosis description that best fit what they were looking for. To make diagnosis selection easier, we created an organ system‐based dropdown list in the handoff tool so that when residents select an organ system, another list opens up containing commonly used ICD‐9 codes. This grouping is based on organ system alone and does not necessarily follow the ICD‐9 grouping (in contrast, our reported data in this article are all based on ICD‐9 grouping). A search tool to allow searching the entire ICD‐9 database was also made available on the handoff tool. Other factors that could limit diagnosis code accuracy could be lack of clinical knowledge, and error as a result of pressure to come up with a diagnosis because of the hard stop design of our system, in which residents were required to enter a primary diagnosis, potentially causing alert fatigue. A validation assessment that we carried out revealed fairly good agreement with the specific ICD‐9 codes chosen by the resident, but greater accuracy would be desirable. Further education on diagnosis selection and refinements to the handoff tool should help facilitate this. We are currently addressing this by ongoing education on diagnosis selection and by having the hospitalists share the handoff tool with the residents, allowing them to provide direct feedback on diagnostic selections.

More than 19% of the diagnoses selected by residents fell into the category of symptoms and ill‐defined conditions. This raises a number of potential educational issues. One of those is that if residents do, in fact, encounter such entities at such a high frequency, then the internal medicine curriculum must be structured in such a way as to complement this clinical experience with a comprehensive learning program. However, we must also consider the possibility that, in many such instances, a more definitive diagnosis became evident by the time of discharge and this may not have been reflected in the ICD‐9 code that the resident chose. Hence, the category of symptoms and ill‐defined conditions may actually be somewhat smaller than our findings would suggest.

Many issues will need to be addressed as programs obtain more data on their residents' clinical experience. While there may be many reasons to use the ICD‐9 system for selecting diagnoses including those listed above, the system by which ICD‐9 groups diagnoses might not provide ideal educational information, again as the ICD‐9 system was not designed for this purpose. While in this article we have reported the residents' diagnostic encounters grouped according to the ICD‐9 grouping system to provide an initial standardized description, grouping according to another diagnostic system that is felt to be more educationally meaningful may be preferred.

While one might assume that a higher frequency of exposure to certain clinical conditions should enhance competency, that relationship may not be straightforward in internal medicine. For surgical procedures, there are, in fact, data to show improved outcomes for surgeons with higher operative volumes for those procedures,10 but in internal medicine, we do not have data to demonstrate that competence of a resident caring for a particular condition is enhanced by experience alone. Therefore, as programs obtain more data on clinical experience, it will be important that the focus be kept on quality as opposed to quantity.

Obtaining data on resident clinical experience might greatly facilitate experiential learning approaches. For example, as residents go through training and encounter specific diagnostic conditions, those experiences could be supplemented by various learning innovations to make those experiences more meaningful and, hopefully, more likely to result in the development of competence, though that will require measurement. In our program, for example, we have incorporated an approach using illness scenarios, in that when residents have had a certain level of clinical experience with a given clinical condition, they are assembled in small groups and competency‐based case discussions are carried out with a preceptor. In addition, for those instances in which an individual resident may lack direct clinical experience in a certain area, this might be addressed by interventions to increase their contact with those conditions and/or targeted learning interventions to help develop competence. A resident found to be lacking in clinical experience in a certain area could be assigned to the care of more patients with that condition, or to spending more time in a venue in which that condition is more likely to be encountered. Various learning activities including didactics, case discussions, simulation, self‐directed learning, and others could also be used to compensate for such variability. Furthermore, if a residency program's aggregate clinical experience is divergent from some desirable standard yet to be determined, a detailed knowledge of this could help guide that program's curriculum revision. For example, for residents in a program in which there is relatively low exposure to patients with oncological issues, this could be compensated for by external rotations to achieve more clinical experience in oncology, as well as supplementation of the curriculum with additional learning activities in oncology, which could include small group discussions, self‐directed learning activities, case discussions, and others. While at present there are no defined standards for clinical experience and it remains to be seen if there would be a correlation with development of competence, no such standard would serve a purpose if programs did not have reliable and practical means of clinical experience assessment.

In summary, resident‐selected ICD‐9 codes may be a useful means to obtain data regarding resident clinical experience in internal medicine. Such data may be useful to residency training programs in developing new curricula based on experiential learning.

Severe sepsis and septic shock are associated with excess mortality when inappropriate initial antimicrobial therapy, defined as an antimicrobial regimen that lacks in vitro activity against the isolated organism(s) responsible for the infection, is administered.14 Unfortunately, bacterial resistance to antibiotics is increasing and creates a therapeutic challenge for clinicians when treating patients with serious infections, such as severe sepsis. Increasing rates of bacterial resistance leads many clinicians to empirically treat critically ill patients with broad‐spectrum antibiotics, which can perpetuate the cycle of increasing resistance.5, 6 Conversely, inappropriate initial antimicrobial therapy can lead to treatment failures and adverse patient outcomes.7 Individuals with severe sepsis appear to be at particularly high risk of excess mortality when inappropriate initial antimicrobial therapy is administered.8, 9

The most recent Surviving Sepsis Guidelines recommend empiric combination therapy targeting Gram‐negative bacteria, particularly for patients with known or suspected Pseudomonas infections, as a means to decrease the likelihood of administering inappropriate initial antimicrobial therapy.10 However, the selection of an antimicrobial regimen that is active against the causative pathogen(s) is problematic, as the treating physician usually does not know the susceptibilities of the pathogen(s) for the selected empiric antibiotics. Therefore, we performed a study with the main goal of determining whether resistance to the initially prescribed antimicrobial regimen was associated with clinical outcome in patients with severe sepsis attributed to Gram‐negative bacteremia.

Materials and Methods

Results

Antimicrobial Treatment and Resistance

Among the study patients, 358 (66.9%) received cefepime, 102 (19.1%) received piperacillin‐tazobactam, and 75 (14.0%) received a carbapenem (meropenem or imipenem) as their initial antibiotic treatment. There were 169 (31.6%) patients who received initial combination therapy with either an aminoglycoside (n = 99, 58.6%) or ciprofloxacin (n = 70, 41.4%). Eighty‐two (15.3%) patients were infected with a pathogen that was resistant to the initial antibiotic treatment regimen [cefepime (n = 41; 50.0%), piperacillin‐tazobactam (n = 25; 30.5%), or imipenem/meropenem (n = 16; 19.5%), plus either an aminoglycoside or ciprofloxacin (n = 28; 34.1%)], and were classified as receiving inappropriate initial antibiotic therapy. Among the 453 (84.7%) patients infected with a pathogen that was susceptible to the initial antibiotic regimen, there was no relationship identified between minimum inhibitory concentration values and hospital mortality.

Patients infected with a pathogen resistant to the initial antibiotic regimen had significantly greater risk of hospital mortality (63.4% vs 40.0%; P < 0.001) (Figure 1). For the 82 individuals infected with a pathogen that was resistant to the initial antibiotic regimen, no difference in hospital mortality was observed among those prescribed initial combination treatment with an aminoglycoside (n = 17) (64.7% vs 61.1%; P = 0.790) or ciprofloxacin (n = 11) (72.7% vs 61.1%; P = 0.733) compared to monotherapy (n = 54). Similarly, among the patients infected with a pathogen that was susceptible to the initial antibiotic regimen, there was no difference in hospital mortality among those whose bloodstream isolate was only susceptible to the prescribed aminoglycoside (n = 12) compared to patients with isolates that were susceptible to the prescribed beta‐lactam antibiotic (n = 441) (41.7% vs 39.9%; P = 0.902).

Figure 1

Logistic regression analysis identified infection with a pathogen resistant to the initial antibiotic regimen [adjusted odds ratio (AOR), 2.28; 95% confidence interval (CI), 1.69‐3.08; P = 0.006], increasing APACHE II scores (1‐point increments) (AOR, 1.13; 95% CI, 1.10‐1.15; P < 0.001), the need for vasopressors (AOR, 2.57; 95% CI, 2.15‐3.53; P < 0.001), the need for mechanical ventilation (AOR, 2.54; 95% CI, 2.19‐3.47; P < 0.001), healthcare‐associated hospital‐onset infection (AOR, 1.67; 95% CI, 1.32‐2.10; P =0.027), and infection with Pseudomonas aeruginosa (AOR, 2.21; 95% CI, 1.74‐2.86; P =0.002) as independent risk factors for hospital mortality (Hosmer‐Lemeshow goodness‐of‐fit test = 0.305). The model explained between 29.7% (Cox and Snell R square) and 39.8% (Nagelkerke R squared) of the variance in hospital mortality, and correctly classified 75.3% of cases.

Discussion

Our study demonstrated that hospital nonsurvivors with severe sepsis attributed to Gram‐negative bacteremia had significantly greater rates of resistance to their initially prescribed antibiotic regimen compared to hospital survivors. This observation was confirmed in a multivariate analysis controlling for severity of illness and other potential confounding variables. Additionally, acquired organ system derangements and hospital length of stay were greater for patients infected with Gram‐negative pathogens resistant to the empiric antibiotic regimen. We also observed no survival advantage with the use of combination antimicrobial therapy for the subgroup of patients whose pathogens were resistant to the initially prescribed antibiotic regimen. Lastly, no difference in mortality was observed for patients with bacterial isolates that were susceptible only to the prescribed aminoglycoside compared to those with isolates susceptible to the prescribed beta‐lactam antibiotic.

Several previous investigators have linked antibiotic resistance and outcome in patients with serious infections attributed to Gram‐negative bacteria. Tam et al. examined 34 patients with Pseudomonas aeruginosa bacteremia having elevated MICs to piperacillin‐tazobactam (32 g/mL) that were reported as susceptible.19 In seven of these cases, piperacillin‐tazobactam was prescribed empirically, whereas other agents directed against Gram‐negative bacteria were employed in the other patients (carbapenems, aminoglycosides). Thirty‐day mortality was significantly greater for the patients treated with piperacillin‐tazobactam (85.7% vs 22.2%; P = 0.004), and a multivariate analysis found treatment with piperacillin‐tazobactam to be independently associated with 30‐day mortality. Similarly, Bhat et al. examined 204 episodes of bacteremia caused by Gram‐negative bacteria for which patients received cefepime.20 Patients infected with a Gram‐negative bacteria having an MIC to cefepime greater than, or equal to, 8 g/mL had a significantly greater 28‐day mortality compared to patients infected with isolates having an MIC to cefepime that was less than 8 g/mL (54.8% vs 24.1%; P = 0.001).

Our findings are consistent with earlier studies of patients with serious Gram‐negative infections including bacteremia and nosocomial pneumonia. Micek et al. showed that patients with Pseudomonas aeruginosa bacteremia who received inappropriate initial antimicrobial therapy had a greater risk of hospital mortality compared to patients initially treated with an antimicrobial regimen having activity for the Pseudomonas isolate based on in vitro susceptibility testing.21 Similarly, Trouillet et al.,22 Beardsley et al.,23 and Heyland et al.24 found that combination antimicrobial regimens directed against Gram‐negative bacteria in patients with nosocomial pneumonia were more likely to be appropriate based on the antimicrobial susceptibility patterns of the organisms compared to monotherapy. In a more recent study, Micek et al. demonstrated that combination antimicrobial therapy directed against severe sepsis attributed to Gram‐negative bacteria was associated with improved outcomes compared to monotherapy, especially when the combination agent was an aminoglycoside.25 However, empiric combination therapy that included an aminoglycoside was also associated with increased nephrotoxicity which makes the empiric use of aminoglycosides in all patients with suspected Gram‐negative severe sepsis problematic.25, 26 Nevertheless, the use of combination therapy represents a potential strategy to maximize the administration of appropriate treatment for serious Gram‐negative bacterial infections.

Rapid assessment of antimicrobial susceptibility is another strategy that offers the possibility of identifying the resistance pattern of Gram‐negative pathogens quickly in order to provide more appropriate treatment. Bouza et al. found that use of a rapid E‐test on the respiratory specimens of patients with ventilator‐associated pneumonia was associated with fewer days of fever, fewer days of antibiotic administration until resolution of the episode of ventilator‐associated pneumonia, decreased antibiotic consumption, less Clostridium difficile‐associated diarrhea, lower costs of antimicrobial agents, and fewer days receiving mechanical ventilation.27 Other methods for the rapid identification of resistant bacteria include real‐time polymerase chain reaction assays based on hybridization probes to identify specific resistance mechanisms in bacteria.28 Application of such methods for identification of broad categories of resistance mechanisms in Gram‐negative bacteria offer the possibility of tailoring initial antimicrobial regimens in order to provide appropriate therapy in a more timely manner.

Our study has several important limitations that should be noted. First, the study was performed at a single center and the results may not be generalizable to other institutions. However, the findings from other investigators corroborate the importance of antimicrobial resistance as a predictor of outcome for patients with serious Gram‐negative infections.19, 20 Additionally, a similar association has been observed in patients with methicillin‐resistant Staphylococcus aureus bacteremia, supporting the more general importance of antimicrobial resistance as an outcome predictor.29 Second, the method employed for determining MICs was a literature‐based linear regression method correlating disk diffusion diameters with broth dilution MIC determinations. Therefore, the lack of correlation we observed between MIC values and outcome for susceptible Gram‐negative isolates associated with severe sepsis requires further confirmation. Third, we only examined 3 antibiotics, or antibiotic classes, so our results may not be applicable to other agents. This also applies to doripenem, as we did not have that specific carbapenem available at the time this investigation took place.

Another important limitation of our study is the relatively small number of individuals infected with a pathogen that was resistant to the initial treatment regimen, or only susceptible to the aminoglycoside when combination therapy was prescribed. This limited our ability to detect meaningful associations in these subgroups of patients, to include whether or not combination therapy influenced their clinical outcome. Finally, we did not examine the exact timing of antibiotic therapy relative to the onset of severe sepsis. Instead we used a 12‐hour window from when subsequently positive blood cultures were drawn to the administration of initial antibiotic therapy. Other investigators have shown that delays in initial appropriate therapy of more than one hour for patients with septic shock increases the risk of death.9, 30 Failure to include the exact timing of therapy could have resulted in a final multivariate model that includes prediction variables that would not otherwise have been incorporated.

In summary, we demonstrated that resistance to the initial antibiotic treatment regimen was associated with a greater risk of hospital mortality in patients with severe sepsis attributed to Gram‐negative bacteremia. These findings imply that more rapid assessment of antimicrobial susceptibility could result in improved prescription of antibiotics in order to maximize initial administration of appropriate therapy. Future studies are required to address whether rapid determination of antimicrobial susceptibility can result in more effective administration of appropriate therapy, and if this can result in improved patient outcomes.

Severe sepsis and septic shock are associated with excess mortality when inappropriate initial antimicrobial therapy, defined as an antimicrobial regimen that lacks in vitro activity against the isolated organism(s) responsible for the infection, is administered.14 Unfortunately, bacterial resistance to antibiotics is increasing and creates a therapeutic challenge for clinicians when treating patients with serious infections, such as severe sepsis. Increasing rates of bacterial resistance leads many clinicians to empirically treat critically ill patients with broad‐spectrum antibiotics, which can perpetuate the cycle of increasing resistance.5, 6 Conversely, inappropriate initial antimicrobial therapy can lead to treatment failures and adverse patient outcomes.7 Individuals with severe sepsis appear to be at particularly high risk of excess mortality when inappropriate initial antimicrobial therapy is administered.8, 9

The most recent Surviving Sepsis Guidelines recommend empiric combination therapy targeting Gram‐negative bacteria, particularly for patients with known or suspected Pseudomonas infections, as a means to decrease the likelihood of administering inappropriate initial antimicrobial therapy.10 However, the selection of an antimicrobial regimen that is active against the causative pathogen(s) is problematic, as the treating physician usually does not know the susceptibilities of the pathogen(s) for the selected empiric antibiotics. Therefore, we performed a study with the main goal of determining whether resistance to the initially prescribed antimicrobial regimen was associated with clinical outcome in patients with severe sepsis attributed to Gram‐negative bacteremia.

Materials and Methods

Results

Antimicrobial Treatment and Resistance

Among the study patients, 358 (66.9%) received cefepime, 102 (19.1%) received piperacillin‐tazobactam, and 75 (14.0%) received a carbapenem (meropenem or imipenem) as their initial antibiotic treatment. There were 169 (31.6%) patients who received initial combination therapy with either an aminoglycoside (n = 99, 58.6%) or ciprofloxacin (n = 70, 41.4%). Eighty‐two (15.3%) patients were infected with a pathogen that was resistant to the initial antibiotic treatment regimen [cefepime (n = 41; 50.0%), piperacillin‐tazobactam (n = 25; 30.5%), or imipenem/meropenem (n = 16; 19.5%), plus either an aminoglycoside or ciprofloxacin (n = 28; 34.1%)], and were classified as receiving inappropriate initial antibiotic therapy. Among the 453 (84.7%) patients infected with a pathogen that was susceptible to the initial antibiotic regimen, there was no relationship identified between minimum inhibitory concentration values and hospital mortality.

Patients infected with a pathogen resistant to the initial antibiotic regimen had significantly greater risk of hospital mortality (63.4% vs 40.0%; P < 0.001) (Figure 1). For the 82 individuals infected with a pathogen that was resistant to the initial antibiotic regimen, no difference in hospital mortality was observed among those prescribed initial combination treatment with an aminoglycoside (n = 17) (64.7% vs 61.1%; P = 0.790) or ciprofloxacin (n = 11) (72.7% vs 61.1%; P = 0.733) compared to monotherapy (n = 54). Similarly, among the patients infected with a pathogen that was susceptible to the initial antibiotic regimen, there was no difference in hospital mortality among those whose bloodstream isolate was only susceptible to the prescribed aminoglycoside (n = 12) compared to patients with isolates that were susceptible to the prescribed beta‐lactam antibiotic (n = 441) (41.7% vs 39.9%; P = 0.902).

Figure 1

Logistic regression analysis identified infection with a pathogen resistant to the initial antibiotic regimen [adjusted odds ratio (AOR), 2.28; 95% confidence interval (CI), 1.69‐3.08; P = 0.006], increasing APACHE II scores (1‐point increments) (AOR, 1.13; 95% CI, 1.10‐1.15; P < 0.001), the need for vasopressors (AOR, 2.57; 95% CI, 2.15‐3.53; P < 0.001), the need for mechanical ventilation (AOR, 2.54; 95% CI, 2.19‐3.47; P < 0.001), healthcare‐associated hospital‐onset infection (AOR, 1.67; 95% CI, 1.32‐2.10; P =0.027), and infection with Pseudomonas aeruginosa (AOR, 2.21; 95% CI, 1.74‐2.86; P =0.002) as independent risk factors for hospital mortality (Hosmer‐Lemeshow goodness‐of‐fit test = 0.305). The model explained between 29.7% (Cox and Snell R square) and 39.8% (Nagelkerke R squared) of the variance in hospital mortality, and correctly classified 75.3% of cases.

Discussion

Our study demonstrated that hospital nonsurvivors with severe sepsis attributed to Gram‐negative bacteremia had significantly greater rates of resistance to their initially prescribed antibiotic regimen compared to hospital survivors. This observation was confirmed in a multivariate analysis controlling for severity of illness and other potential confounding variables. Additionally, acquired organ system derangements and hospital length of stay were greater for patients infected with Gram‐negative pathogens resistant to the empiric antibiotic regimen. We also observed no survival advantage with the use of combination antimicrobial therapy for the subgroup of patients whose pathogens were resistant to the initially prescribed antibiotic regimen. Lastly, no difference in mortality was observed for patients with bacterial isolates that were susceptible only to the prescribed aminoglycoside compared to those with isolates susceptible to the prescribed beta‐lactam antibiotic.

Several previous investigators have linked antibiotic resistance and outcome in patients with serious infections attributed to Gram‐negative bacteria. Tam et al. examined 34 patients with Pseudomonas aeruginosa bacteremia having elevated MICs to piperacillin‐tazobactam (32 g/mL) that were reported as susceptible.19 In seven of these cases, piperacillin‐tazobactam was prescribed empirically, whereas other agents directed against Gram‐negative bacteria were employed in the other patients (carbapenems, aminoglycosides). Thirty‐day mortality was significantly greater for the patients treated with piperacillin‐tazobactam (85.7% vs 22.2%; P = 0.004), and a multivariate analysis found treatment with piperacillin‐tazobactam to be independently associated with 30‐day mortality. Similarly, Bhat et al. examined 204 episodes of bacteremia caused by Gram‐negative bacteria for which patients received cefepime.20 Patients infected with a Gram‐negative bacteria having an MIC to cefepime greater than, or equal to, 8 g/mL had a significantly greater 28‐day mortality compared to patients infected with isolates having an MIC to cefepime that was less than 8 g/mL (54.8% vs 24.1%; P = 0.001).

Our findings are consistent with earlier studies of patients with serious Gram‐negative infections including bacteremia and nosocomial pneumonia. Micek et al. showed that patients with Pseudomonas aeruginosa bacteremia who received inappropriate initial antimicrobial therapy had a greater risk of hospital mortality compared to patients initially treated with an antimicrobial regimen having activity for the Pseudomonas isolate based on in vitro susceptibility testing.21 Similarly, Trouillet et al.,22 Beardsley et al.,23 and Heyland et al.24 found that combination antimicrobial regimens directed against Gram‐negative bacteria in patients with nosocomial pneumonia were more likely to be appropriate based on the antimicrobial susceptibility patterns of the organisms compared to monotherapy. In a more recent study, Micek et al. demonstrated that combination antimicrobial therapy directed against severe sepsis attributed to Gram‐negative bacteria was associated with improved outcomes compared to monotherapy, especially when the combination agent was an aminoglycoside.25 However, empiric combination therapy that included an aminoglycoside was also associated with increased nephrotoxicity which makes the empiric use of aminoglycosides in all patients with suspected Gram‐negative severe sepsis problematic.25, 26 Nevertheless, the use of combination therapy represents a potential strategy to maximize the administration of appropriate treatment for serious Gram‐negative bacterial infections.

Rapid assessment of antimicrobial susceptibility is another strategy that offers the possibility of identifying the resistance pattern of Gram‐negative pathogens quickly in order to provide more appropriate treatment. Bouza et al. found that use of a rapid E‐test on the respiratory specimens of patients with ventilator‐associated pneumonia was associated with fewer days of fever, fewer days of antibiotic administration until resolution of the episode of ventilator‐associated pneumonia, decreased antibiotic consumption, less Clostridium difficile‐associated diarrhea, lower costs of antimicrobial agents, and fewer days receiving mechanical ventilation.27 Other methods for the rapid identification of resistant bacteria include real‐time polymerase chain reaction assays based on hybridization probes to identify specific resistance mechanisms in bacteria.28 Application of such methods for identification of broad categories of resistance mechanisms in Gram‐negative bacteria offer the possibility of tailoring initial antimicrobial regimens in order to provide appropriate therapy in a more timely manner.

Our study has several important limitations that should be noted. First, the study was performed at a single center and the results may not be generalizable to other institutions. However, the findings from other investigators corroborate the importance of antimicrobial resistance as a predictor of outcome for patients with serious Gram‐negative infections.19, 20 Additionally, a similar association has been observed in patients with methicillin‐resistant Staphylococcus aureus bacteremia, supporting the more general importance of antimicrobial resistance as an outcome predictor.29 Second, the method employed for determining MICs was a literature‐based linear regression method correlating disk diffusion diameters with broth dilution MIC determinations. Therefore, the lack of correlation we observed between MIC values and outcome for susceptible Gram‐negative isolates associated with severe sepsis requires further confirmation. Third, we only examined 3 antibiotics, or antibiotic classes, so our results may not be applicable to other agents. This also applies to doripenem, as we did not have that specific carbapenem available at the time this investigation took place.

Another important limitation of our study is the relatively small number of individuals infected with a pathogen that was resistant to the initial treatment regimen, or only susceptible to the aminoglycoside when combination therapy was prescribed. This limited our ability to detect meaningful associations in these subgroups of patients, to include whether or not combination therapy influenced their clinical outcome. Finally, we did not examine the exact timing of antibiotic therapy relative to the onset of severe sepsis. Instead we used a 12‐hour window from when subsequently positive blood cultures were drawn to the administration of initial antibiotic therapy. Other investigators have shown that delays in initial appropriate therapy of more than one hour for patients with septic shock increases the risk of death.9, 30 Failure to include the exact timing of therapy could have resulted in a final multivariate model that includes prediction variables that would not otherwise have been incorporated.

In summary, we demonstrated that resistance to the initial antibiotic treatment regimen was associated with a greater risk of hospital mortality in patients with severe sepsis attributed to Gram‐negative bacteremia. These findings imply that more rapid assessment of antimicrobial susceptibility could result in improved prescription of antibiotics in order to maximize initial administration of appropriate therapy. Future studies are required to address whether rapid determination of antimicrobial susceptibility can result in more effective administration of appropriate therapy, and if this can result in improved patient outcomes.

Severe sepsis and septic shock are associated with excess mortality when inappropriate initial antimicrobial therapy, defined as an antimicrobial regimen that lacks in vitro activity against the isolated organism(s) responsible for the infection, is administered.14 Unfortunately, bacterial resistance to antibiotics is increasing and creates a therapeutic challenge for clinicians when treating patients with serious infections, such as severe sepsis. Increasing rates of bacterial resistance leads many clinicians to empirically treat critically ill patients with broad‐spectrum antibiotics, which can perpetuate the cycle of increasing resistance.5, 6 Conversely, inappropriate initial antimicrobial therapy can lead to treatment failures and adverse patient outcomes.7 Individuals with severe sepsis appear to be at particularly high risk of excess mortality when inappropriate initial antimicrobial therapy is administered.8, 9

The most recent Surviving Sepsis Guidelines recommend empiric combination therapy targeting Gram‐negative bacteria, particularly for patients with known or suspected Pseudomonas infections, as a means to decrease the likelihood of administering inappropriate initial antimicrobial therapy.10 However, the selection of an antimicrobial regimen that is active against the causative pathogen(s) is problematic, as the treating physician usually does not know the susceptibilities of the pathogen(s) for the selected empiric antibiotics. Therefore, we performed a study with the main goal of determining whether resistance to the initially prescribed antimicrobial regimen was associated with clinical outcome in patients with severe sepsis attributed to Gram‐negative bacteremia.

Materials and Methods

Results

Antimicrobial Treatment and Resistance

Among the study patients, 358 (66.9%) received cefepime, 102 (19.1%) received piperacillin‐tazobactam, and 75 (14.0%) received a carbapenem (meropenem or imipenem) as their initial antibiotic treatment. There were 169 (31.6%) patients who received initial combination therapy with either an aminoglycoside (n = 99, 58.6%) or ciprofloxacin (n = 70, 41.4%). Eighty‐two (15.3%) patients were infected with a pathogen that was resistant to the initial antibiotic treatment regimen [cefepime (n = 41; 50.0%), piperacillin‐tazobactam (n = 25; 30.5%), or imipenem/meropenem (n = 16; 19.5%), plus either an aminoglycoside or ciprofloxacin (n = 28; 34.1%)], and were classified as receiving inappropriate initial antibiotic therapy. Among the 453 (84.7%) patients infected with a pathogen that was susceptible to the initial antibiotic regimen, there was no relationship identified between minimum inhibitory concentration values and hospital mortality.

Patients infected with a pathogen resistant to the initial antibiotic regimen had significantly greater risk of hospital mortality (63.4% vs 40.0%; P < 0.001) (Figure 1). For the 82 individuals infected with a pathogen that was resistant to the initial antibiotic regimen, no difference in hospital mortality was observed among those prescribed initial combination treatment with an aminoglycoside (n = 17) (64.7% vs 61.1%; P = 0.790) or ciprofloxacin (n = 11) (72.7% vs 61.1%; P = 0.733) compared to monotherapy (n = 54). Similarly, among the patients infected with a pathogen that was susceptible to the initial antibiotic regimen, there was no difference in hospital mortality among those whose bloodstream isolate was only susceptible to the prescribed aminoglycoside (n = 12) compared to patients with isolates that were susceptible to the prescribed beta‐lactam antibiotic (n = 441) (41.7% vs 39.9%; P = 0.902).

Figure 1

Logistic regression analysis identified infection with a pathogen resistant to the initial antibiotic regimen [adjusted odds ratio (AOR), 2.28; 95% confidence interval (CI), 1.69‐3.08; P = 0.006], increasing APACHE II scores (1‐point increments) (AOR, 1.13; 95% CI, 1.10‐1.15; P < 0.001), the need for vasopressors (AOR, 2.57; 95% CI, 2.15‐3.53; P < 0.001), the need for mechanical ventilation (AOR, 2.54; 95% CI, 2.19‐3.47; P < 0.001), healthcare‐associated hospital‐onset infection (AOR, 1.67; 95% CI, 1.32‐2.10; P =0.027), and infection with Pseudomonas aeruginosa (AOR, 2.21; 95% CI, 1.74‐2.86; P =0.002) as independent risk factors for hospital mortality (Hosmer‐Lemeshow goodness‐of‐fit test = 0.305). The model explained between 29.7% (Cox and Snell R square) and 39.8% (Nagelkerke R squared) of the variance in hospital mortality, and correctly classified 75.3% of cases.

Discussion

Our study demonstrated that hospital nonsurvivors with severe sepsis attributed to Gram‐negative bacteremia had significantly greater rates of resistance to their initially prescribed antibiotic regimen compared to hospital survivors. This observation was confirmed in a multivariate analysis controlling for severity of illness and other potential confounding variables. Additionally, acquired organ system derangements and hospital length of stay were greater for patients infected with Gram‐negative pathogens resistant to the empiric antibiotic regimen. We also observed no survival advantage with the use of combination antimicrobial therapy for the subgroup of patients whose pathogens were resistant to the initially prescribed antibiotic regimen. Lastly, no difference in mortality was observed for patients with bacterial isolates that were susceptible only to the prescribed aminoglycoside compared to those with isolates susceptible to the prescribed beta‐lactam antibiotic.

Several previous investigators have linked antibiotic resistance and outcome in patients with serious infections attributed to Gram‐negative bacteria. Tam et al. examined 34 patients with Pseudomonas aeruginosa bacteremia having elevated MICs to piperacillin‐tazobactam (32 g/mL) that were reported as susceptible.19 In seven of these cases, piperacillin‐tazobactam was prescribed empirically, whereas other agents directed against Gram‐negative bacteria were employed in the other patients (carbapenems, aminoglycosides). Thirty‐day mortality was significantly greater for the patients treated with piperacillin‐tazobactam (85.7% vs 22.2%; P = 0.004), and a multivariate analysis found treatment with piperacillin‐tazobactam to be independently associated with 30‐day mortality. Similarly, Bhat et al. examined 204 episodes of bacteremia caused by Gram‐negative bacteria for which patients received cefepime.20 Patients infected with a Gram‐negative bacteria having an MIC to cefepime greater than, or equal to, 8 g/mL had a significantly greater 28‐day mortality compared to patients infected with isolates having an MIC to cefepime that was less than 8 g/mL (54.8% vs 24.1%; P = 0.001).

Our findings are consistent with earlier studies of patients with serious Gram‐negative infections including bacteremia and nosocomial pneumonia. Micek et al. showed that patients with Pseudomonas aeruginosa bacteremia who received inappropriate initial antimicrobial therapy had a greater risk of hospital mortality compared to patients initially treated with an antimicrobial regimen having activity for the Pseudomonas isolate based on in vitro susceptibility testing.21 Similarly, Trouillet et al.,22 Beardsley et al.,23 and Heyland et al.24 found that combination antimicrobial regimens directed against Gram‐negative bacteria in patients with nosocomial pneumonia were more likely to be appropriate based on the antimicrobial susceptibility patterns of the organisms compared to monotherapy. In a more recent study, Micek et al. demonstrated that combination antimicrobial therapy directed against severe sepsis attributed to Gram‐negative bacteria was associated with improved outcomes compared to monotherapy, especially when the combination agent was an aminoglycoside.25 However, empiric combination therapy that included an aminoglycoside was also associated with increased nephrotoxicity which makes the empiric use of aminoglycosides in all patients with suspected Gram‐negative severe sepsis problematic.25, 26 Nevertheless, the use of combination therapy represents a potential strategy to maximize the administration of appropriate treatment for serious Gram‐negative bacterial infections.

Rapid assessment of antimicrobial susceptibility is another strategy that offers the possibility of identifying the resistance pattern of Gram‐negative pathogens quickly in order to provide more appropriate treatment. Bouza et al. found that use of a rapid E‐test on the respiratory specimens of patients with ventilator‐associated pneumonia was associated with fewer days of fever, fewer days of antibiotic administration until resolution of the episode of ventilator‐associated pneumonia, decreased antibiotic consumption, less Clostridium difficile‐associated diarrhea, lower costs of antimicrobial agents, and fewer days receiving mechanical ventilation.27 Other methods for the rapid identification of resistant bacteria include real‐time polymerase chain reaction assays based on hybridization probes to identify specific resistance mechanisms in bacteria.28 Application of such methods for identification of broad categories of resistance mechanisms in Gram‐negative bacteria offer the possibility of tailoring initial antimicrobial regimens in order to provide appropriate therapy in a more timely manner.

Our study has several important limitations that should be noted. First, the study was performed at a single center and the results may not be generalizable to other institutions. However, the findings from other investigators corroborate the importance of antimicrobial resistance as a predictor of outcome for patients with serious Gram‐negative infections.19, 20 Additionally, a similar association has been observed in patients with methicillin‐resistant Staphylococcus aureus bacteremia, supporting the more general importance of antimicrobial resistance as an outcome predictor.29 Second, the method employed for determining MICs was a literature‐based linear regression method correlating disk diffusion diameters with broth dilution MIC determinations. Therefore, the lack of correlation we observed between MIC values and outcome for susceptible Gram‐negative isolates associated with severe sepsis requires further confirmation. Third, we only examined 3 antibiotics, or antibiotic classes, so our results may not be applicable to other agents. This also applies to doripenem, as we did not have that specific carbapenem available at the time this investigation took place.

Another important limitation of our study is the relatively small number of individuals infected with a pathogen that was resistant to the initial treatment regimen, or only susceptible to the aminoglycoside when combination therapy was prescribed. This limited our ability to detect meaningful associations in these subgroups of patients, to include whether or not combination therapy influenced their clinical outcome. Finally, we did not examine the exact timing of antibiotic therapy relative to the onset of severe sepsis. Instead we used a 12‐hour window from when subsequently positive blood cultures were drawn to the administration of initial antibiotic therapy. Other investigators have shown that delays in initial appropriate therapy of more than one hour for patients with septic shock increases the risk of death.9, 30 Failure to include the exact timing of therapy could have resulted in a final multivariate model that includes prediction variables that would not otherwise have been incorporated.

In summary, we demonstrated that resistance to the initial antibiotic treatment regimen was associated with a greater risk of hospital mortality in patients with severe sepsis attributed to Gram‐negative bacteremia. These findings imply that more rapid assessment of antimicrobial susceptibility could result in improved prescription of antibiotics in order to maximize initial administration of appropriate therapy. Future studies are required to address whether rapid determination of antimicrobial susceptibility can result in more effective administration of appropriate therapy, and if this can result in improved patient outcomes.

In personal terms, all Americans are connected by recollections of the experience. 97% can remember exactly where they were or what they were doing the moment they heard about the attacks. (Pew Research survey, September 5, 2002)

The classic flashbulb memories of our parents' generation, who were young adults in the 1960s, were the assassinations of Martin Luther King and President John Kennedy. In the same way, the 9/11 attacks seem destined to endure as our generation's flashbulb memory, with the Space Shuttle Challenger explosion a distant second for those of us on the far side of age 40. Few of us are likely to ever forget the grief, anger, and confusion of September 11, 2001 and the days that followed, and it seems appropriate 10 years later to remember those who died that day, and to reflect on the lessons we learnedor should have. As hospitalists, we are at least somewhat familiar with the tragic and senseless loss of life that day, as the terrorist attacks were in a sense a reflection, writ large, of the unexpected and inexplicable deaths we have all been a part of: the healthy young woman exsanguinating from DIC in the immediate postpartum period, the preschool teacher rapidly succumbing to pneumococcal meningitis, the young adult dying of acute leukemia or necrotizing fasciitis.

On September 11, 2001, one of us (B.J.H.) was 2 years out of residency and in private practice near San Francisco.

For the other of us (J.C.P.), the news came in a patient's room during rounds.

What lessons should we take away from the 9/11 tragedy a decade later, and indeed from our work with our patients? Certainly that mass casualties and disaster preparedness are an unfortunate fact of life in the 21st century, and that hospitalists have a responsibility to engage with our institutions in preparing for these eventualities. Possibly that life is uncertain and, at best, goes by much more quickly than any of us could have imagined when we embarked on our medical training. In the end, that our lives are measured primarily not by the number of years we live, but by how we live them, and the lives that we touch along the way.

Once a year, we pause to remember the nearly 3000 individuals who lost their lives on 9/11. As hospitalists, we practice a profession that demands a great deal from us and encourages workaholism; perhaps the 10th anniversary of those heinous acts should make each of us, as we remember the lives touched most directly by the attacks on the World Trade Center, the Pentagon, and United Flight 93, also pause to consider our work‐life balance, and to ensure that we are reserving sufficient quality time for our families and friends, as well as for activities that renew and enrich us.

In personal terms, all Americans are connected by recollections of the experience. 97% can remember exactly where they were or what they were doing the moment they heard about the attacks. (Pew Research survey, September 5, 2002)

The classic flashbulb memories of our parents' generation, who were young adults in the 1960s, were the assassinations of Martin Luther King and President John Kennedy. In the same way, the 9/11 attacks seem destined to endure as our generation's flashbulb memory, with the Space Shuttle Challenger explosion a distant second for those of us on the far side of age 40. Few of us are likely to ever forget the grief, anger, and confusion of September 11, 2001 and the days that followed, and it seems appropriate 10 years later to remember those who died that day, and to reflect on the lessons we learnedor should have. As hospitalists, we are at least somewhat familiar with the tragic and senseless loss of life that day, as the terrorist attacks were in a sense a reflection, writ large, of the unexpected and inexplicable deaths we have all been a part of: the healthy young woman exsanguinating from DIC in the immediate postpartum period, the preschool teacher rapidly succumbing to pneumococcal meningitis, the young adult dying of acute leukemia or necrotizing fasciitis.

On September 11, 2001, one of us (B.J.H.) was 2 years out of residency and in private practice near San Francisco.

For the other of us (J.C.P.), the news came in a patient's room during rounds.

What lessons should we take away from the 9/11 tragedy a decade later, and indeed from our work with our patients? Certainly that mass casualties and disaster preparedness are an unfortunate fact of life in the 21st century, and that hospitalists have a responsibility to engage with our institutions in preparing for these eventualities. Possibly that life is uncertain and, at best, goes by much more quickly than any of us could have imagined when we embarked on our medical training. In the end, that our lives are measured primarily not by the number of years we live, but by how we live them, and the lives that we touch along the way.

Once a year, we pause to remember the nearly 3000 individuals who lost their lives on 9/11. As hospitalists, we practice a profession that demands a great deal from us and encourages workaholism; perhaps the 10th anniversary of those heinous acts should make each of us, as we remember the lives touched most directly by the attacks on the World Trade Center, the Pentagon, and United Flight 93, also pause to consider our work‐life balance, and to ensure that we are reserving sufficient quality time for our families and friends, as well as for activities that renew and enrich us.

In personal terms, all Americans are connected by recollections of the experience. 97% can remember exactly where they were or what they were doing the moment they heard about the attacks. (Pew Research survey, September 5, 2002)

The classic flashbulb memories of our parents' generation, who were young adults in the 1960s, were the assassinations of Martin Luther King and President John Kennedy. In the same way, the 9/11 attacks seem destined to endure as our generation's flashbulb memory, with the Space Shuttle Challenger explosion a distant second for those of us on the far side of age 40. Few of us are likely to ever forget the grief, anger, and confusion of September 11, 2001 and the days that followed, and it seems appropriate 10 years later to remember those who died that day, and to reflect on the lessons we learnedor should have. As hospitalists, we are at least somewhat familiar with the tragic and senseless loss of life that day, as the terrorist attacks were in a sense a reflection, writ large, of the unexpected and inexplicable deaths we have all been a part of: the healthy young woman exsanguinating from DIC in the immediate postpartum period, the preschool teacher rapidly succumbing to pneumococcal meningitis, the young adult dying of acute leukemia or necrotizing fasciitis.

On September 11, 2001, one of us (B.J.H.) was 2 years out of residency and in private practice near San Francisco.

For the other of us (J.C.P.), the news came in a patient's room during rounds.

What lessons should we take away from the 9/11 tragedy a decade later, and indeed from our work with our patients? Certainly that mass casualties and disaster preparedness are an unfortunate fact of life in the 21st century, and that hospitalists have a responsibility to engage with our institutions in preparing for these eventualities. Possibly that life is uncertain and, at best, goes by much more quickly than any of us could have imagined when we embarked on our medical training. In the end, that our lives are measured primarily not by the number of years we live, but by how we live them, and the lives that we touch along the way.

Once a year, we pause to remember the nearly 3000 individuals who lost their lives on 9/11. As hospitalists, we practice a profession that demands a great deal from us and encourages workaholism; perhaps the 10th anniversary of those heinous acts should make each of us, as we remember the lives touched most directly by the attacks on the World Trade Center, the Pentagon, and United Flight 93, also pause to consider our work‐life balance, and to ensure that we are reserving sufficient quality time for our families and friends, as well as for activities that renew and enrich us.

Hospital readmissions have become a focus of national attention as a potential indicator of poor quality and health care waste.13 Geographic variations in readmission rates, a high rate of unplanned readmissions, and the emergence of promising interventions all suggest that some portion of readmissions are preventable.4, 5 This work adds to the work of the Agency for Healthcare Research and Quality (AHRQ) on reports of preventable hospital admissions, using hospitalization rates for ambulatory‐sensitive conditions as prevention quality indicators.6

The actual proportion of preventable readmissions is unknown. In previous research using physician reviewers, estimates have ranged from 5% to 38%.713 More recently, studies using a methodology based on relationships between diagnoses at the initial and subsequent hospitalizations have flagged as many as 76% of 30‐day readmissions as preventable.14

Understanding the preventability of readmissions is important if we are to gauge the true size of this quality and cost opportunity. Moreover, it is important to assess the beliefs of the front‐line clinicians who will be playing key roles in prevention.

The objective of the current study was to examine readmission preventability from the perspective of hospital medicine experts practicing at a community hospital. Through detailed chart review, we identify patient factors and care processes that affect preventability and describe clinicians' ideas for preventing future readmissions.

METHODS

RESULTS

Two hundred thirteen patients (85%) had a single readmission. Another 33 patients had 2 readmissions, and 5 patients accounted for 21 readmissions for a total sample of 300 cases. Table 1 provides characteristics of readmitted patients. They were likely to be elderly; the mean (SD) age was 75.3 (15.3), and more than 48% were 80 or older. Sixty‐six percent of patients were taking more than ten medications, and a quarter (25%) had more than three new medications prescribed at discharge. Frequent diagnoses at the index admission included renal insufficiency, heart failure, dementia, atrial fibrillation, and chronic obstructive pulmonary disease (COPD). The majority of cases had more than one diagnosis identified at their first admission. These diagnoses are what hospitalists believe are significant patient issues rather than the hospital‐coded principal and secondary diagnoses.

Sixty‐four percent readmitted cases had been discharged to home (including those with home services), and 36% were discharged to a care facility (skilled nursing facility [SNF], foster care, assisted living) (Table 2). Fifty‐eight percent of cases were readmitted within seven days of the index admission, and another 29% within the first two weeks. Exactly 75% of the time, the readmission was for the same or related diagnosis as the index admission. Primary care follow‐up did not occur as recommended 69% of the time, and 57% of the time the patient was readmitted prior seeing their primary care physician (PCP).

Overall, only 15% of readmissions were termed preventable by the hospital reviewers, although another 46% were deemed possibly preventable. Preventability ratings varied by reviewer, ranging from a high of 27% to a low of 0% among hospitalists rating ten or more cases (Table 3). There was similar variation in the number of recommended interventions. For readmissions deemed preventable or possibly preventable, the number of potential interventions ranged from more than three per patient to less than one per patient.

The most frequently mentioned intervention that could have prevented a readmission was to extend the hospital stay by one to two days (Table 4). An earlier PCP appointment was suggested for another 21% of readmissions. Other interventions received a scattering of mentions. The types of recommended interventions varied with the rater's perception of preventability (Figure 1, available online). Hospitalists were more likely to recommend a longer initial stay, medication changes, or additional education at discharge, and earlier contact from a care facility, for readmissions they thought were preventable. For possibly preventable readmissions, these same recommendations were important, but hospitalists were also likely to recommend case management, disposition to a higher level of care, or a home health visit.

Table 5 shows the most important characteristics associated with preventability, using a cutoff of 0.2 in statistical significance. Readmissions for the same diagnosis were more likely than others to be rated preventable, as were cases with a short readmission interval, more than three new medications at discharge, and patients with COPD or depression/anxiety. Initial hospital length of stay did not influence preventability, nor did it influence the likelihood of a reviewer recommending a longer initial stay.

Potential predictors associated with preventability were included in a hierarchical logistic regression model, with hospital site and reviewer included as random effects. In this modeling, preventable readmissions were more likely than nonpreventable readmissions to be influenced by three process factors: having the same index and readmission diagnosis; readmission in the first post‐hospital week; being readmitted prior to a primary care follow‐up; and three patient factors: having more than three new discharge medications, having anticoagulation treatment, and having a COPD diagnosis (data available online). Other chronic diseases, age, discharge location, or previous readmissions were not important in the rating of preventability. When entered as random effects in a hierarchical logistic regression model, the categorical variable representing hospital site did not significantly improve prediction (P = 0.42), but the reviewer variable (categorized by the top six reviewers and others) had marginal significance at P = 0.088.

DISCUSSION

Reported high Medicare 30‐day readmission rates and associate excess costs have created a national climate for eliminating unnecessary hospital readmissions.1 Recently passed healthcare legislation in the USA will put in place diagnosis‐related group (DRG) payment reductions for excess readmission rates by 2013. As the definitions and methodologies for determining the relatedness and preventable nature of readmissions continues to be clarified, this study contributes to the understanding of preventability and specific preventative strategies from a physician perspective. Although potential savings in readmission reduction work is attractive, our study indicates that most front‐line clinicians are not convinced that a large portion of readmissions are preventable.

The proportion of preventable readmissions found in our study is very much in line with previous research.713 Certain predictors of preventable readmissions were also similar. Several researchers have found that preventable readmissions are more likely to be early,8, 10, 12 and have the same or related diagnosis as the initial stay.8 On the other hand, our data did not show an independent effect of age on preventability, as others have suggested.9, 17 Patients with a large number of diagnoses and medications have been shown to be at risk for preventable readmissions,9 but the importance of new discharge medications has not been widely researched and is a factor that deserves further exploration.

One key message from our study was found in the variation in the ratings of preventability by individual physicians. At first blush, it may appear to reflect a lack of inter‐rater reliability or understanding of the underlying concept of preventability. We believe this is unlikely, given the discussions among raters and the clear descriptions offered in writing. Moreover, there was much less variation in other judgments such as the ratings of relatedness of the readmission diagnosis (chi‐square = 21.7, P = .041)

There are a number of possible reasons for variation in reviewer ratings of preventability. Reviewers did vary with regard to age, experience, tenure in the organization, gender, and full/part‐time status. They practiced at different hospitals. None of these factors were related to ratings of preventability. On the other hand, three explanations are worth noting.

First, the hierarchical regression models found that reviewer only slightly improved prediction (P = 0.088), above and beyond the other diagnosis and process factors. This would lead us to reject the factor of reviewer as the most important predictor of preventability; the other case characteristics mentioned above were more important.

Second, the three hospitalists who were more optimistic (rated more cases as preventable) reviewed more charts than others. It is possible that these three were more engaged, not only in the chart review process, but more eager to uncover potential remedies to prevent readmissions. While generating more ideas about how to do that, they rated more readmissions as preventable. We do not believe that actually doing more reviews caused them to rate a greater portion as preventable; none of the reviewers showed progression to more preventable ratings over time (analysis not shown).

Finally, it is worth noting that two of the more optimistic physicians had previous primary care experience. This is an intriguing explanation that would benefit from further research. First‐hand experience with primary care case management, rapid appointment follow‐up, home service referrals, and the like may give the practicing hospitalist reason to believe that actions in the ambulatory setting can prevent readmissions.

Regardless of the source, the variation demonstrates cultural or philosophical biases among clinicians regarding how much influence additional planning, education, and care coordination can have on readmissions. We believe that this variation must be addressed in the implementation of readmission reduction programs. Physician engagement will be more likely if there is optimism about the potential to prevent readmissions. In addition, it will be important to develop more consensus about effective interventions from the perspectives of hospital physicians, primary care physicians, nurses, and patients, as others have alluded.18, 19

The significant rate of related readmissions (75%) has implications for the potential Centers for Medicare and Medicaid Services (CMS) methodology that will be used to reduce DRG payments, given the legislation's current intent to exclude only unrelated and planned readmissions from the calculations. Providing clear definitions on relatedness and a methodology to code this criterion in administrative datasets may need to be developed. The views of hospitalists in the current study suggest that the relatedness methodology may be overly sensitive and not yet specific enough to isolate truly preventable readmissions. Less than a quarter of related readmissions were deemed preventable by these raters.

Hospitalists found both patient and process factors important in assessing the preventability of a readmission. This kind of analysis can point to subgroups with potential for targeted intervention. For example, over a third of patients readmitted within a week for the same diagnosis were rated as preventable, indicating a critical follow‐up period for some patients. Higher ratings of preventability among the readmissions for patients on anticoagulation or who were given more than three new medications at discharge indicates that better medication management may indeed be a fruitful strategy for readmission reduction.

The finding that increasing the length of the initial hospital stay was rated as the most prevalent strategy to mitigate against readmission in our retrospective review was surprising. It emphasizes the tension between efficient hospital throughput which reduces unnecessary hospital days and the necessity for appropriate monitoring to ensure clinical stability prior to discharge. Excess hospital days can prolong the exposure to a multitude of hospital acquired conditions (HAC), and this risk must be weighed against a longer length of stay and the time required delivering the appropriate hospital services.

Exploring alternative strategies to reduce readmissions without increasing the hospital length of stay is a reasonable response to this tension. Better discharge education and attention to discharge medications and dosages were also recommended strategies for preventable readmissions. These are interventions hospitalists are familiar with and can control. Relatively smaller percentages of patients were thought to benefit from case management, hospice, home health, or an MD visit to their nursing home, and hospitalists were more likely to recommend these for the possibly preventable patients. These interventions are not fully implemented within the study health system so there is understandably less confidence in them.

Limitations of this study include its relatively small sample size and the fact that all patients were served by a single medical practice. No extensive inter‐rater reliability checks were performed, although all reviewers were trained in the definitions of the most important judgment items. Other limitations include possible confounding biases which were not controlled, such as the number of charts reviewed, timing of review, and hospital reviewed (ie, each reviewer did not review the same proportion of charts from each hospital).

SUMMARY

We have presented a retrospective chart review study of hospital readmissions in a community hospital setting. This study adds to the increasing literature describing the factors that contribute to hospital readmissions, how preventable they are, and what strategies may reduce the likelihood of readmission. This study is unique in its contribution to the understanding of hospital readmissions by studying front‐line clinician (hospitalist) perceptions of those factors.

Hospital readmissions have become a focus of national attention as a potential indicator of poor quality and health care waste.13 Geographic variations in readmission rates, a high rate of unplanned readmissions, and the emergence of promising interventions all suggest that some portion of readmissions are preventable.4, 5 This work adds to the work of the Agency for Healthcare Research and Quality (AHRQ) on reports of preventable hospital admissions, using hospitalization rates for ambulatory‐sensitive conditions as prevention quality indicators.6

The actual proportion of preventable readmissions is unknown. In previous research using physician reviewers, estimates have ranged from 5% to 38%.713 More recently, studies using a methodology based on relationships between diagnoses at the initial and subsequent hospitalizations have flagged as many as 76% of 30‐day readmissions as preventable.14

Understanding the preventability of readmissions is important if we are to gauge the true size of this quality and cost opportunity. Moreover, it is important to assess the beliefs of the front‐line clinicians who will be playing key roles in prevention.

The objective of the current study was to examine readmission preventability from the perspective of hospital medicine experts practicing at a community hospital. Through detailed chart review, we identify patient factors and care processes that affect preventability and describe clinicians' ideas for preventing future readmissions.

METHODS

RESULTS

Two hundred thirteen patients (85%) had a single readmission. Another 33 patients had 2 readmissions, and 5 patients accounted for 21 readmissions for a total sample of 300 cases. Table 1 provides characteristics of readmitted patients. They were likely to be elderly; the mean (SD) age was 75.3 (15.3), and more than 48% were 80 or older. Sixty‐six percent of patients were taking more than ten medications, and a quarter (25%) had more than three new medications prescribed at discharge. Frequent diagnoses at the index admission included renal insufficiency, heart failure, dementia, atrial fibrillation, and chronic obstructive pulmonary disease (COPD). The majority of cases had more than one diagnosis identified at their first admission. These diagnoses are what hospitalists believe are significant patient issues rather than the hospital‐coded principal and secondary diagnoses.

Sixty‐four percent readmitted cases had been discharged to home (including those with home services), and 36% were discharged to a care facility (skilled nursing facility [SNF], foster care, assisted living) (Table 2). Fifty‐eight percent of cases were readmitted within seven days of the index admission, and another 29% within the first two weeks. Exactly 75% of the time, the readmission was for the same or related diagnosis as the index admission. Primary care follow‐up did not occur as recommended 69% of the time, and 57% of the time the patient was readmitted prior seeing their primary care physician (PCP).

Overall, only 15% of readmissions were termed preventable by the hospital reviewers, although another 46% were deemed possibly preventable. Preventability ratings varied by reviewer, ranging from a high of 27% to a low of 0% among hospitalists rating ten or more cases (Table 3). There was similar variation in the number of recommended interventions. For readmissions deemed preventable or possibly preventable, the number of potential interventions ranged from more than three per patient to less than one per patient.

The most frequently mentioned intervention that could have prevented a readmission was to extend the hospital stay by one to two days (Table 4). An earlier PCP appointment was suggested for another 21% of readmissions. Other interventions received a scattering of mentions. The types of recommended interventions varied with the rater's perception of preventability (Figure 1, available online). Hospitalists were more likely to recommend a longer initial stay, medication changes, or additional education at discharge, and earlier contact from a care facility, for readmissions they thought were preventable. For possibly preventable readmissions, these same recommendations were important, but hospitalists were also likely to recommend case management, disposition to a higher level of care, or a home health visit.

Table 5 shows the most important characteristics associated with preventability, using a cutoff of 0.2 in statistical significance. Readmissions for the same diagnosis were more likely than others to be rated preventable, as were cases with a short readmission interval, more than three new medications at discharge, and patients with COPD or depression/anxiety. Initial hospital length of stay did not influence preventability, nor did it influence the likelihood of a reviewer recommending a longer initial stay.

Potential predictors associated with preventability were included in a hierarchical logistic regression model, with hospital site and reviewer included as random effects. In this modeling, preventable readmissions were more likely than nonpreventable readmissions to be influenced by three process factors: having the same index and readmission diagnosis; readmission in the first post‐hospital week; being readmitted prior to a primary care follow‐up; and three patient factors: having more than three new discharge medications, having anticoagulation treatment, and having a COPD diagnosis (data available online). Other chronic diseases, age, discharge location, or previous readmissions were not important in the rating of preventability. When entered as random effects in a hierarchical logistic regression model, the categorical variable representing hospital site did not significantly improve prediction (P = 0.42), but the reviewer variable (categorized by the top six reviewers and others) had marginal significance at P = 0.088.

DISCUSSION

Reported high Medicare 30‐day readmission rates and associate excess costs have created a national climate for eliminating unnecessary hospital readmissions.1 Recently passed healthcare legislation in the USA will put in place diagnosis‐related group (DRG) payment reductions for excess readmission rates by 2013. As the definitions and methodologies for determining the relatedness and preventable nature of readmissions continues to be clarified, this study contributes to the understanding of preventability and specific preventative strategies from a physician perspective. Although potential savings in readmission reduction work is attractive, our study indicates that most front‐line clinicians are not convinced that a large portion of readmissions are preventable.

The proportion of preventable readmissions found in our study is very much in line with previous research.713 Certain predictors of preventable readmissions were also similar. Several researchers have found that preventable readmissions are more likely to be early,8, 10, 12 and have the same or related diagnosis as the initial stay.8 On the other hand, our data did not show an independent effect of age on preventability, as others have suggested.9, 17 Patients with a large number of diagnoses and medications have been shown to be at risk for preventable readmissions,9 but the importance of new discharge medications has not been widely researched and is a factor that deserves further exploration.

One key message from our study was found in the variation in the ratings of preventability by individual physicians. At first blush, it may appear to reflect a lack of inter‐rater reliability or understanding of the underlying concept of preventability. We believe this is unlikely, given the discussions among raters and the clear descriptions offered in writing. Moreover, there was much less variation in other judgments such as the ratings of relatedness of the readmission diagnosis (chi‐square = 21.7, P = .041)

There are a number of possible reasons for variation in reviewer ratings of preventability. Reviewers did vary with regard to age, experience, tenure in the organization, gender, and full/part‐time status. They practiced at different hospitals. None of these factors were related to ratings of preventability. On the other hand, three explanations are worth noting.

First, the hierarchical regression models found that reviewer only slightly improved prediction (P = 0.088), above and beyond the other diagnosis and process factors. This would lead us to reject the factor of reviewer as the most important predictor of preventability; the other case characteristics mentioned above were more important.

Second, the three hospitalists who were more optimistic (rated more cases as preventable) reviewed more charts than others. It is possible that these three were more engaged, not only in the chart review process, but more eager to uncover potential remedies to prevent readmissions. While generating more ideas about how to do that, they rated more readmissions as preventable. We do not believe that actually doing more reviews caused them to rate a greater portion as preventable; none of the reviewers showed progression to more preventable ratings over time (analysis not shown).

Finally, it is worth noting that two of the more optimistic physicians had previous primary care experience. This is an intriguing explanation that would benefit from further research. First‐hand experience with primary care case management, rapid appointment follow‐up, home service referrals, and the like may give the practicing hospitalist reason to believe that actions in the ambulatory setting can prevent readmissions.

Regardless of the source, the variation demonstrates cultural or philosophical biases among clinicians regarding how much influence additional planning, education, and care coordination can have on readmissions. We believe that this variation must be addressed in the implementation of readmission reduction programs. Physician engagement will be more likely if there is optimism about the potential to prevent readmissions. In addition, it will be important to develop more consensus about effective interventions from the perspectives of hospital physicians, primary care physicians, nurses, and patients, as others have alluded.18, 19

The significant rate of related readmissions (75%) has implications for the potential Centers for Medicare and Medicaid Services (CMS) methodology that will be used to reduce DRG payments, given the legislation's current intent to exclude only unrelated and planned readmissions from the calculations. Providing clear definitions on relatedness and a methodology to code this criterion in administrative datasets may need to be developed. The views of hospitalists in the current study suggest that the relatedness methodology may be overly sensitive and not yet specific enough to isolate truly preventable readmissions. Less than a quarter of related readmissions were deemed preventable by these raters.

Hospitalists found both patient and process factors important in assessing the preventability of a readmission. This kind of analysis can point to subgroups with potential for targeted intervention. For example, over a third of patients readmitted within a week for the same diagnosis were rated as preventable, indicating a critical follow‐up period for some patients. Higher ratings of preventability among the readmissions for patients on anticoagulation or who were given more than three new medications at discharge indicates that better medication management may indeed be a fruitful strategy for readmission reduction.

The finding that increasing the length of the initial hospital stay was rated as the most prevalent strategy to mitigate against readmission in our retrospective review was surprising. It emphasizes the tension between efficient hospital throughput which reduces unnecessary hospital days and the necessity for appropriate monitoring to ensure clinical stability prior to discharge. Excess hospital days can prolong the exposure to a multitude of hospital acquired conditions (HAC), and this risk must be weighed against a longer length of stay and the time required delivering the appropriate hospital services.

Exploring alternative strategies to reduce readmissions without increasing the hospital length of stay is a reasonable response to this tension. Better discharge education and attention to discharge medications and dosages were also recommended strategies for preventable readmissions. These are interventions hospitalists are familiar with and can control. Relatively smaller percentages of patients were thought to benefit from case management, hospice, home health, or an MD visit to their nursing home, and hospitalists were more likely to recommend these for the possibly preventable patients. These interventions are not fully implemented within the study health system so there is understandably less confidence in them.

Limitations of this study include its relatively small sample size and the fact that all patients were served by a single medical practice. No extensive inter‐rater reliability checks were performed, although all reviewers were trained in the definitions of the most important judgment items. Other limitations include possible confounding biases which were not controlled, such as the number of charts reviewed, timing of review, and hospital reviewed (ie, each reviewer did not review the same proportion of charts from each hospital).

SUMMARY

We have presented a retrospective chart review study of hospital readmissions in a community hospital setting. This study adds to the increasing literature describing the factors that contribute to hospital readmissions, how preventable they are, and what strategies may reduce the likelihood of readmission. This study is unique in its contribution to the understanding of hospital readmissions by studying front‐line clinician (hospitalist) perceptions of those factors.

Hospital readmissions have become a focus of national attention as a potential indicator of poor quality and health care waste.13 Geographic variations in readmission rates, a high rate of unplanned readmissions, and the emergence of promising interventions all suggest that some portion of readmissions are preventable.4, 5 This work adds to the work of the Agency for Healthcare Research and Quality (AHRQ) on reports of preventable hospital admissions, using hospitalization rates for ambulatory‐sensitive conditions as prevention quality indicators.6

The actual proportion of preventable readmissions is unknown. In previous research using physician reviewers, estimates have ranged from 5% to 38%.713 More recently, studies using a methodology based on relationships between diagnoses at the initial and subsequent hospitalizations have flagged as many as 76% of 30‐day readmissions as preventable.14

Understanding the preventability of readmissions is important if we are to gauge the true size of this quality and cost opportunity. Moreover, it is important to assess the beliefs of the front‐line clinicians who will be playing key roles in prevention.

The objective of the current study was to examine readmission preventability from the perspective of hospital medicine experts practicing at a community hospital. Through detailed chart review, we identify patient factors and care processes that affect preventability and describe clinicians' ideas for preventing future readmissions.

METHODS

RESULTS

Two hundred thirteen patients (85%) had a single readmission. Another 33 patients had 2 readmissions, and 5 patients accounted for 21 readmissions for a total sample of 300 cases. Table 1 provides characteristics of readmitted patients. They were likely to be elderly; the mean (SD) age was 75.3 (15.3), and more than 48% were 80 or older. Sixty‐six percent of patients were taking more than ten medications, and a quarter (25%) had more than three new medications prescribed at discharge. Frequent diagnoses at the index admission included renal insufficiency, heart failure, dementia, atrial fibrillation, and chronic obstructive pulmonary disease (COPD). The majority of cases had more than one diagnosis identified at their first admission. These diagnoses are what hospitalists believe are significant patient issues rather than the hospital‐coded principal and secondary diagnoses.

Sixty‐four percent readmitted cases had been discharged to home (including those with home services), and 36% were discharged to a care facility (skilled nursing facility [SNF], foster care, assisted living) (Table 2). Fifty‐eight percent of cases were readmitted within seven days of the index admission, and another 29% within the first two weeks. Exactly 75% of the time, the readmission was for the same or related diagnosis as the index admission. Primary care follow‐up did not occur as recommended 69% of the time, and 57% of the time the patient was readmitted prior seeing their primary care physician (PCP).

Overall, only 15% of readmissions were termed preventable by the hospital reviewers, although another 46% were deemed possibly preventable. Preventability ratings varied by reviewer, ranging from a high of 27% to a low of 0% among hospitalists rating ten or more cases (Table 3). There was similar variation in the number of recommended interventions. For readmissions deemed preventable or possibly preventable, the number of potential interventions ranged from more than three per patient to less than one per patient.

The most frequently mentioned intervention that could have prevented a readmission was to extend the hospital stay by one to two days (Table 4). An earlier PCP appointment was suggested for another 21% of readmissions. Other interventions received a scattering of mentions. The types of recommended interventions varied with the rater's perception of preventability (Figure 1, available online). Hospitalists were more likely to recommend a longer initial stay, medication changes, or additional education at discharge, and earlier contact from a care facility, for readmissions they thought were preventable. For possibly preventable readmissions, these same recommendations were important, but hospitalists were also likely to recommend case management, disposition to a higher level of care, or a home health visit.

Table 5 shows the most important characteristics associated with preventability, using a cutoff of 0.2 in statistical significance. Readmissions for the same diagnosis were more likely than others to be rated preventable, as were cases with a short readmission interval, more than three new medications at discharge, and patients with COPD or depression/anxiety. Initial hospital length of stay did not influence preventability, nor did it influence the likelihood of a reviewer recommending a longer initial stay.

Potential predictors associated with preventability were included in a hierarchical logistic regression model, with hospital site and reviewer included as random effects. In this modeling, preventable readmissions were more likely than nonpreventable readmissions to be influenced by three process factors: having the same index and readmission diagnosis; readmission in the first post‐hospital week; being readmitted prior to a primary care follow‐up; and three patient factors: having more than three new discharge medications, having anticoagulation treatment, and having a COPD diagnosis (data available online). Other chronic diseases, age, discharge location, or previous readmissions were not important in the rating of preventability. When entered as random effects in a hierarchical logistic regression model, the categorical variable representing hospital site did not significantly improve prediction (P = 0.42), but the reviewer variable (categorized by the top six reviewers and others) had marginal significance at P = 0.088.

DISCUSSION

Reported high Medicare 30‐day readmission rates and associate excess costs have created a national climate for eliminating unnecessary hospital readmissions.1 Recently passed healthcare legislation in the USA will put in place diagnosis‐related group (DRG) payment reductions for excess readmission rates by 2013. As the definitions and methodologies for determining the relatedness and preventable nature of readmissions continues to be clarified, this study contributes to the understanding of preventability and specific preventative strategies from a physician perspective. Although potential savings in readmission reduction work is attractive, our study indicates that most front‐line clinicians are not convinced that a large portion of readmissions are preventable.

The proportion of preventable readmissions found in our study is very much in line with previous research.713 Certain predictors of preventable readmissions were also similar. Several researchers have found that preventable readmissions are more likely to be early,8, 10, 12 and have the same or related diagnosis as the initial stay.8 On the other hand, our data did not show an independent effect of age on preventability, as others have suggested.9, 17 Patients with a large number of diagnoses and medications have been shown to be at risk for preventable readmissions,9 but the importance of new discharge medications has not been widely researched and is a factor that deserves further exploration.

One key message from our study was found in the variation in the ratings of preventability by individual physicians. At first blush, it may appear to reflect a lack of inter‐rater reliability or understanding of the underlying concept of preventability. We believe this is unlikely, given the discussions among raters and the clear descriptions offered in writing. Moreover, there was much less variation in other judgments such as the ratings of relatedness of the readmission diagnosis (chi‐square = 21.7, P = .041)

There are a number of possible reasons for variation in reviewer ratings of preventability. Reviewers did vary with regard to age, experience, tenure in the organization, gender, and full/part‐time status. They practiced at different hospitals. None of these factors were related to ratings of preventability. On the other hand, three explanations are worth noting.

First, the hierarchical regression models found that reviewer only slightly improved prediction (P = 0.088), above and beyond the other diagnosis and process factors. This would lead us to reject the factor of reviewer as the most important predictor of preventability; the other case characteristics mentioned above were more important.

Second, the three hospitalists who were more optimistic (rated more cases as preventable) reviewed more charts than others. It is possible that these three were more engaged, not only in the chart review process, but more eager to uncover potential remedies to prevent readmissions. While generating more ideas about how to do that, they rated more readmissions as preventable. We do not believe that actually doing more reviews caused them to rate a greater portion as preventable; none of the reviewers showed progression to more preventable ratings over time (analysis not shown).

Finally, it is worth noting that two of the more optimistic physicians had previous primary care experience. This is an intriguing explanation that would benefit from further research. First‐hand experience with primary care case management, rapid appointment follow‐up, home service referrals, and the like may give the practicing hospitalist reason to believe that actions in the ambulatory setting can prevent readmissions.

Regardless of the source, the variation demonstrates cultural or philosophical biases among clinicians regarding how much influence additional planning, education, and care coordination can have on readmissions. We believe that this variation must be addressed in the implementation of readmission reduction programs. Physician engagement will be more likely if there is optimism about the potential to prevent readmissions. In addition, it will be important to develop more consensus about effective interventions from the perspectives of hospital physicians, primary care physicians, nurses, and patients, as others have alluded.18, 19

The significant rate of related readmissions (75%) has implications for the potential Centers for Medicare and Medicaid Services (CMS) methodology that will be used to reduce DRG payments, given the legislation's current intent to exclude only unrelated and planned readmissions from the calculations. Providing clear definitions on relatedness and a methodology to code this criterion in administrative datasets may need to be developed. The views of hospitalists in the current study suggest that the relatedness methodology may be overly sensitive and not yet specific enough to isolate truly preventable readmissions. Less than a quarter of related readmissions were deemed preventable by these raters.

Hospitalists found both patient and process factors important in assessing the preventability of a readmission. This kind of analysis can point to subgroups with potential for targeted intervention. For example, over a third of patients readmitted within a week for the same diagnosis were rated as preventable, indicating a critical follow‐up period for some patients. Higher ratings of preventability among the readmissions for patients on anticoagulation or who were given more than three new medications at discharge indicates that better medication management may indeed be a fruitful strategy for readmission reduction.

The finding that increasing the length of the initial hospital stay was rated as the most prevalent strategy to mitigate against readmission in our retrospective review was surprising. It emphasizes the tension between efficient hospital throughput which reduces unnecessary hospital days and the necessity for appropriate monitoring to ensure clinical stability prior to discharge. Excess hospital days can prolong the exposure to a multitude of hospital acquired conditions (HAC), and this risk must be weighed against a longer length of stay and the time required delivering the appropriate hospital services.

Exploring alternative strategies to reduce readmissions without increasing the hospital length of stay is a reasonable response to this tension. Better discharge education and attention to discharge medications and dosages were also recommended strategies for preventable readmissions. These are interventions hospitalists are familiar with and can control. Relatively smaller percentages of patients were thought to benefit from case management, hospice, home health, or an MD visit to their nursing home, and hospitalists were more likely to recommend these for the possibly preventable patients. These interventions are not fully implemented within the study health system so there is understandably less confidence in them.

Limitations of this study include its relatively small sample size and the fact that all patients were served by a single medical practice. No extensive inter‐rater reliability checks were performed, although all reviewers were trained in the definitions of the most important judgment items. Other limitations include possible confounding biases which were not controlled, such as the number of charts reviewed, timing of review, and hospital reviewed (ie, each reviewer did not review the same proportion of charts from each hospital).

SUMMARY

We have presented a retrospective chart review study of hospital readmissions in a community hospital setting. This study adds to the increasing literature describing the factors that contribute to hospital readmissions, how preventable they are, and what strategies may reduce the likelihood of readmission. This study is unique in its contribution to the understanding of hospital readmissions by studying front‐line clinician (hospitalist) perceptions of those factors.

If you wish to receive credit for this activity, please refer to the website: www.wileyblackwellcme.com.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this journal‐based CME activity for a maximum of 1 AMA PRA Category 1 Credit.. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

The objectives need to be changed. Please remove the existing ones, and include these two:

• To describe the correlation between inpatient and outpatient blood pressure measurements.

• To assess the potential benefits of prescribing antihypertensive medication in hospitalized patients with hypertension.

This manuscript underwent peer review in line with the standards of editorial integrity and publication ethics maintained by Journal of Hospital Medicine. The peer reviewers have no relevant financial relationships. The peer review process for Journal of Hospital Medicine is single‐blinded. As such, the identities of the reviewers are not disclosed in line with the standard accepted practices of medical journal peer review.

Conflicts of interest have been identified and resolved in accordance with Blackwell Futura Media Services's Policy on Activity Disclosure and Conflict of Interest. The primary resolution method used was peer review and review by a non‐conflicted expert.

Instructions on Receiving Credit

For information on applicability and acceptance of CME credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within an hour; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period, which is up to two years from initial publication.

Follow these steps to earn credit:

• Log on to www.wileyblackwellcme.com

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

This activity will be available for CME credit for twelve months following its publication date. At that time, it will be reviewed and potentially updated and extended for an additional twelve months.

If you wish to receive credit for this activity, please refer to the website: www.wileyblackwellcme.com.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this journal‐based CME activity for a maximum of 1 AMA PRA Category 1 Credit.. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

The objectives need to be changed. Please remove the existing ones, and include these two:

• To describe the correlation between inpatient and outpatient blood pressure measurements.

• To assess the potential benefits of prescribing antihypertensive medication in hospitalized patients with hypertension.

This manuscript underwent peer review in line with the standards of editorial integrity and publication ethics maintained by Journal of Hospital Medicine. The peer reviewers have no relevant financial relationships. The peer review process for Journal of Hospital Medicine is single‐blinded. As such, the identities of the reviewers are not disclosed in line with the standard accepted practices of medical journal peer review.

Conflicts of interest have been identified and resolved in accordance with Blackwell Futura Media Services's Policy on Activity Disclosure and Conflict of Interest. The primary resolution method used was peer review and review by a non‐conflicted expert.

Instructions on Receiving Credit

For information on applicability and acceptance of CME credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within an hour; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period, which is up to two years from initial publication.

Follow these steps to earn credit:

• Log on to www.wileyblackwellcme.com

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

This activity will be available for CME credit for twelve months following its publication date. At that time, it will be reviewed and potentially updated and extended for an additional twelve months.

If you wish to receive credit for this activity, please refer to the website: www.wileyblackwellcme.com.

Accreditation and Designation Statement

Blackwell Futura Media Services designates this journal‐based CME activity for a maximum of 1 AMA PRA Category 1 Credit.. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Blackwell Futura Media Services is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Educational Objectives

The objectives need to be changed. Please remove the existing ones, and include these two:

• To describe the correlation between inpatient and outpatient blood pressure measurements.

• To assess the potential benefits of prescribing antihypertensive medication in hospitalized patients with hypertension.

This manuscript underwent peer review in line with the standards of editorial integrity and publication ethics maintained by Journal of Hospital Medicine. The peer reviewers have no relevant financial relationships. The peer review process for Journal of Hospital Medicine is single‐blinded. As such, the identities of the reviewers are not disclosed in line with the standard accepted practices of medical journal peer review.

Conflicts of interest have been identified and resolved in accordance with Blackwell Futura Media Services's Policy on Activity Disclosure and Conflict of Interest. The primary resolution method used was peer review and review by a non‐conflicted expert.

Instructions on Receiving Credit

For information on applicability and acceptance of CME credit for this activity, please consult your professional licensing board.

This activity is designed to be completed within an hour; physicians should claim only those credits that reflect the time actually spent in the activity. To successfully earn credit, participants must complete the activity during the valid credit period, which is up to two years from initial publication.

Follow these steps to earn credit:

• Log on to www.wileyblackwellcme.com

• Read the target audience, learning objectives, and author disclosures.

• Read the article in print or online format.

• Reflect on the article.

• Access the CME Exam, and choose the best answer to each question.

• Complete the required evaluation component of the activity.

This activity will be available for CME credit for twelve months following its publication date. At that time, it will be reviewed and potentially updated and extended for an additional twelve months.

Providing family centered care has been identified as a goal in the Institute of Medicine's report Crossing the Quality Chiasm1 and endorsed by the American Academy of Pediatrics.2 Traditionally, rounds are the central organizing structure for clinical work, decision making, and teaching in the inpatient setting. Patient care and educational goals emanate from rounds. Over the past several decades rounds have migrated from the patient's bedside to the privacy of the conference room. In our experience, although conference room rounds offer some advantages, patients and families are not privy to the data or decision‐making process used to determine their diagnosis and plan of care. The ritual that frequently occurs after conference‐room rounds is that the team members (medical students, residents, nurses, attending) visit the patient and family independently throughout the course of the day, communicating their understanding of medical and affective issues in a manner that families often view as providing confusing, if not contradictory information.

Conducting rounds entirely at the bedside can bypass this systemic flaw, allowing parents and patients to correct inaccurate data, and enable them to make their values and concerns known to the team. This model can help to connect the caregivers and receivers of care, and represents a collaborative communication process, the foundation for effective family‐centered rounds (FCR). When team members discuss how they interpret clinical data in the presence of the family it helps them to understand how and why a management plan is conceived. The care team develops an alliance of trust with the family through this transparent communication and joint decision‐making.

Despite the potential for enhancing patient/family satisfaction and endorsements by public and professional organizations, in a recent study less than half of pediatric hospitalists reported conducting FCR.3 Trainees and attending physicians raised concerns about the potential for FCR to waste time and diminish teaching.4 Trainees' perceptions of the educational value of FCR has not been well studied, but a recent qualitative study of pediatric residents reported that if conducted well, FCRs enhance education and clinical skills by increasing the number of patients seen by each team member, and by offering opportunities to improve physical examination skills. Trainees appreciated role‐modeling and realtime feedback by attending physicians. Senior residents reported enhanced leadership and teaching opportunities.5

The aim of this study was to design and implement a faculty development program to address the need of our junior hospitalist faculty members to enhance teaching during FCR.

Methods

We determined, based upon direct observation, a focus group and survey feedback from our pediatric residents, that for inpatient teaching during FCR to be successful, our faculty needed training in the following areas: orienting learners, providing feedback, teaching assessment of key physical exam findings, correcting errors in clinical reasoning, and promoting the role of the senior resident as team leader. We developed the Observed Structured Teaching Exercises (OSTE)6 and related workshops to promote key behaviors identified from the literature for each of the areas.

All of the Children's National Medical Center (CNMC) Pediatric hospitalists (N = 14) who were not investigators in the study were asked to participate. They were informed of the study design and the overall goal of making inpatient rounds more effective and efficient through better teaching skills. The study was approved by the CNMC institutional review board and was conducted from August to September 2007 in the CLASS (Clinical Learning and Simulation Skills) at The George Washington University School of Medicine and Health Sciences.

To assess faculty and fellow baseline knowledge and skills, the authors conducted a preintervention OSTE consisting of 4 stations: 1) physical exam interpretation and promoting PL‐3 autonomy (Established Patient), 2) stimulating clinical reasoning (New Patient), 3) feedback, and 4) facilitating an orientation. This exercise was followed within 2 weeks by four 90‐minute interactive workshops that focused on the topic areas as evaluated in the OSTEs. Each workshop consisted of a brief evidence‐based didactic component, interactive discussion, and skill building exercises to practice desired teaching behaviors. Two weeks following the workshops, the group participated in postintervention OSTEs similar to the preintervention scenarios, with minor changes, such as presenting diagnoses, to avoid pattern recognition.

Results

All 14 eligible hospitalists voluntarily participated. Their mean year postcompletion of residency training for the faculty was 17 months 14 months; 71% were female. None of the participants experienced previous training in the areas proposed in the study.

Participants assigned high scores to the quality of the workshops, the OSTE experience and their learning from the participating in the faculty development exercise. The differences between pre‐ and post‐OSTE scenario as well as overall scores for the 4 stations were statistically significant (P < .0001). Particular improvements were noted in the correction of incorrect new admission diagnoses (56% pre, 86% post) and orientation (65% pre, 95% post; see Table 2).

Discussion

If FCR are to be universally adopted in the academic pediatric inpatient setting, faculty must successfully balance the educational needs of trainees as well as efficiently negotiating a plan of management with patients and families. The ability of faculty to consistently orchestrate rounds so that they meet educational needs of varied levels of learners, while ensuring that patient management is correct and well communicated to families is a very complex task.

We found using OSTEs to frame desired behavior, supplemented with background information to validate the desired behaviors followed by deliberate practice opportunities during the workshops to be an effective faculty development strategy. We not only provided participants with feedback on the group's performance according to the rating scale, but also gave them the opportunity to practice rating each other using the scale so that they could reflect on the elements of their performance that merited a specific rating.

This strategy for training faculty to perform well in the complex environment within the patient's room during FCR is similar in some respects with training military personnel for complex battle situations.15 Desired behaviors are broken down and packaged within a framework to be implemented in a specific context. For example, we combined aspects of the One Minute Preceptor model (OMP)12 with Bordage's Problem Representation model to create a framework of behaviors to promote and correct errors in clinical reasoning.16, 17 Another framework was created and practiced to promote assessment of the physical exam at the bedside. Orientation and feedback, although not frequently used components of actual FCR, are necessary to set expectations and calibrate learner's performance during FCR.

The OSTE is an observed examination that has been validated for evaluating the teaching skills of faculty and residents.18 We planned to use learner‐centered, interactive workshops as the key component of the training intervention with the pre‐ and posttest OSTE as a measure of their effectiveness. However, we found in faculty feedback that the OSTEs were actually a key adjuvant, to the workshop training in that they provided a major source of feedback and learning opportunities in addition to their inherent evaluative qualities.

Each of 4 workshops was designed to teach participants the behaviors assessed in the 4 OSTE stations. The pretest OSTE provided a baseline for participant performance and served to activate the participants to focus on key teaching behaviors during the workshop. During the workshop following the pretest OSTE, participants were given copies of the rating scales and feedback on the performance of the group as a whole on each rated behavior. The evidence used to create the rating instruments was presented and participants had the opportunity to debate and agree on the instrument's construct validity. They then had the opportunity to engage in deliberate practice during role plays depicting challenges to orienting a learner, providing feedback, and to family centered rounds. The posttest OSTE served as summative evaluation of the participants' ability to perform the practiced behaviors effectively in a simulated teaching environment.

We chose to focus the FCR scenarios on correcting mistakes in clinical reasoning for a new patient and on teaching key parts of the physical exam during rounds for an established patient. Errors in clinical reasoning lead to misdirected patient management and are the number 1 cause of medical errors.19, 20 Bedside rounds are a perfect venue for reinforcing and fine‐tuning diagnostic reasoning because all the crucial sources of data are present: the patient, the parent, the nurse, and the computer with lab and imaging results. Faculty members and trainees have both expressed discomfort at correcting errors in clinical reasoning in front of families, leading to missed learning opportunities.21

During the workshop on clinical reasoning, we taught faculty how to use the Problem Representation method to analyze and correct errors in clinical reasoning. The method, studied by Bordage and associates22 forces learners to identify the key features of a presentation and relate their interpretation of the findings by using semantic qualifiers. We trained faculty to deliberately listen for the learner's interpretation of the key features to determine how a misdiagnosis occurred. They were also trained to walk trainees back through their thought process in an objective way, correcting the misinterpretation of data, so that the trainee's competence is not compromised in the eyes of the team or the parents. Teaching the trainee to think correctly about a clinical problem benefits the other members of the team, as well as providing the parents with a better understanding of the rationale for the management plan.

Correct interpretation of the physical exam findings is crucial to making the correct diagnosis. However, there have been several articles chronicling the lost art of eliciting and interpreting physical exam findings, ranging from the cardiac exam to neurological exam.23 A minority of physical exam teaching occurs at the patient's bedside, partly attributed to faculty members discomfort with this type of teaching.24, 25 To enhance the comfort of our faculty members, we included behaviors referenced in articles on teaching a skill,26 activated demonstration,10 and effective bedside teaching27 to guide faculty to incorporate eliciting and interpreting focused aspects of the physical exam during rounds.

Our FCR evaluation templates awarded the highest scores if the hospitalist encouraged senior residents to model clinical reasoning or physical exam skills for junior learners. Hospitalists' presence, especially on work rounds, can diminish the senior resident's opportunity to gain experience and confidence in leading the team.28 We therefore explicitly directed hospitalists to promote the role of the senior resident as the team leader during workshops, while priming faculty to assume the role of educational coach.29

We hypothesize that several factors contributed to the success of the OSTE workshopOSTE intervention. First, faculty members willingly volunteered to participate because they recognized gaps in their own knowledge and skill at leading FCR. They found the ability to deliberately practice the desired behaviors in the OSTE exercises to be the most useful part of the exercise, because the scenarios were authentic and SLs were real trainees.

Although we included all the junior faculty members of our large Pediatric Hospitalist Division in our study, our sample size is still small; limiting our ability to generalize our findings. We found the scheduling of 14 hospitalists to attend 4 different events in close succession to be problematic. Conducting the OSTE sessions at the GW CLASS center 5 miles away from our hospital was also logistically challenging.

We plan to simplify the logistics so that we can incorporate this model in the training of new hospitalists in the division. We still plan to use preintervention FCR OSTEs, but instead of workshops, will provide background information by means of self‐directed Web‐based modules. We will also videotape the OSTEs and provide faculty with a template to rate their own performance and then compare it with ratings from SLs. This individualized feedback and self‐reflection could result in better performance30 than the summary group feedback we gave during the workshops.

Another limitation of this study is the lack of data regarding the consistency of our faculty participants' performance in real FCR. Finally, we did not study the impact of the desired behaviors on patient, trainee, or nursing satisfaction, learning, or efficiency.

Conclusion

In conclusion, we found incorporating OSTEs into a faculty development program to improve FCR to be an effective strategy for changing faculty behavior in leading FCR. Additional study is needed to determine if replacing the workshops with Web‐based tutorials is equally effective and to determine if this faculty development strategy results in long‐term consistent practice in conducting rounds in real inpatient settings.