The transition from hospital to home is a complex event offering multiple provider‐identified opportunities to improve healthcare quality.18 Centering care delivery around patient needs and preferences is both inherently valuable and linked with better outcomes.9

The Care Transitions Measure (CTM) identifies 4 domains of patient experience related to hospital discharge: information transfer, patient and caregiver preparation, self‐management support, and empowerment to assert preferences.10 It discriminates between patients who do or do not experience a subsequent readmission or emergency room visit and between levels of care coordination.11 Quality indicators like the CTM are important tools for systematic healthcare improvement, but they provide a limited understanding of patient experiences, which can drive the transformation of systems.12, 13

With the exception of patients with a few specific clinical conditions, relatively little is known about how adult patients perceive the hospital‐to‐home transition.1417 They recall receiving discharge instructions but lack details about what to do if problems arise.18 They may lack important information despite receiving instruction.19 Caregivers report problems related to emotional support, discharge planning, and family participation,20 and patients and caregivers express anxiety, confusion, a sense of abandonment by the healthcare system, and the perception that their preferences are disregarded.21

As part of ongoing quality improvement activities, we sought to develop a richly detailed, patient‐centered view of the hospital‐to‐home transition. Our purpose was to understand patient and caregiver experiences during this pivotal healthcare experience.

METHODS

We used an applied ethnographic approach,22 conducting participant observation and video recording in‐depth, semi‐structured interviews in Kaiser Permanente Southern California, Colorado, and Hawaii. The United States' largest, private, not‐for‐profit integrated healthcare delivery system, Kaiser Permanente addresses all health needs for more than 8.9 million members.

To balance the pragmatic imperatives of quality improvement with obtaining enough information to understand patient experiences, we planned a sample of 24 patients across 3 settings with a mix of resource‐intensive and less‐intensive healthcare needs. We defined resource‐intensive needs as occurring among patients aged 65 or older with 3 or more chronic conditions. We asked hospital staff to identify patients by level of need and variety in diagnoses and illness severity, planned or unplanned hospitalizations, age, and ability to self manage. Reasons for admission included joint replacement, acute appendicitis, chronic illness exacerbation, complications of cancer chemotherapy, and others. We included patients who were inpatients or discharged no more than 3 weeks before interview. We excluded those under the age of 18 or discharged to non‐home settings. The project took place between September and November of 2008; 24 patients, half of whom were male, gave written informed consent for video recordings and authorization to distribute protected health information throughout and beyond Kaiser Permanente for quality improvement and educational purposes. Participants took part in interviews and observations lasting 1 to 3 hours; caregivers and family members participated in 9 instances.

Two or 3 observers attended each interview, which took place in the hospital on discharge day, at postdischarge appointments, or in patients' homes. Open‐ended questions prompted broad‐ranging inquiry into patients' lives, medical history, hospitalization experience, medications, care network, challenges, personal goals, and inner experience. Some questions were adapted and expanded from the CTM; others were prompts to demonstrate activities (eg, Can you show us how you organize your medications?). In addition to interviewing patients and caregivers, we observed interactions between patients, families, and hospital staff before discharge. We also observed patients and caregivers at home and when interacting with outpatient primary care providers. The purpose of observation was to understand the context of patient and caregiver experiences and to identify consistencies or discrepancies with their descriptions of experiences. (see Supporting Information In‐Home Interview Guide in the online version of this article)

Data included field notes and video recordings. In addition, observers summarized their strongest daily impressions as brief team stories that were shared with the observation team, local operations staff, and Kaiser Permanente national subject matter experts.23 Consistent with a grounded theory approach, interviews were professionally transcribed and qualitatively analyzed by multiple observers in iterative stages to develop broad domains of patient experiences.24 We clustered similar experiences and identified exemplar statements and behaviors. Team stories were analyzed separately, using a similar process. We reviewed recorded interviews to refine our emerging understanding of patient and caregiver experiences and discussed our observations and impressions about each domain. To maximize internal validity, an independent researcher who did not attend the interviews reviewed the transcripts and coding and participated in final qualitative analysis. Institutional review board approval was not required for this quality improvement project.

RESULTS

Patients and caregivers expressed or demonstrated 6 domains of experience as they transitioned from hospital to home (Table 1).

DISCUSSION

Using an ethnographic approach, we identified 6 domains of patient and caregiver experience during the hospital‐to‐home transition. Many needs in these domains arose in the hours and days after patients returned home, and patients and caregivers often found it challenging to meet them. Our project adds a detailed, patient‐centered perspective on the transition from hospital to home.

The domains we identified share some conceptual territory with the dimensions of the Care Transition Measure and the Transitional Care Model,25 but generate a more detailed understanding of patient and caregiver experiences. Key findings include the fact that patients can find it challenging to translate knowledge into contextually appropriate action at home. This confirms some published results. For instance, estimates of outpatient adherence to complicated regimens range from 5% to 77%.2629 Significant opportunities exist to improve the reliability of translating medication instructions into systems that work at home,30 including aligning medication lists with physical aids (such as weekly pill boxes) and explaining medications in patient‐friendly terms. We also found that same‐day discharge teaching can be ineffective because patients are anxious to leave the hospital or staff members feel rushed. Emotion can interfere with cognition, and transferring information shortly before hospital discharge may overlook learning readiness, a fundamental principle of patient education.31, 32 In addition, the desire to return to normal life, coupled with uncertainty about who to call for clarification, can lead patients to simply do the best they can with whatever information they recall.

The literature refers to handoffs of patients from one provider to another as an episode of care is completed, but our findings suggest patients perceive hospitalization as an event occurring within ongoing relationships with the healthcare providers to whom they feel most connected.33, 34 Some patient and caregiver needs could be addressed by actively supporting these relationships during the hospital‐to‐home transition: explicitly acknowledging their importance to patients, ensuring that providers have discharge information, and framing discharge as a transition back to the care of trusted providers. Some of our findings require system‐level changes. Patients and caregivers with unmet transportation needs expressed anxiety about how or if help would materialize. Partnerships with community organizations could enable healthcare organizations to address needs like transportation that fall outside traditional discharge activities but significantly impact patient experiences. In addition, healthcare organizations are rarely designed for straightforward navigation; patient‐centered organizational designs could eliminate the need for patients and caregivers to learn how to navigate. For instance, a single point of contact for recently discharged patients might improve the process of finding help.

Strengths of our quality improvement project include the range of patients we interviewed and in‐depth observations and interviews across settings. Ethnography is ideal for generating a rich understanding of patient experiences, allowing us to observe needs patients did not mention, as well as the physical and emotional context of the transition. Weaknesses of our approach include the fact that the experiences reflected in each category were determined, to some extent, by the questions we asked. This may have constrained the variety of experiences patients reported. In addition, Kaiser Permanente's integrated nature may have affected our findings, although we believe patients and caregivers reported experiences that are likely universal.

Our project occurred in a healthcare system with an integrated electronic health record (EHR). Interventions to improve provider‐identified gaps in the discharge process often rely on information technology.3543 However, information technology does not eliminate continuity of care issues.44 Our EHR is widely used, but available information did not consistently ensure strong enough care coordination or good communication.

Including the patient's primary caregiver in discharge teaching appeared to be a relatively low priority for hospital staff, unless there was a hands‐on care task. Even when a primary caregiver was present, hospital staff frequently directed teaching exclusively toward the patient. The extent to which caregivers feel adequately prepared for their roles and responsibilities needs further exploration.

CONCLUSION

Our applied ethnographic approach reveals that patients experience several challenges while transitioning from hospital to home. Reducing readmissions is likely to remain challenging unless we broaden our understanding of the types of support and coaching required. We are translating our findings into quality improvement activities, conducting pilot projects focusing on risk stratification and tailoring of care, a specialized phone number for recently discharged patients, standardized same‐day discharge summaries to primary care providers, medication reconciliation, follow‐up phone calls, and scheduling appointments before discharge.

The transition from hospital to home is a complex event offering multiple provider‐identified opportunities to improve healthcare quality.18 Centering care delivery around patient needs and preferences is both inherently valuable and linked with better outcomes.9

The Care Transitions Measure (CTM) identifies 4 domains of patient experience related to hospital discharge: information transfer, patient and caregiver preparation, self‐management support, and empowerment to assert preferences.10 It discriminates between patients who do or do not experience a subsequent readmission or emergency room visit and between levels of care coordination.11 Quality indicators like the CTM are important tools for systematic healthcare improvement, but they provide a limited understanding of patient experiences, which can drive the transformation of systems.12, 13

With the exception of patients with a few specific clinical conditions, relatively little is known about how adult patients perceive the hospital‐to‐home transition.1417 They recall receiving discharge instructions but lack details about what to do if problems arise.18 They may lack important information despite receiving instruction.19 Caregivers report problems related to emotional support, discharge planning, and family participation,20 and patients and caregivers express anxiety, confusion, a sense of abandonment by the healthcare system, and the perception that their preferences are disregarded.21

As part of ongoing quality improvement activities, we sought to develop a richly detailed, patient‐centered view of the hospital‐to‐home transition. Our purpose was to understand patient and caregiver experiences during this pivotal healthcare experience.

METHODS

We used an applied ethnographic approach,22 conducting participant observation and video recording in‐depth, semi‐structured interviews in Kaiser Permanente Southern California, Colorado, and Hawaii. The United States' largest, private, not‐for‐profit integrated healthcare delivery system, Kaiser Permanente addresses all health needs for more than 8.9 million members.

To balance the pragmatic imperatives of quality improvement with obtaining enough information to understand patient experiences, we planned a sample of 24 patients across 3 settings with a mix of resource‐intensive and less‐intensive healthcare needs. We defined resource‐intensive needs as occurring among patients aged 65 or older with 3 or more chronic conditions. We asked hospital staff to identify patients by level of need and variety in diagnoses and illness severity, planned or unplanned hospitalizations, age, and ability to self manage. Reasons for admission included joint replacement, acute appendicitis, chronic illness exacerbation, complications of cancer chemotherapy, and others. We included patients who were inpatients or discharged no more than 3 weeks before interview. We excluded those under the age of 18 or discharged to non‐home settings. The project took place between September and November of 2008; 24 patients, half of whom were male, gave written informed consent for video recordings and authorization to distribute protected health information throughout and beyond Kaiser Permanente for quality improvement and educational purposes. Participants took part in interviews and observations lasting 1 to 3 hours; caregivers and family members participated in 9 instances.

Two or 3 observers attended each interview, which took place in the hospital on discharge day, at postdischarge appointments, or in patients' homes. Open‐ended questions prompted broad‐ranging inquiry into patients' lives, medical history, hospitalization experience, medications, care network, challenges, personal goals, and inner experience. Some questions were adapted and expanded from the CTM; others were prompts to demonstrate activities (eg, Can you show us how you organize your medications?). In addition to interviewing patients and caregivers, we observed interactions between patients, families, and hospital staff before discharge. We also observed patients and caregivers at home and when interacting with outpatient primary care providers. The purpose of observation was to understand the context of patient and caregiver experiences and to identify consistencies or discrepancies with their descriptions of experiences. (see Supporting Information In‐Home Interview Guide in the online version of this article)

Data included field notes and video recordings. In addition, observers summarized their strongest daily impressions as brief team stories that were shared with the observation team, local operations staff, and Kaiser Permanente national subject matter experts.23 Consistent with a grounded theory approach, interviews were professionally transcribed and qualitatively analyzed by multiple observers in iterative stages to develop broad domains of patient experiences.24 We clustered similar experiences and identified exemplar statements and behaviors. Team stories were analyzed separately, using a similar process. We reviewed recorded interviews to refine our emerging understanding of patient and caregiver experiences and discussed our observations and impressions about each domain. To maximize internal validity, an independent researcher who did not attend the interviews reviewed the transcripts and coding and participated in final qualitative analysis. Institutional review board approval was not required for this quality improvement project.

RESULTS

Patients and caregivers expressed or demonstrated 6 domains of experience as they transitioned from hospital to home (Table 1).

DISCUSSION

Using an ethnographic approach, we identified 6 domains of patient and caregiver experience during the hospital‐to‐home transition. Many needs in these domains arose in the hours and days after patients returned home, and patients and caregivers often found it challenging to meet them. Our project adds a detailed, patient‐centered perspective on the transition from hospital to home.

The domains we identified share some conceptual territory with the dimensions of the Care Transition Measure and the Transitional Care Model,25 but generate a more detailed understanding of patient and caregiver experiences. Key findings include the fact that patients can find it challenging to translate knowledge into contextually appropriate action at home. This confirms some published results. For instance, estimates of outpatient adherence to complicated regimens range from 5% to 77%.2629 Significant opportunities exist to improve the reliability of translating medication instructions into systems that work at home,30 including aligning medication lists with physical aids (such as weekly pill boxes) and explaining medications in patient‐friendly terms. We also found that same‐day discharge teaching can be ineffective because patients are anxious to leave the hospital or staff members feel rushed. Emotion can interfere with cognition, and transferring information shortly before hospital discharge may overlook learning readiness, a fundamental principle of patient education.31, 32 In addition, the desire to return to normal life, coupled with uncertainty about who to call for clarification, can lead patients to simply do the best they can with whatever information they recall.

The literature refers to handoffs of patients from one provider to another as an episode of care is completed, but our findings suggest patients perceive hospitalization as an event occurring within ongoing relationships with the healthcare providers to whom they feel most connected.33, 34 Some patient and caregiver needs could be addressed by actively supporting these relationships during the hospital‐to‐home transition: explicitly acknowledging their importance to patients, ensuring that providers have discharge information, and framing discharge as a transition back to the care of trusted providers. Some of our findings require system‐level changes. Patients and caregivers with unmet transportation needs expressed anxiety about how or if help would materialize. Partnerships with community organizations could enable healthcare organizations to address needs like transportation that fall outside traditional discharge activities but significantly impact patient experiences. In addition, healthcare organizations are rarely designed for straightforward navigation; patient‐centered organizational designs could eliminate the need for patients and caregivers to learn how to navigate. For instance, a single point of contact for recently discharged patients might improve the process of finding help.

Strengths of our quality improvement project include the range of patients we interviewed and in‐depth observations and interviews across settings. Ethnography is ideal for generating a rich understanding of patient experiences, allowing us to observe needs patients did not mention, as well as the physical and emotional context of the transition. Weaknesses of our approach include the fact that the experiences reflected in each category were determined, to some extent, by the questions we asked. This may have constrained the variety of experiences patients reported. In addition, Kaiser Permanente's integrated nature may have affected our findings, although we believe patients and caregivers reported experiences that are likely universal.

Our project occurred in a healthcare system with an integrated electronic health record (EHR). Interventions to improve provider‐identified gaps in the discharge process often rely on information technology.3543 However, information technology does not eliminate continuity of care issues.44 Our EHR is widely used, but available information did not consistently ensure strong enough care coordination or good communication.

Including the patient's primary caregiver in discharge teaching appeared to be a relatively low priority for hospital staff, unless there was a hands‐on care task. Even when a primary caregiver was present, hospital staff frequently directed teaching exclusively toward the patient. The extent to which caregivers feel adequately prepared for their roles and responsibilities needs further exploration.

CONCLUSION

Our applied ethnographic approach reveals that patients experience several challenges while transitioning from hospital to home. Reducing readmissions is likely to remain challenging unless we broaden our understanding of the types of support and coaching required. We are translating our findings into quality improvement activities, conducting pilot projects focusing on risk stratification and tailoring of care, a specialized phone number for recently discharged patients, standardized same‐day discharge summaries to primary care providers, medication reconciliation, follow‐up phone calls, and scheduling appointments before discharge.

The transition from hospital to home is a complex event offering multiple provider‐identified opportunities to improve healthcare quality.18 Centering care delivery around patient needs and preferences is both inherently valuable and linked with better outcomes.9

The Care Transitions Measure (CTM) identifies 4 domains of patient experience related to hospital discharge: information transfer, patient and caregiver preparation, self‐management support, and empowerment to assert preferences.10 It discriminates between patients who do or do not experience a subsequent readmission or emergency room visit and between levels of care coordination.11 Quality indicators like the CTM are important tools for systematic healthcare improvement, but they provide a limited understanding of patient experiences, which can drive the transformation of systems.12, 13

With the exception of patients with a few specific clinical conditions, relatively little is known about how adult patients perceive the hospital‐to‐home transition.1417 They recall receiving discharge instructions but lack details about what to do if problems arise.18 They may lack important information despite receiving instruction.19 Caregivers report problems related to emotional support, discharge planning, and family participation,20 and patients and caregivers express anxiety, confusion, a sense of abandonment by the healthcare system, and the perception that their preferences are disregarded.21

As part of ongoing quality improvement activities, we sought to develop a richly detailed, patient‐centered view of the hospital‐to‐home transition. Our purpose was to understand patient and caregiver experiences during this pivotal healthcare experience.

METHODS

We used an applied ethnographic approach,22 conducting participant observation and video recording in‐depth, semi‐structured interviews in Kaiser Permanente Southern California, Colorado, and Hawaii. The United States' largest, private, not‐for‐profit integrated healthcare delivery system, Kaiser Permanente addresses all health needs for more than 8.9 million members.

To balance the pragmatic imperatives of quality improvement with obtaining enough information to understand patient experiences, we planned a sample of 24 patients across 3 settings with a mix of resource‐intensive and less‐intensive healthcare needs. We defined resource‐intensive needs as occurring among patients aged 65 or older with 3 or more chronic conditions. We asked hospital staff to identify patients by level of need and variety in diagnoses and illness severity, planned or unplanned hospitalizations, age, and ability to self manage. Reasons for admission included joint replacement, acute appendicitis, chronic illness exacerbation, complications of cancer chemotherapy, and others. We included patients who were inpatients or discharged no more than 3 weeks before interview. We excluded those under the age of 18 or discharged to non‐home settings. The project took place between September and November of 2008; 24 patients, half of whom were male, gave written informed consent for video recordings and authorization to distribute protected health information throughout and beyond Kaiser Permanente for quality improvement and educational purposes. Participants took part in interviews and observations lasting 1 to 3 hours; caregivers and family members participated in 9 instances.

Two or 3 observers attended each interview, which took place in the hospital on discharge day, at postdischarge appointments, or in patients' homes. Open‐ended questions prompted broad‐ranging inquiry into patients' lives, medical history, hospitalization experience, medications, care network, challenges, personal goals, and inner experience. Some questions were adapted and expanded from the CTM; others were prompts to demonstrate activities (eg, Can you show us how you organize your medications?). In addition to interviewing patients and caregivers, we observed interactions between patients, families, and hospital staff before discharge. We also observed patients and caregivers at home and when interacting with outpatient primary care providers. The purpose of observation was to understand the context of patient and caregiver experiences and to identify consistencies or discrepancies with their descriptions of experiences. (see Supporting Information In‐Home Interview Guide in the online version of this article)

Data included field notes and video recordings. In addition, observers summarized their strongest daily impressions as brief team stories that were shared with the observation team, local operations staff, and Kaiser Permanente national subject matter experts.23 Consistent with a grounded theory approach, interviews were professionally transcribed and qualitatively analyzed by multiple observers in iterative stages to develop broad domains of patient experiences.24 We clustered similar experiences and identified exemplar statements and behaviors. Team stories were analyzed separately, using a similar process. We reviewed recorded interviews to refine our emerging understanding of patient and caregiver experiences and discussed our observations and impressions about each domain. To maximize internal validity, an independent researcher who did not attend the interviews reviewed the transcripts and coding and participated in final qualitative analysis. Institutional review board approval was not required for this quality improvement project.

RESULTS

Patients and caregivers expressed or demonstrated 6 domains of experience as they transitioned from hospital to home (Table 1).

DISCUSSION

Using an ethnographic approach, we identified 6 domains of patient and caregiver experience during the hospital‐to‐home transition. Many needs in these domains arose in the hours and days after patients returned home, and patients and caregivers often found it challenging to meet them. Our project adds a detailed, patient‐centered perspective on the transition from hospital to home.

The domains we identified share some conceptual territory with the dimensions of the Care Transition Measure and the Transitional Care Model,25 but generate a more detailed understanding of patient and caregiver experiences. Key findings include the fact that patients can find it challenging to translate knowledge into contextually appropriate action at home. This confirms some published results. For instance, estimates of outpatient adherence to complicated regimens range from 5% to 77%.2629 Significant opportunities exist to improve the reliability of translating medication instructions into systems that work at home,30 including aligning medication lists with physical aids (such as weekly pill boxes) and explaining medications in patient‐friendly terms. We also found that same‐day discharge teaching can be ineffective because patients are anxious to leave the hospital or staff members feel rushed. Emotion can interfere with cognition, and transferring information shortly before hospital discharge may overlook learning readiness, a fundamental principle of patient education.31, 32 In addition, the desire to return to normal life, coupled with uncertainty about who to call for clarification, can lead patients to simply do the best they can with whatever information they recall.

The literature refers to handoffs of patients from one provider to another as an episode of care is completed, but our findings suggest patients perceive hospitalization as an event occurring within ongoing relationships with the healthcare providers to whom they feel most connected.33, 34 Some patient and caregiver needs could be addressed by actively supporting these relationships during the hospital‐to‐home transition: explicitly acknowledging their importance to patients, ensuring that providers have discharge information, and framing discharge as a transition back to the care of trusted providers. Some of our findings require system‐level changes. Patients and caregivers with unmet transportation needs expressed anxiety about how or if help would materialize. Partnerships with community organizations could enable healthcare organizations to address needs like transportation that fall outside traditional discharge activities but significantly impact patient experiences. In addition, healthcare organizations are rarely designed for straightforward navigation; patient‐centered organizational designs could eliminate the need for patients and caregivers to learn how to navigate. For instance, a single point of contact for recently discharged patients might improve the process of finding help.

Strengths of our quality improvement project include the range of patients we interviewed and in‐depth observations and interviews across settings. Ethnography is ideal for generating a rich understanding of patient experiences, allowing us to observe needs patients did not mention, as well as the physical and emotional context of the transition. Weaknesses of our approach include the fact that the experiences reflected in each category were determined, to some extent, by the questions we asked. This may have constrained the variety of experiences patients reported. In addition, Kaiser Permanente's integrated nature may have affected our findings, although we believe patients and caregivers reported experiences that are likely universal.

Our project occurred in a healthcare system with an integrated electronic health record (EHR). Interventions to improve provider‐identified gaps in the discharge process often rely on information technology.3543 However, information technology does not eliminate continuity of care issues.44 Our EHR is widely used, but available information did not consistently ensure strong enough care coordination or good communication.

Including the patient's primary caregiver in discharge teaching appeared to be a relatively low priority for hospital staff, unless there was a hands‐on care task. Even when a primary caregiver was present, hospital staff frequently directed teaching exclusively toward the patient. The extent to which caregivers feel adequately prepared for their roles and responsibilities needs further exploration.

CONCLUSION

Our applied ethnographic approach reveals that patients experience several challenges while transitioning from hospital to home. Reducing readmissions is likely to remain challenging unless we broaden our understanding of the types of support and coaching required. We are translating our findings into quality improvement activities, conducting pilot projects focusing on risk stratification and tailoring of care, a specialized phone number for recently discharged patients, standardized same‐day discharge summaries to primary care providers, medication reconciliation, follow‐up phone calls, and scheduling appointments before discharge.

Over the past decade, research has demonstrated a value gap in US healthcare, characterized by rapidly rising costs and substandard quality.1, 2 Public reporting of hospital performance data is one of several strategies promoted to help address these deficiencies. To this end, a number of hospital rating services have created Web sites aimed at healthcare consumers.3 These services provide information about multiple aspects of healthcare quality, which in theory might be used by patients when deciding where to seek medical care.

Despite the increasing availability of publicly reported quality data comparing doctors and hospitals, a 2008 survey found that only 14% of Americans have seen and used such information in the past year, a decrease from 2006 (36%).4 A similar study in 2007 found that after seeking input from family and friends, patients generally rely on their primary care physician (PCP) to assist them to make decisions about where to have elective surgery.5 Surprisingly, almost nothing is known about how publicly reported data is used, if at all, by PCPs in the referral of patients to hospitals.

The physician is an important intermediary in the buying process for many healthcare services.6 Tertiary care hospitals depend on physician referrals for much of their patient volume.7 Until the emergence of the hospitalist model of care, most primary care physicians cared for their own hospitalized patients, and thus hospital referral decisions were largely driven by the PCP's admitting privileges. However, following the rapid expansion of the hospitalist movement,8, 9 there has been a sharp decrease in the number of PCPs who provide direct patient care for their hospitalized patients.8 As a result, PCPs may now have more choice in regards to hospital referrals for general medical conditions. Potential factors influencing a PCP's referral decisions might include familiarity with the hospital, care quality, patient convenience, satisfaction with the hospital, or hospital reputation.

Studies of cardiac surgery report cards in New York9 and Pennsylvania,10 conducted in the mid‐1990s, found that cardiologists did not use publicly reported mortality data in referral decisions, nor did they share it with patients. Over the past 2 decades, public reporting has grown exponentially, and now includes many measures of structure, processes, and outcomes for almost all US hospitals, available for free over the Internet. The growth of the patient safety movement and mandated public reporting might also have affected physicians' views about publicly reported quality data. We surveyed primary care physicians to determine the extent to which they use information about hospital quality in their referral decisions for community‐acquired pneumonia, and to identify other factors that might influence referral decisions.

METHODS

We obtained an e‐mail list of primary care physicians from the medical staff offices of all area hospitals within a 10‐mile radius of Springfield, MA (Baystate Medical Center, Holyoke Medical Center, and Mercy Medical Center). Baystate Medical Center is a 659‐bed academic medical center and Level 1 trauma center, while Holyoke and Mercy Medical Center are both 180‐bed acute care hospitals. Physicians were contacted via e‐mail from June through September of 2009, and asked to participate in an anonymous, 10‐minute, online survey accessible through an Internet link (SurveyMonkey.com) about factors influencing a primary care physician's hospital referral choice for a patient with pneumonia. To facilitate participation, we sent 2 follow‐up e‐mail reminders, and respondents who completed the entire survey received a $15 gift card. The study was approved by the institutional review board of Baystate Medical Center and closed to participation on September 23, 2009.

We created the online survey based on previous research7 and approximately 10 key informant interviews. The survey (see Supporting Information, Appendix, in the online version of this article) contained 13 demographic questions and 10 questions based on a case study of pneumonia (Figure 1). The instrument was pilot tested for clarity with a small group of primary care physicians at the author's institution and subsequently modified. We chose pneumonia because it is a common reason for a PCP to make an urgent hospital referral,11 and because there is a well‐established set of quality measures that are publicly reported.12 Unlike elective surgery, for which patients might research hospitals or surgeons on their own, patients with pneumonia would likely rely on their PCP to recommend a hospital for urgent referral. In contrast, PCPs know they will refer a number of pneumonia patients to hospitals each year and therefore might have an interest in comparing the publicly reported quality measures for local hospitals.

Figure 1

Respondents were shown the case study and asked to refer the hypothetical patient to 1 of 4 area hospitals. Respondents were asked to rate (on a 3‐point scale: not at all, somewhat, or very) the importance of the following factors in their referral decision: waiting time in the emergency room, distance traveled by the patient, experience of other patients, severity of patient's illness, patient's insurance, hospital's reputation among other physicians and partners, admitting privileges with a specific hospital, admitting arrangements with a hospitalist group, familiarity with the hospital, availability of subspecialists, quality of subspecialists, nursing quality, nursing staffing ratios, hospital's case volume for pneumonia, publicly available quality measures, patient preference, distance from your practice, shared electronic record system, and quality of hospital discharge summaries. Next, we measured provider's awareness of publicly reported hospital quality data and whether they used such data in referring patients or choosing their own medical care. Specifically, we asked about familiarity with the following 4 Web sites: Massachusetts Quality and Cost (a state‐specific Web site produced by the Massachusetts Executive Office of Health and Human Services)13; Hospital Compare (a Web site developed and maintained by Centers for Medicare and Medicaid Services [CMS] and the Department of Health and Human Services)14; Leapfrog Group (a private, nonprofit organization)15; and Health Grades (a private, for‐profit company).16

We then asked participants to rate the importance of the following performance measures when judging a hospital's performance: antibiotics within 6 hours of arrival to the hospital, appropriate initial antibiotic, blood culture drawn before antibiotics given, smoking cessation advice/counseling, oxygenation assessment, risk‐adjusted mortality, intensive care unit staffing, influenza vaccination, pneumococcal vaccination, Leapfrog's never events,15 volume, Leapfrog safe practices score, cost, computerized physician order entry system, Magnet status,17 and U.S. News & World Report's Best Hospitals designation.18 Lastly, we asked participants to state, using a 3‐point scale (agree, disagree, neutral), their level of agreement that the following factors, adapted from Schneider and Epstein,10 represented limitations of public reporting: 1) risk‐adjusted methods are inadequate to compare hospitals fairly; 2) mortality rates are an incomplete indication of the quality of a hospital's care; 3) hospitals can manipulate the data; and 4) ratings are inaccurate for hospitals with small caseloads.

Factors associated with physicians' knowledge of publicly reported data were analyzed with bivariate analysis. Since all factors are categorical, chi‐square analysis was used for bivariate analysis. No factor had a P value <0.2 on bivariate analysis, thus multiple logistic regression was not performed.

RESULTS

Of 194 primary care physicians who received invitations, 92 responded (response rate of 47%). See Table 1 for respondents' characteristics. All age groups were represented; most were male and between 3554 years of age. Respondents were evenly divided between those who owned their own practices (54%) and those working for a health system (46%). Ninety‐three percent of PCPs maintained admitting privileges (45% to more than 1 hospital), but only 20% continued to admit their own patients. When asked where they would send a hypothetical pneumonia patient, only 4% of PCPs chose a hospital to which they had never had admitting privileges.

Physician's ratings of the importance of various factors in their referral decision are shown in Figure 2. The following factors were most often considered very important: familiarity with the hospital (70%), patient preference (62%), and admitting arrangements with a hospitalist group (62%). In contrast, only 18% of physicians viewed publicly available hospital quality measures as very important when making a referral decision. Factors most often rated not at all important to participants' decisions were patient insurance (48%), hospital's case volume for pneumonia (48%), and publicly available quality measures (42%).

Figure 2

Of the 61% who were aware of Web sites that report hospital quality, most (52%) were familiar with Massachusetts Quality and Cost, while few (27%) were familiar with Hospital Compare. None of the physicians we surveyed reported having used publicly reported quality information when making a referral decision or having discussed such data with their patients. However, 49% stated that publicly reported performance data was somewhat and 10% very important to decisions regarding the medical care they receive. None of the demographic characteristics that we assessed (including age, gender, or years out of medical school) were associated with awareness of publicly reported data in bivariate analyses.

Respondents' ratings of specific quality measures appear in Figure 3. PCPs most often identified the following factors as being very important when judging hospital quality: percent of pneumonia patients given initial antibiotics within 6 hours after arrival (66%), percent of pneumonia patients given the most appropriate initial antibiotic (63%), and percent of pneumonia patients whose initial emergency room (ER) blood culture was performed prior to the administration of the first hospital dose of antibiotics (51%). The factors most often rated not at all important included: U.S. News & World Report's Best Hospitals designation (57%), Magnet Status (42%), and computer physician order entry system (40%).

Figure 3

When asked about limitations of publicly reported performance data, 42% agreed that risk‐adjusted methods were inadequate to compare hospitals fairly, 76% agreed that mortality rates were an incomplete indication of the quality of hospitals care, 62% agreed that hospitals could manipulate the data, and 72% agreed that the ratings were inaccurate for hospitals with small caseloads.

DISCUSSION

In 2003, the Hospital Quality Alliance began a voluntary public reporting program of hospital performance measures, for pneumonia, acute myocardial infarction, and congestive heart failure, that was intended to encourage quality improvement activity by hospitals, and to provide patients and referring physicians with information to make better‐informed choices.19 These data are now easily available to the public through a free Web site (http://www.hospitalcompare.hhs.gov) sponsored by CMS and promoted in various ways, including newspaper advertisements.20 We found that, despite these efforts, just over half of the respondents were aware of Web sites that report hospital quality data, and only 1 in 6 had heard of Hospital Compare. Even those PCPs who were knowledgeable about public reporting did not incorporate publicly reported data into their referral decisions. Instead, they base their referral decisions on familiarity with the hospital, patient preference, and admitting arrangements with a hospitalist group.

Despite their lack of familiarity with Hospital Compare, it was the quality measures that are reported by Hospital Compare that they identified as the best indicators of hospital quality: appropriate initial antibiotic, antibiotics within 6 hours, and blood cultures performed prior to the administration of antibiotics. In fact, the 5 measures most often cited as very important to judging hospital quality were all measures reported on Hospital Compare.

As the US healthcare system becomes increasingly complex and costly, there is a growing interest in providing patients with physician and hospital performance data to help them select the provider.21 It is postulated that if patients took a more active role in choosing healthcare providers, and were forced to assume greater financial responsibility, then consumerism will force improvements in quality of care while maintaining or even lowering costs.21 However, studies demonstrate that most patients are unaware of performance data and, if they are aware, still value familiarity over quality ratings.4 Moreover, patients rely on the knowledge of their primary care physician to guide them.5

This is the first study we are aware of that examines how primary care physicians use publicly reported quality data in hospital referral decisions. Studies from more than a decade ago found that publicly reported data had minimal impact on referral decisions from cardiologists to cardiac surgeons. A survey of Pennsylvania's cardiologists and cardiac surgeons showed that although 82% were aware of risk‐adjusted mortality rates published for surgeons, only 10% of cardiologists reported these to be very important when evaluating the performance of a cardiothoracic surgeon. Furthermore, 87% of cardiologists stated that mortality and case volume information reported on cardiac surgeons had minimal or no influence on their referral practices.10 In 1997, a survey of cardiologists in New York found that only 38% of respondents reported that risk‐adjusted outcome data had affected their referrals to surgeons very much or somewhat.9 In addition, most authors conclude that public reporting has had little or no effect on market share.22 Despite growth in the number of measures and improved accessibility, our physicians were even less likely to be aware of, or use, publicly reported data than physicians a decade earlier.

Of course, even if public reporting does not influence referral patterns, it could still improve healthcare quality in several ways. First, feedback about performance may focus quality improvement activities in specific areas that represent gaps in care.10 This could take the form of an appeal to professionalism,23 or the desire to preserve one's reputation by not appearing on a list of poor performers.24 Second, hospitals' desire to appear on lists of high performers, such as U.S. News & World Report's hospital rankings, for marketing purposes, might stimulate improvement activities.10 Finally, publicly reported measures could form the basis for pay‐for‐performance incentives that further speed improvement.25

Our study has several limitations. First, our sample size was small and restricted to 1 region of 1 state, and may not be representative of either the state or nation as a whole. Still, our area has a high level of Internet use, and several local hospitals have been at the vanguard of the quality movement, generally scoring above both state and national averages on Hospital Compare. In addition, Massachusetts has made substantial efforts to promote its own public reporting program, and half the surveyed physicians reported being aware of the Massachusetts Quality and Cost Web site. The fact that not a single area physician surveyed used publicly reported data when making referral decisions is sobering. We believe it is unlikely that other areas of the country will have a substantially higher rate of use. Similarly, our response rate was under 50%. Physicians who did not take the survey may have differed in important ways from those who did. Nevertheless, our sample included a broad range of physician ages, practice types, and affiliations. It seems unlikely that those who did not respond would be more inclined to use publicly reported data than those who did. Second, we assessed decision‐making around a single medical condition. Physicians may have used publicly reported data for other decisions. However, the condition we chose was both urgent (as opposed to emergent) and possesses a robust set of publicly reported quality measures. If physicians do not use publicly reported data for this decision, it seems unlikely they would use it for conditions that have fewer reliable measures (eg, gall bladder surgery) or where the choice of hospital is generally made in an ambulance (eg, myocardial infarction). Finally, the low awareness of public reporting made it difficult for some physicians to answer some of the questions regarding publicly reported hospital quality data because they were unfamiliar with the language utilized by the Web sites (eg, magnet status, Leapfrog never events). It is possible that our results may have been altered slightly if a glossary had been provided.

Despite these limitations, our study suggests that more than 6 years after the launch of the Hospital Quality Alliance, primary care physicians do not appear to make use of these data when choosing a hospital for their patients suffering from pneumonia. Instead, they rely on familiarity with a hospital and past relationships. Even though a majority of the physicians surveyed no longer admitted their own patients, they continue to send patients to hospitals where they had privileges. This finding is not surprising, as physicians also cling to familiar therapies, and may be reluctant to prescribe a new medication or perform an unfamiliar procedure, even if it is indicated. Such reliance on familiarity may make physicians feel comfortable, but does not always result in the best care for patients. Acquiring familiarity, however, requires time and effort, something that physicians generally have in short supply; and while there are plenty of industry representatives to overcome physicians' hesitancy to prescribe new treatments, there are no analogous agents to educate physicians about public reporting or to help them overcome hesitancy about trying a new hospital.

Suspicion about the validity of public reporting may also play a role in the physicians' reported behavior. In past studies of cardiac report cards, cardiologists were most concerned that risk adjustment methods were inadequate (77%) and that mortality rates were an incomplete indicator of the quality of surgical care (74%). They were less concerned about manipulation of data (52%) or small caseloads (15%).10 Our physicians were also concerned that mortality rates were an incomplete measure of quality (76%) but less concerned about risk adjustment (42%), perhaps because many structure and process measures are not subject to risk adjustment. In contrast, they were somewhat more concerned that hospitals could manipulate the data (62%), which again may reflect process measures versus mortality statistics. Other reasons for not using the data may include a lack of awareness of the data or how to access it, or a belief that hospitals do not vary in quality.

Interestingly, even though most respondents were not aware of Hospital Compare, they found the information presented there to best reflect the overall hospital quality. Also, while respondents indicated that they did not use publicly reported data when referring patients, almost half of PCPs reported that publicly reported performance data was at least somewhat important in choosing their own medical care. Thus, although public reporting appears not to have reached its full potential, some publicly reported quality measures have clearly entered the consciousness of PCPs. In contrast, other highly touted measures such as computerized physician order entry systems were not appreciated, and popular designations such as U.S. News & World Report's Best Hospitals were least valued, even though 1 area hospital carries this designation. One conclusion might be that CMS should abandon Hospital Compare since neither patients4 nor providers use it. However, public reporting may improve quality in other ways. Moreover, physicians appear interested in the data even if they are not aware of it. Therefore, given the large investment by CMS and individual hospitals in collecting the data required for Hospital Compare, CMS might consider making greater efforts to increase primary care physician awareness of the Hospital Compare Web site. At the same time, high‐performing hospitals may want to communicate their performance scores to local PCPs as part of their marketing strategy. Future studies could assess whether such practices affect physician referral decisions and subsequent market share of high‐performing hospitals.

Over the past decade, research has demonstrated a value gap in US healthcare, characterized by rapidly rising costs and substandard quality.1, 2 Public reporting of hospital performance data is one of several strategies promoted to help address these deficiencies. To this end, a number of hospital rating services have created Web sites aimed at healthcare consumers.3 These services provide information about multiple aspects of healthcare quality, which in theory might be used by patients when deciding where to seek medical care.

Despite the increasing availability of publicly reported quality data comparing doctors and hospitals, a 2008 survey found that only 14% of Americans have seen and used such information in the past year, a decrease from 2006 (36%).4 A similar study in 2007 found that after seeking input from family and friends, patients generally rely on their primary care physician (PCP) to assist them to make decisions about where to have elective surgery.5 Surprisingly, almost nothing is known about how publicly reported data is used, if at all, by PCPs in the referral of patients to hospitals.

The physician is an important intermediary in the buying process for many healthcare services.6 Tertiary care hospitals depend on physician referrals for much of their patient volume.7 Until the emergence of the hospitalist model of care, most primary care physicians cared for their own hospitalized patients, and thus hospital referral decisions were largely driven by the PCP's admitting privileges. However, following the rapid expansion of the hospitalist movement,8, 9 there has been a sharp decrease in the number of PCPs who provide direct patient care for their hospitalized patients.8 As a result, PCPs may now have more choice in regards to hospital referrals for general medical conditions. Potential factors influencing a PCP's referral decisions might include familiarity with the hospital, care quality, patient convenience, satisfaction with the hospital, or hospital reputation.

Studies of cardiac surgery report cards in New York9 and Pennsylvania,10 conducted in the mid‐1990s, found that cardiologists did not use publicly reported mortality data in referral decisions, nor did they share it with patients. Over the past 2 decades, public reporting has grown exponentially, and now includes many measures of structure, processes, and outcomes for almost all US hospitals, available for free over the Internet. The growth of the patient safety movement and mandated public reporting might also have affected physicians' views about publicly reported quality data. We surveyed primary care physicians to determine the extent to which they use information about hospital quality in their referral decisions for community‐acquired pneumonia, and to identify other factors that might influence referral decisions.

METHODS

We obtained an e‐mail list of primary care physicians from the medical staff offices of all area hospitals within a 10‐mile radius of Springfield, MA (Baystate Medical Center, Holyoke Medical Center, and Mercy Medical Center). Baystate Medical Center is a 659‐bed academic medical center and Level 1 trauma center, while Holyoke and Mercy Medical Center are both 180‐bed acute care hospitals. Physicians were contacted via e‐mail from June through September of 2009, and asked to participate in an anonymous, 10‐minute, online survey accessible through an Internet link (SurveyMonkey.com) about factors influencing a primary care physician's hospital referral choice for a patient with pneumonia. To facilitate participation, we sent 2 follow‐up e‐mail reminders, and respondents who completed the entire survey received a $15 gift card. The study was approved by the institutional review board of Baystate Medical Center and closed to participation on September 23, 2009.

We created the online survey based on previous research7 and approximately 10 key informant interviews. The survey (see Supporting Information, Appendix, in the online version of this article) contained 13 demographic questions and 10 questions based on a case study of pneumonia (Figure 1). The instrument was pilot tested for clarity with a small group of primary care physicians at the author's institution and subsequently modified. We chose pneumonia because it is a common reason for a PCP to make an urgent hospital referral,11 and because there is a well‐established set of quality measures that are publicly reported.12 Unlike elective surgery, for which patients might research hospitals or surgeons on their own, patients with pneumonia would likely rely on their PCP to recommend a hospital for urgent referral. In contrast, PCPs know they will refer a number of pneumonia patients to hospitals each year and therefore might have an interest in comparing the publicly reported quality measures for local hospitals.

Figure 1

Respondents were shown the case study and asked to refer the hypothetical patient to 1 of 4 area hospitals. Respondents were asked to rate (on a 3‐point scale: not at all, somewhat, or very) the importance of the following factors in their referral decision: waiting time in the emergency room, distance traveled by the patient, experience of other patients, severity of patient's illness, patient's insurance, hospital's reputation among other physicians and partners, admitting privileges with a specific hospital, admitting arrangements with a hospitalist group, familiarity with the hospital, availability of subspecialists, quality of subspecialists, nursing quality, nursing staffing ratios, hospital's case volume for pneumonia, publicly available quality measures, patient preference, distance from your practice, shared electronic record system, and quality of hospital discharge summaries. Next, we measured provider's awareness of publicly reported hospital quality data and whether they used such data in referring patients or choosing their own medical care. Specifically, we asked about familiarity with the following 4 Web sites: Massachusetts Quality and Cost (a state‐specific Web site produced by the Massachusetts Executive Office of Health and Human Services)13; Hospital Compare (a Web site developed and maintained by Centers for Medicare and Medicaid Services [CMS] and the Department of Health and Human Services)14; Leapfrog Group (a private, nonprofit organization)15; and Health Grades (a private, for‐profit company).16

We then asked participants to rate the importance of the following performance measures when judging a hospital's performance: antibiotics within 6 hours of arrival to the hospital, appropriate initial antibiotic, blood culture drawn before antibiotics given, smoking cessation advice/counseling, oxygenation assessment, risk‐adjusted mortality, intensive care unit staffing, influenza vaccination, pneumococcal vaccination, Leapfrog's never events,15 volume, Leapfrog safe practices score, cost, computerized physician order entry system, Magnet status,17 and U.S. News & World Report's Best Hospitals designation.18 Lastly, we asked participants to state, using a 3‐point scale (agree, disagree, neutral), their level of agreement that the following factors, adapted from Schneider and Epstein,10 represented limitations of public reporting: 1) risk‐adjusted methods are inadequate to compare hospitals fairly; 2) mortality rates are an incomplete indication of the quality of a hospital's care; 3) hospitals can manipulate the data; and 4) ratings are inaccurate for hospitals with small caseloads.

Factors associated with physicians' knowledge of publicly reported data were analyzed with bivariate analysis. Since all factors are categorical, chi‐square analysis was used for bivariate analysis. No factor had a P value <0.2 on bivariate analysis, thus multiple logistic regression was not performed.

RESULTS

Of 194 primary care physicians who received invitations, 92 responded (response rate of 47%). See Table 1 for respondents' characteristics. All age groups were represented; most were male and between 3554 years of age. Respondents were evenly divided between those who owned their own practices (54%) and those working for a health system (46%). Ninety‐three percent of PCPs maintained admitting privileges (45% to more than 1 hospital), but only 20% continued to admit their own patients. When asked where they would send a hypothetical pneumonia patient, only 4% of PCPs chose a hospital to which they had never had admitting privileges.

Physician's ratings of the importance of various factors in their referral decision are shown in Figure 2. The following factors were most often considered very important: familiarity with the hospital (70%), patient preference (62%), and admitting arrangements with a hospitalist group (62%). In contrast, only 18% of physicians viewed publicly available hospital quality measures as very important when making a referral decision. Factors most often rated not at all important to participants' decisions were patient insurance (48%), hospital's case volume for pneumonia (48%), and publicly available quality measures (42%).

Figure 2

Of the 61% who were aware of Web sites that report hospital quality, most (52%) were familiar with Massachusetts Quality and Cost, while few (27%) were familiar with Hospital Compare. None of the physicians we surveyed reported having used publicly reported quality information when making a referral decision or having discussed such data with their patients. However, 49% stated that publicly reported performance data was somewhat and 10% very important to decisions regarding the medical care they receive. None of the demographic characteristics that we assessed (including age, gender, or years out of medical school) were associated with awareness of publicly reported data in bivariate analyses.

Respondents' ratings of specific quality measures appear in Figure 3. PCPs most often identified the following factors as being very important when judging hospital quality: percent of pneumonia patients given initial antibiotics within 6 hours after arrival (66%), percent of pneumonia patients given the most appropriate initial antibiotic (63%), and percent of pneumonia patients whose initial emergency room (ER) blood culture was performed prior to the administration of the first hospital dose of antibiotics (51%). The factors most often rated not at all important included: U.S. News & World Report's Best Hospitals designation (57%), Magnet Status (42%), and computer physician order entry system (40%).

Figure 3

When asked about limitations of publicly reported performance data, 42% agreed that risk‐adjusted methods were inadequate to compare hospitals fairly, 76% agreed that mortality rates were an incomplete indication of the quality of hospitals care, 62% agreed that hospitals could manipulate the data, and 72% agreed that the ratings were inaccurate for hospitals with small caseloads.

DISCUSSION

In 2003, the Hospital Quality Alliance began a voluntary public reporting program of hospital performance measures, for pneumonia, acute myocardial infarction, and congestive heart failure, that was intended to encourage quality improvement activity by hospitals, and to provide patients and referring physicians with information to make better‐informed choices.19 These data are now easily available to the public through a free Web site (http://www.hospitalcompare.hhs.gov) sponsored by CMS and promoted in various ways, including newspaper advertisements.20 We found that, despite these efforts, just over half of the respondents were aware of Web sites that report hospital quality data, and only 1 in 6 had heard of Hospital Compare. Even those PCPs who were knowledgeable about public reporting did not incorporate publicly reported data into their referral decisions. Instead, they base their referral decisions on familiarity with the hospital, patient preference, and admitting arrangements with a hospitalist group.

Despite their lack of familiarity with Hospital Compare, it was the quality measures that are reported by Hospital Compare that they identified as the best indicators of hospital quality: appropriate initial antibiotic, antibiotics within 6 hours, and blood cultures performed prior to the administration of antibiotics. In fact, the 5 measures most often cited as very important to judging hospital quality were all measures reported on Hospital Compare.

As the US healthcare system becomes increasingly complex and costly, there is a growing interest in providing patients with physician and hospital performance data to help them select the provider.21 It is postulated that if patients took a more active role in choosing healthcare providers, and were forced to assume greater financial responsibility, then consumerism will force improvements in quality of care while maintaining or even lowering costs.21 However, studies demonstrate that most patients are unaware of performance data and, if they are aware, still value familiarity over quality ratings.4 Moreover, patients rely on the knowledge of their primary care physician to guide them.5

This is the first study we are aware of that examines how primary care physicians use publicly reported quality data in hospital referral decisions. Studies from more than a decade ago found that publicly reported data had minimal impact on referral decisions from cardiologists to cardiac surgeons. A survey of Pennsylvania's cardiologists and cardiac surgeons showed that although 82% were aware of risk‐adjusted mortality rates published for surgeons, only 10% of cardiologists reported these to be very important when evaluating the performance of a cardiothoracic surgeon. Furthermore, 87% of cardiologists stated that mortality and case volume information reported on cardiac surgeons had minimal or no influence on their referral practices.10 In 1997, a survey of cardiologists in New York found that only 38% of respondents reported that risk‐adjusted outcome data had affected their referrals to surgeons very much or somewhat.9 In addition, most authors conclude that public reporting has had little or no effect on market share.22 Despite growth in the number of measures and improved accessibility, our physicians were even less likely to be aware of, or use, publicly reported data than physicians a decade earlier.

Of course, even if public reporting does not influence referral patterns, it could still improve healthcare quality in several ways. First, feedback about performance may focus quality improvement activities in specific areas that represent gaps in care.10 This could take the form of an appeal to professionalism,23 or the desire to preserve one's reputation by not appearing on a list of poor performers.24 Second, hospitals' desire to appear on lists of high performers, such as U.S. News & World Report's hospital rankings, for marketing purposes, might stimulate improvement activities.10 Finally, publicly reported measures could form the basis for pay‐for‐performance incentives that further speed improvement.25

Our study has several limitations. First, our sample size was small and restricted to 1 region of 1 state, and may not be representative of either the state or nation as a whole. Still, our area has a high level of Internet use, and several local hospitals have been at the vanguard of the quality movement, generally scoring above both state and national averages on Hospital Compare. In addition, Massachusetts has made substantial efforts to promote its own public reporting program, and half the surveyed physicians reported being aware of the Massachusetts Quality and Cost Web site. The fact that not a single area physician surveyed used publicly reported data when making referral decisions is sobering. We believe it is unlikely that other areas of the country will have a substantially higher rate of use. Similarly, our response rate was under 50%. Physicians who did not take the survey may have differed in important ways from those who did. Nevertheless, our sample included a broad range of physician ages, practice types, and affiliations. It seems unlikely that those who did not respond would be more inclined to use publicly reported data than those who did. Second, we assessed decision‐making around a single medical condition. Physicians may have used publicly reported data for other decisions. However, the condition we chose was both urgent (as opposed to emergent) and possesses a robust set of publicly reported quality measures. If physicians do not use publicly reported data for this decision, it seems unlikely they would use it for conditions that have fewer reliable measures (eg, gall bladder surgery) or where the choice of hospital is generally made in an ambulance (eg, myocardial infarction). Finally, the low awareness of public reporting made it difficult for some physicians to answer some of the questions regarding publicly reported hospital quality data because they were unfamiliar with the language utilized by the Web sites (eg, magnet status, Leapfrog never events). It is possible that our results may have been altered slightly if a glossary had been provided.

Despite these limitations, our study suggests that more than 6 years after the launch of the Hospital Quality Alliance, primary care physicians do not appear to make use of these data when choosing a hospital for their patients suffering from pneumonia. Instead, they rely on familiarity with a hospital and past relationships. Even though a majority of the physicians surveyed no longer admitted their own patients, they continue to send patients to hospitals where they had privileges. This finding is not surprising, as physicians also cling to familiar therapies, and may be reluctant to prescribe a new medication or perform an unfamiliar procedure, even if it is indicated. Such reliance on familiarity may make physicians feel comfortable, but does not always result in the best care for patients. Acquiring familiarity, however, requires time and effort, something that physicians generally have in short supply; and while there are plenty of industry representatives to overcome physicians' hesitancy to prescribe new treatments, there are no analogous agents to educate physicians about public reporting or to help them overcome hesitancy about trying a new hospital.

Suspicion about the validity of public reporting may also play a role in the physicians' reported behavior. In past studies of cardiac report cards, cardiologists were most concerned that risk adjustment methods were inadequate (77%) and that mortality rates were an incomplete indicator of the quality of surgical care (74%). They were less concerned about manipulation of data (52%) or small caseloads (15%).10 Our physicians were also concerned that mortality rates were an incomplete measure of quality (76%) but less concerned about risk adjustment (42%), perhaps because many structure and process measures are not subject to risk adjustment. In contrast, they were somewhat more concerned that hospitals could manipulate the data (62%), which again may reflect process measures versus mortality statistics. Other reasons for not using the data may include a lack of awareness of the data or how to access it, or a belief that hospitals do not vary in quality.

Interestingly, even though most respondents were not aware of Hospital Compare, they found the information presented there to best reflect the overall hospital quality. Also, while respondents indicated that they did not use publicly reported data when referring patients, almost half of PCPs reported that publicly reported performance data was at least somewhat important in choosing their own medical care. Thus, although public reporting appears not to have reached its full potential, some publicly reported quality measures have clearly entered the consciousness of PCPs. In contrast, other highly touted measures such as computerized physician order entry systems were not appreciated, and popular designations such as U.S. News & World Report's Best Hospitals were least valued, even though 1 area hospital carries this designation. One conclusion might be that CMS should abandon Hospital Compare since neither patients4 nor providers use it. However, public reporting may improve quality in other ways. Moreover, physicians appear interested in the data even if they are not aware of it. Therefore, given the large investment by CMS and individual hospitals in collecting the data required for Hospital Compare, CMS might consider making greater efforts to increase primary care physician awareness of the Hospital Compare Web site. At the same time, high‐performing hospitals may want to communicate their performance scores to local PCPs as part of their marketing strategy. Future studies could assess whether such practices affect physician referral decisions and subsequent market share of high‐performing hospitals.

Over the past decade, research has demonstrated a value gap in US healthcare, characterized by rapidly rising costs and substandard quality.1, 2 Public reporting of hospital performance data is one of several strategies promoted to help address these deficiencies. To this end, a number of hospital rating services have created Web sites aimed at healthcare consumers.3 These services provide information about multiple aspects of healthcare quality, which in theory might be used by patients when deciding where to seek medical care.

Despite the increasing availability of publicly reported quality data comparing doctors and hospitals, a 2008 survey found that only 14% of Americans have seen and used such information in the past year, a decrease from 2006 (36%).4 A similar study in 2007 found that after seeking input from family and friends, patients generally rely on their primary care physician (PCP) to assist them to make decisions about where to have elective surgery.5 Surprisingly, almost nothing is known about how publicly reported data is used, if at all, by PCPs in the referral of patients to hospitals.

The physician is an important intermediary in the buying process for many healthcare services.6 Tertiary care hospitals depend on physician referrals for much of their patient volume.7 Until the emergence of the hospitalist model of care, most primary care physicians cared for their own hospitalized patients, and thus hospital referral decisions were largely driven by the PCP's admitting privileges. However, following the rapid expansion of the hospitalist movement,8, 9 there has been a sharp decrease in the number of PCPs who provide direct patient care for their hospitalized patients.8 As a result, PCPs may now have more choice in regards to hospital referrals for general medical conditions. Potential factors influencing a PCP's referral decisions might include familiarity with the hospital, care quality, patient convenience, satisfaction with the hospital, or hospital reputation.

Studies of cardiac surgery report cards in New York9 and Pennsylvania,10 conducted in the mid‐1990s, found that cardiologists did not use publicly reported mortality data in referral decisions, nor did they share it with patients. Over the past 2 decades, public reporting has grown exponentially, and now includes many measures of structure, processes, and outcomes for almost all US hospitals, available for free over the Internet. The growth of the patient safety movement and mandated public reporting might also have affected physicians' views about publicly reported quality data. We surveyed primary care physicians to determine the extent to which they use information about hospital quality in their referral decisions for community‐acquired pneumonia, and to identify other factors that might influence referral decisions.

METHODS

We obtained an e‐mail list of primary care physicians from the medical staff offices of all area hospitals within a 10‐mile radius of Springfield, MA (Baystate Medical Center, Holyoke Medical Center, and Mercy Medical Center). Baystate Medical Center is a 659‐bed academic medical center and Level 1 trauma center, while Holyoke and Mercy Medical Center are both 180‐bed acute care hospitals. Physicians were contacted via e‐mail from June through September of 2009, and asked to participate in an anonymous, 10‐minute, online survey accessible through an Internet link (SurveyMonkey.com) about factors influencing a primary care physician's hospital referral choice for a patient with pneumonia. To facilitate participation, we sent 2 follow‐up e‐mail reminders, and respondents who completed the entire survey received a $15 gift card. The study was approved by the institutional review board of Baystate Medical Center and closed to participation on September 23, 2009.

We created the online survey based on previous research7 and approximately 10 key informant interviews. The survey (see Supporting Information, Appendix, in the online version of this article) contained 13 demographic questions and 10 questions based on a case study of pneumonia (Figure 1). The instrument was pilot tested for clarity with a small group of primary care physicians at the author's institution and subsequently modified. We chose pneumonia because it is a common reason for a PCP to make an urgent hospital referral,11 and because there is a well‐established set of quality measures that are publicly reported.12 Unlike elective surgery, for which patients might research hospitals or surgeons on their own, patients with pneumonia would likely rely on their PCP to recommend a hospital for urgent referral. In contrast, PCPs know they will refer a number of pneumonia patients to hospitals each year and therefore might have an interest in comparing the publicly reported quality measures for local hospitals.

Figure 1

Respondents were shown the case study and asked to refer the hypothetical patient to 1 of 4 area hospitals. Respondents were asked to rate (on a 3‐point scale: not at all, somewhat, or very) the importance of the following factors in their referral decision: waiting time in the emergency room, distance traveled by the patient, experience of other patients, severity of patient's illness, patient's insurance, hospital's reputation among other physicians and partners, admitting privileges with a specific hospital, admitting arrangements with a hospitalist group, familiarity with the hospital, availability of subspecialists, quality of subspecialists, nursing quality, nursing staffing ratios, hospital's case volume for pneumonia, publicly available quality measures, patient preference, distance from your practice, shared electronic record system, and quality of hospital discharge summaries. Next, we measured provider's awareness of publicly reported hospital quality data and whether they used such data in referring patients or choosing their own medical care. Specifically, we asked about familiarity with the following 4 Web sites: Massachusetts Quality and Cost (a state‐specific Web site produced by the Massachusetts Executive Office of Health and Human Services)13; Hospital Compare (a Web site developed and maintained by Centers for Medicare and Medicaid Services [CMS] and the Department of Health and Human Services)14; Leapfrog Group (a private, nonprofit organization)15; and Health Grades (a private, for‐profit company).16

We then asked participants to rate the importance of the following performance measures when judging a hospital's performance: antibiotics within 6 hours of arrival to the hospital, appropriate initial antibiotic, blood culture drawn before antibiotics given, smoking cessation advice/counseling, oxygenation assessment, risk‐adjusted mortality, intensive care unit staffing, influenza vaccination, pneumococcal vaccination, Leapfrog's never events,15 volume, Leapfrog safe practices score, cost, computerized physician order entry system, Magnet status,17 and U.S. News & World Report's Best Hospitals designation.18 Lastly, we asked participants to state, using a 3‐point scale (agree, disagree, neutral), their level of agreement that the following factors, adapted from Schneider and Epstein,10 represented limitations of public reporting: 1) risk‐adjusted methods are inadequate to compare hospitals fairly; 2) mortality rates are an incomplete indication of the quality of a hospital's care; 3) hospitals can manipulate the data; and 4) ratings are inaccurate for hospitals with small caseloads.

Factors associated with physicians' knowledge of publicly reported data were analyzed with bivariate analysis. Since all factors are categorical, chi‐square analysis was used for bivariate analysis. No factor had a P value <0.2 on bivariate analysis, thus multiple logistic regression was not performed.

RESULTS

Of 194 primary care physicians who received invitations, 92 responded (response rate of 47%). See Table 1 for respondents' characteristics. All age groups were represented; most were male and between 3554 years of age. Respondents were evenly divided between those who owned their own practices (54%) and those working for a health system (46%). Ninety‐three percent of PCPs maintained admitting privileges (45% to more than 1 hospital), but only 20% continued to admit their own patients. When asked where they would send a hypothetical pneumonia patient, only 4% of PCPs chose a hospital to which they had never had admitting privileges.

Physician's ratings of the importance of various factors in their referral decision are shown in Figure 2. The following factors were most often considered very important: familiarity with the hospital (70%), patient preference (62%), and admitting arrangements with a hospitalist group (62%). In contrast, only 18% of physicians viewed publicly available hospital quality measures as very important when making a referral decision. Factors most often rated not at all important to participants' decisions were patient insurance (48%), hospital's case volume for pneumonia (48%), and publicly available quality measures (42%).

Figure 2

Of the 61% who were aware of Web sites that report hospital quality, most (52%) were familiar with Massachusetts Quality and Cost, while few (27%) were familiar with Hospital Compare. None of the physicians we surveyed reported having used publicly reported quality information when making a referral decision or having discussed such data with their patients. However, 49% stated that publicly reported performance data was somewhat and 10% very important to decisions regarding the medical care they receive. None of the demographic characteristics that we assessed (including age, gender, or years out of medical school) were associated with awareness of publicly reported data in bivariate analyses.

Respondents' ratings of specific quality measures appear in Figure 3. PCPs most often identified the following factors as being very important when judging hospital quality: percent of pneumonia patients given initial antibiotics within 6 hours after arrival (66%), percent of pneumonia patients given the most appropriate initial antibiotic (63%), and percent of pneumonia patients whose initial emergency room (ER) blood culture was performed prior to the administration of the first hospital dose of antibiotics (51%). The factors most often rated not at all important included: U.S. News & World Report's Best Hospitals designation (57%), Magnet Status (42%), and computer physician order entry system (40%).

Figure 3

When asked about limitations of publicly reported performance data, 42% agreed that risk‐adjusted methods were inadequate to compare hospitals fairly, 76% agreed that mortality rates were an incomplete indication of the quality of hospitals care, 62% agreed that hospitals could manipulate the data, and 72% agreed that the ratings were inaccurate for hospitals with small caseloads.

DISCUSSION

In 2003, the Hospital Quality Alliance began a voluntary public reporting program of hospital performance measures, for pneumonia, acute myocardial infarction, and congestive heart failure, that was intended to encourage quality improvement activity by hospitals, and to provide patients and referring physicians with information to make better‐informed choices.19 These data are now easily available to the public through a free Web site (http://www.hospitalcompare.hhs.gov) sponsored by CMS and promoted in various ways, including newspaper advertisements.20 We found that, despite these efforts, just over half of the respondents were aware of Web sites that report hospital quality data, and only 1 in 6 had heard of Hospital Compare. Even those PCPs who were knowledgeable about public reporting did not incorporate publicly reported data into their referral decisions. Instead, they base their referral decisions on familiarity with the hospital, patient preference, and admitting arrangements with a hospitalist group.

Despite their lack of familiarity with Hospital Compare, it was the quality measures that are reported by Hospital Compare that they identified as the best indicators of hospital quality: appropriate initial antibiotic, antibiotics within 6 hours, and blood cultures performed prior to the administration of antibiotics. In fact, the 5 measures most often cited as very important to judging hospital quality were all measures reported on Hospital Compare.

As the US healthcare system becomes increasingly complex and costly, there is a growing interest in providing patients with physician and hospital performance data to help them select the provider.21 It is postulated that if patients took a more active role in choosing healthcare providers, and were forced to assume greater financial responsibility, then consumerism will force improvements in quality of care while maintaining or even lowering costs.21 However, studies demonstrate that most patients are unaware of performance data and, if they are aware, still value familiarity over quality ratings.4 Moreover, patients rely on the knowledge of their primary care physician to guide them.5

This is the first study we are aware of that examines how primary care physicians use publicly reported quality data in hospital referral decisions. Studies from more than a decade ago found that publicly reported data had minimal impact on referral decisions from cardiologists to cardiac surgeons. A survey of Pennsylvania's cardiologists and cardiac surgeons showed that although 82% were aware of risk‐adjusted mortality rates published for surgeons, only 10% of cardiologists reported these to be very important when evaluating the performance of a cardiothoracic surgeon. Furthermore, 87% of cardiologists stated that mortality and case volume information reported on cardiac surgeons had minimal or no influence on their referral practices.10 In 1997, a survey of cardiologists in New York found that only 38% of respondents reported that risk‐adjusted outcome data had affected their referrals to surgeons very much or somewhat.9 In addition, most authors conclude that public reporting has had little or no effect on market share.22 Despite growth in the number of measures and improved accessibility, our physicians were even less likely to be aware of, or use, publicly reported data than physicians a decade earlier.

Of course, even if public reporting does not influence referral patterns, it could still improve healthcare quality in several ways. First, feedback about performance may focus quality improvement activities in specific areas that represent gaps in care.10 This could take the form of an appeal to professionalism,23 or the desire to preserve one's reputation by not appearing on a list of poor performers.24 Second, hospitals' desire to appear on lists of high performers, such as U.S. News & World Report's hospital rankings, for marketing purposes, might stimulate improvement activities.10 Finally, publicly reported measures could form the basis for pay‐for‐performance incentives that further speed improvement.25

Our study has several limitations. First, our sample size was small and restricted to 1 region of 1 state, and may not be representative of either the state or nation as a whole. Still, our area has a high level of Internet use, and several local hospitals have been at the vanguard of the quality movement, generally scoring above both state and national averages on Hospital Compare. In addition, Massachusetts has made substantial efforts to promote its own public reporting program, and half the surveyed physicians reported being aware of the Massachusetts Quality and Cost Web site. The fact that not a single area physician surveyed used publicly reported data when making referral decisions is sobering. We believe it is unlikely that other areas of the country will have a substantially higher rate of use. Similarly, our response rate was under 50%. Physicians who did not take the survey may have differed in important ways from those who did. Nevertheless, our sample included a broad range of physician ages, practice types, and affiliations. It seems unlikely that those who did not respond would be more inclined to use publicly reported data than those who did. Second, we assessed decision‐making around a single medical condition. Physicians may have used publicly reported data for other decisions. However, the condition we chose was both urgent (as opposed to emergent) and possesses a robust set of publicly reported quality measures. If physicians do not use publicly reported data for this decision, it seems unlikely they would use it for conditions that have fewer reliable measures (eg, gall bladder surgery) or where the choice of hospital is generally made in an ambulance (eg, myocardial infarction). Finally, the low awareness of public reporting made it difficult for some physicians to answer some of the questions regarding publicly reported hospital quality data because they were unfamiliar with the language utilized by the Web sites (eg, magnet status, Leapfrog never events). It is possible that our results may have been altered slightly if a glossary had been provided.

Despite these limitations, our study suggests that more than 6 years after the launch of the Hospital Quality Alliance, primary care physicians do not appear to make use of these data when choosing a hospital for their patients suffering from pneumonia. Instead, they rely on familiarity with a hospital and past relationships. Even though a majority of the physicians surveyed no longer admitted their own patients, they continue to send patients to hospitals where they had privileges. This finding is not surprising, as physicians also cling to familiar therapies, and may be reluctant to prescribe a new medication or perform an unfamiliar procedure, even if it is indicated. Such reliance on familiarity may make physicians feel comfortable, but does not always result in the best care for patients. Acquiring familiarity, however, requires time and effort, something that physicians generally have in short supply; and while there are plenty of industry representatives to overcome physicians' hesitancy to prescribe new treatments, there are no analogous agents to educate physicians about public reporting or to help them overcome hesitancy about trying a new hospital.

Suspicion about the validity of public reporting may also play a role in the physicians' reported behavior. In past studies of cardiac report cards, cardiologists were most concerned that risk adjustment methods were inadequate (77%) and that mortality rates were an incomplete indicator of the quality of surgical care (74%). They were less concerned about manipulation of data (52%) or small caseloads (15%).10 Our physicians were also concerned that mortality rates were an incomplete measure of quality (76%) but less concerned about risk adjustment (42%), perhaps because many structure and process measures are not subject to risk adjustment. In contrast, they were somewhat more concerned that hospitals could manipulate the data (62%), which again may reflect process measures versus mortality statistics. Other reasons for not using the data may include a lack of awareness of the data or how to access it, or a belief that hospitals do not vary in quality.

Interestingly, even though most respondents were not aware of Hospital Compare, they found the information presented there to best reflect the overall hospital quality. Also, while respondents indicated that they did not use publicly reported data when referring patients, almost half of PCPs reported that publicly reported performance data was at least somewhat important in choosing their own medical care. Thus, although public reporting appears not to have reached its full potential, some publicly reported quality measures have clearly entered the consciousness of PCPs. In contrast, other highly touted measures such as computerized physician order entry systems were not appreciated, and popular designations such as U.S. News & World Report's Best Hospitals were least valued, even though 1 area hospital carries this designation. One conclusion might be that CMS should abandon Hospital Compare since neither patients4 nor providers use it. However, public reporting may improve quality in other ways. Moreover, physicians appear interested in the data even if they are not aware of it. Therefore, given the large investment by CMS and individual hospitals in collecting the data required for Hospital Compare, CMS might consider making greater efforts to increase primary care physician awareness of the Hospital Compare Web site. At the same time, high‐performing hospitals may want to communicate their performance scores to local PCPs as part of their marketing strategy. Future studies could assess whether such practices affect physician referral decisions and subsequent market share of high‐performing hospitals.