Understanding Practice from the Ground Up

Clinicians, researchers, and policymakers now recognize that multiple competing demands1 and opportunities² are simultaneously affecting the physicians, staff, and patients within primary care practices. Our current understanding of outpatient practice is largely based on administrative databases, national surveys, and medical record reviews, with additional insights from surveys of patients or clinicians. These data generally are not designed to capture the richness of the content and context that is needed to better understand the realities and complexities of practice.^3-6 The underlying premise of The Prevention and Competing Demands in Primary Care (P&CD) Study is that efforts to change practice should be preceded by efforts to understand it.^2,7 The explicit goal of this study is to understand practice structure and process, including details of patients, physicians, staff, and clinical encounters; the practice as an organization; and its relationship to the larger community and health system.

In this paper we describe a dynamic observational multimethod assessment process (MAP) that can be used to understand the complex reality of primary care practice. MAP is based on a multimethod comparative case study design^8,9 that integrates elements of epidemiology with methods derived from the qualitative traditions of anthropology and sociology and relies most heavily on qualitative observation and interviewing methods. Studies of this type require an iterative data collection and analysis approach that evolves over time so that new methods can be introduced as the investigators gain a better understanding of important issues. A major strength of our study design was that it allowed hypotheses and insights gained from participants and from ongoing analyses to be integrated into the ongoing investigation.

The study’s primary research questions related to how practice characteristics affect preventive service delivery. Thus, the research design included: (1) an examination of the organizational contexts that support preventive services, (2) an examination of the competing demands imposed by carrying out clinical prevention and illness care in clinical encounters and in the practice, (3) a comparison of the approaches used by practices with high versus low intensity of preventive services delivered to eligible patients, and (4) an examination of approaches used to deliver different types of preventive services. Although the particular focus was on preventive services, the rich MAP allowed pursuit of other research topics that are presented in this issue of JFP.

This article describes the evolutionary methods of the P&CD study, focusing on how data were collected to ensure that sufficient details were available to understand a practice’s values, structures, and processes.*

Emergent research design

The P&CD study was conceived in 1994 to be an in-depth follow-up of insights from the Direct Observation of Primary Care (DOPC) Study that was just getting under way in northeastern Ohio.⁵ The DOPC Study provided a largely quantitative assessment of patients, physicians, encounters, and practices using patient questionnaires, physician surveys, medical record audits, and direct observation of clinical encounters using the Davis Observation Code.¹⁰ That study’s initial findings were presented in the May 1998 theme issue of JFP, and the study processes have recently been described.¹¹ Details of the DOPC methods have been published elsewhere.^4,5,12

Although the initial design allowed the DOPC research nurses to collect brief observational notes, the intensity of the quantitative data collection limited the scope of the study’s qualitative data for understanding details of the practice’s organization and the competing demands within clinical encounters. As a consequence, the P&CD study was designed to provide more in-depth description and understanding of the competing demands of family practice, and in particular, to evaluate factors affecting preventive services delivery using a comparative case study design and a MAP.

A key feature of the P&CD study design was an openness to the integration of emerging insights into the data collection protocol. For example, preliminary analyses of the DOPC data¹³ and other ongoing studies¹⁴ led to the discovery that complexity science was valuable for explaining the dynamics of office systems⁶ and needed to be incorporated into the design. (Complexity science is the study of systems that are characterized by nonlinear dynamics and emergent properties; it emphasizes the need to understand the interrelationships of the whole system and not just collect data about the parts.¹⁵) The investigators also developed new ways to display the relationships among physicians and staff in the practices using “practice genograms.”¹⁶ The practice genogram is a diagram of the functional and interpersonal relationships among the clinicians, support staff, and other people and organizations interacting with the practice. Throughout the project and consistent with the standards of qualitative research design,17 there were continued modifications and enhancements in the data collection and analysis strategies in response to insights that were emerging from ongoing analyses and interpretation of the data.

An important feature of the project was the development of an advisory committee of consultants and co-investigators that convened annually to provide multidisciplinary input, review results, and provide feedback. The advisory committee included academic representatives with expertise in nursing, health education, women’s health, minority health, and public policy. Two additional members were added to the project to provide expertise into the study of organizations as complex systems. The annual reviews by the advisory committee led to significant changes in the research design while the study was ongoing.

Practice Sample

Beginning in late 1996, we drew from 91 practices in Nebraska that had been randomly selected to participate in an earlier study on tobacco prevention and cessation.¹⁸ Initially a sample of 10 practices was purposefully selected^19,20 using an iterative process to represent a range in size (small and large), geographic location (urban, suburban, and rural), and rate of delivery of tobacco-related preventive services. Preliminary analyses of these 10 initial practices provided a summary of preventive health delivery strategies in primary care practices and a description of competing demands that enhanced or limited these strategies. To confirm or refute the emerging insights from the original 10 practices, 8 additional practices were selected for further data collection during the second and third years of the study. The sampling strategy in years 2 and 3 ensured that at least 2 practices each from several major regional hospital health systems were included and allowed us to assess emerging hypotheses about the importance of health system context for understanding community practices.

The practices were recruited by contacting one of the physicians to solicit participation; only those in which all family physicians in the practice agreed to participate were included in the study. Twenty-three practices were contacted; all physicians in 18 agreed to participate (78%).

Core Data Collection Methods

Data were collected by trained field researchers who spent 4 weeks or more taking notes at each practice while observing the practice and clinical encounters, conducting informal key informant interviews of staff, collecting office documents, and auditing charts of patients whose encounters were observed. Within each practice, data collection occurred in stages, with a short break after the initial week or 2 of observation to allow preliminary analyses to inform additional data collection.

Observations at the practice level were recorded in a combination of structured observational checklists, unstructured dictated field notes,^21-23 and key informant interviews.²⁴ Detailed floor plans of the practice were used to identify where particular activities occurred and where individual practice participants worked. Each day at the practice, the field researcher took short notes or “jottings” and dictated expanded field notes in the evening.²³ A template of topics was used periodically to ensure that important aspects of the practice were not being overlooked. The template included lists of features of the community, practice, staff, and patients that the researchers saw as important Figure W1.*¹³ A 3-page structured practice environment checklist was adapted from earlier work on the DOPC project¹¹ and included a wide range of practice characteristics and functions, including items such as the number and training of staff, counseling options offered, and management of telephone calls and referrals Figure W2a. This checklist also served as a detailed reminder to the research nurse of topics to be included in the field notes. Throughout the time the field researchers were recording field notes and filling in the checklist, they opportunistically asked clinicians or staff informal key informant questions for confirmation or clarification.

Consecutive patients for each clinician in a practice were approached with a goal of recruiting 30 patients who would consent to have the field researcher observe their visit. This generally required approaching 35 to 40 patients. Because some clinicians worked part-time or were not consistently in the practice, it was not always possible to observe 30 visits. After explaining the study and gaining signed informed consent from patients who agreed to participate, the field researchers observed the outpatient visit as unobtrusively as possible. A 1-page structured encounter checklist Figure W3 that was also modified from the DOPC study11 provided blanks for noting the reason for the visit, chief complaints, and final diagnoses, and for indicating whether any of approximately 100 preventive services were ordered or delivered. Space was provided at the bottom of the form for recording notes that were later used to dictate a re-creation of the encounter. At a later time, a chart audit was done on each observed patient’s medical record using a structure chart audit form Figure W4.

After the initial observational data were transcribed, a preliminary practice genogram¹⁶ was drawn, and an initial practice summary was written. The genogram of practice participants, roles, and relationships was initially diagrammed on a white board by a transdisciplinary research team by interviewing the field researcher about the current and past practice clinicians and staff and about the health system and community. The demographics of individuals were recorded, including age, sex, years with the practice, percentage of work effort, and job responsibilities. Additional details included functional and emotional relationships observed in the practice, such as who worked together and any obvious conflicts among members of the practice or with health system or community affiliations. This process enabled the investigators to identify areas of incomplete data so the field researchers could return to fill in missing details.

In addition to the key informant interviews done as part of the observation activities, more formal semistructured individual depth interviews were arranged with each clinician and many of the office staff.²⁵ These interviews consisted of a 30-minute to 1-hour narrative interview in which the respondent was asked open-ended questions designed to elicit in-depth responses Figure W5. Although the major focus of these interviews was on the delivery of preventive services, more general questions were included to understand perspectives on practice process. For example, respondents were asked: “Could you describe for me a typical day for you in this practice?” and “If you believed a change was needed regarding some specific delivery of a service within this office, could you describe the process you would go through to try to get it implemented?” These interviews were audiotaped and transcribed verbatim.

To supplement the observational and interview data, field researchers gathered existing documents and artifacts from the practice. Items such as blank charts and flow sheets, patient schedules, personnel lists, samples of patient education materials and handouts, mission and vision statements, and annual reports were collected and compiled in binders. In some practices, particularly those affiliated with hospital health systems, materials were also available from Web sites. All transcribed interviews and dictated field notes from the practice and encounter observations were imported into FolioViews 4.11 (NextPage, Provo, Utah), a text-base management software program that facilitates coding, searching, and retrieval of large computerized text files.²⁶

Emerging Design Decisions

Midway through data collection at the first practice, the advisory committee met to review the data and discuss any concerns. The advisory committee identified a number of emerging hypotheses related to complexity science concepts that were used to guide subsequent data collection and management. It was deemed particularly important to identify “attractors”—factors in the practice and in the larger environment that influenced the structure and function of the practice as an organization.⁶ For example, an attractor might be a particular burning interest of one of the physicians, an expectation of the local hospital systems, or a dominant demographic characteristic of the patients being served.¹³ An expanded systemic model of primary care Figure 1 was articulated that characterized 6 core areas for data collection: (1) patient perceptions and behavior, (2) physician perceptions and behavior, (3) encounter structures and processes, (4) practice structures and processes, (5) community characteristics, and (6) the larger health system. This model identified the need for additional data on the community context and patient experience. Checklists were revised and field researchers were asked to spend more time gathering data about the community. It was also apparent that accurate calculation of certain preventive service delivery rates would require patient input and a larger sample size. For example, a patient exit card

Figure W6 was developed to ascertain self-reported tobacco use status (for all patients) and use of obstetrics and gynecology services and history of hysterectomy (for women). These data were used to determine a patient’s eligibility for tobacco cessation counseling and Papanicolaou tests, respectively. Observation of 30 encounters with each clinician was done to increase the stability for calculating rates for common preventive service recommendations.

A larger issue emerged with the discovery that, after contributing data over the course of weeks or months, members of the practices desired feedback in a timely manner. Practice clinicians and staff were very interested in how they were doing and asked when they would be receiving a report of our results. They did not want to wait 3 years for the completion of the study. Although ongoing analyses were anticipated, these had primarily been designed to ensure completeness of data and to provide feedback to field researchers on areas where clarifications were needed. In response to the emergent desire for feedback, the team generated rapid-turnaround summary reports for each practice. A summary report template was designed to present the descriptive details of the practice, including the practice genogram and a summary of the strengths and weaknesses of the practice’s prevention approach. The final page in the report provided the practice with a series of questions or points for self-reflection that often included process questions, such as “How can this organization become a team?” or “How can this practice deliver preventive services more consistently?” These reports were shared interactively with practices at a debriefing meeting within 2 to 3 months of completing data collection at each practice.

The feedback meetings provided an important opportunity to check the validity of the researchers’ analyses by comparing them with the perspectives of the practice participants. In all the practices, the response was a strong overall validation of the research team’s interpretation of the practice and its structures and processes. During the feedback presentations, the practice physicians and staff consistently made comments like, “Wow, did you ever get us,” or “This is like looking in a mirror.” In a number of sessions, the participants mentioned that the report raised issues about which they were vaguely aware and that the findings were stimulating considerable self-reflection. In several practices, the physicians disclosed that they would be taking actions in the future to modify some of the deficiencies the reports uncovered.

The next modification came as the data were being collected simultaneously in the second and third practices. We realized that despite our efforts to be nondisruptive, participating in the project required extra effort on the part of the practice. Each practice therefore received partial compensation in the form of a $500 certificate for the purchase of books or equipment.

After completing data collection at several practices, it was apparent that patients’ perspectives were still under-represented and that this was limiting the understanding of the practice. To gain further insights into patients’ experiences, beginning with the sixth practice we adopted the patient path approach described by Pommerenke and Dietrich,²⁷ following patients from the time they walk into the practice until the time they leave, using a patient path form for recording activities at different stages of the visit Figure W7. Additional brief open-ended interviews were conducted in the waiting room or examination rooms while patients were waiting.

Although we had asked the research nurses to be more thorough in their descriptions of the community, data from the community and health system were still incomplete. This became even more pronounced when studying practices that were part of health systems. Therefore, beginning with the sixth practice, we included in-depth interviews with individuals from health systems (eg, regional managers and medical directors). A further refinement came with the use of community key informant trees, a systematic process of identifying and interviewing members of the local community surrounding the practice.^24,28 These interviews of patients, church leaders, and other individuals from the community began with the ninth practice.

Once all the modifications were incorporated, the final case study design provided data at each of the 6 levels as shown in the Table 1. Particularly detailed data were available at the clinician, encounter, and practice levels. For example, at the clinician level the data included perceptions of roles as ascertained through the in-depth interviews, as well as actual behaviors recorded in the encounter field notes and chart audits. Insights on the structures and process of the practice were obtained through unstructured observations of the practice, structured checklists, written documents, and interviews. Supporting data were collected on patients’ perspectives, the community, and the health system that provided contexts for the practice case studies.

Discussion

The complexity of primary care practices is best understood from multiple perspectives,²⁹ a principle that guided the initial selection of a multimethod comparative case study design for this investigation. The MAP that emerged from this comparative case study design has a number of strengths and weaknesses. A particular asset of the design was our ability to investigate specific phenomena within their context rather than isolated from it.²⁹ The design also encouraged the investigators to pursue emerging insights, thus informing multiple perspectives that might not have otherwise been considered, although this may be somewhat limited by the purposeful sampling strategy that focuses on maximizing information about a particular topic.

Limitations

A limitation for broader implementation of this research design is the intensity of data collection and analysis, which are difficult to accomplish without considerable resources and a research team with diverse skills. There might also be some concerns about the ways that the data collection process alters practice behavior; however, the prolonged observational time frame and the multiple data sources for “triangulation” are designed to limit any potential Hawthorne effect. That is, by collecting and systematically comparing data from multiple sources, including direct observation, different forms of interviews, and existing documents, the investigators were able to identify inconsistencies in patterns of behavior.^17,19

Although the data collection, analysis, and feedback process appeared to increase a practice’s self-reflection, our study limited the input of patient and practice participants in the design, analysis, and interpretation and thus does not approach the participatory research paradigm espoused by McCauley and colleagues.³⁰ Still, our study moved from being primarily observational descriptive research into a more collaborative and interventional project, in part at the request of the participants. This suggests a model and method for future research in the arena of health care process and outcome improvement with the practices as collaborators. The MAP characterized in this paper offers a means for simultaneously describing, understanding, and improving the richly complex and varied processes and outcomes of primary care. By more actively engaging the practices in the research process, the MAP also points toward a new more collaborative model of practice-based research.

Conclusions

The comprehensive data in the P&CD study provide a unique opportunity to understand and describe multiple perspectives from the clinician, patient, encounter, practice, community, and health system spheres. Each of the papers in this issue of JFP used some of these comprehensive data to study one or more of these spheres. For example, the encounter field notes were the primary source of data for exploring how “family” presents in encounters³¹Several of the authors used subsets of patients, including patients presenting with acute respiratory track infections,³³ smokers,³⁴ and frequent attenders.³⁵ These authors each supplemented the encounter field notes with data from the medical record reviews, medical record field notes, and patient exit cards. The complete data set including practice field notes, practice genograms, physician and staff interviews, office environment checklists, and encounter field notes were used to describe staff training, roles, and functions.³⁶ This is only a part of the research adventure available in this type of data. We hope many others will join in the excitement.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Research Center grant from the American Academy of Family Physicians. Drs Crabtree, Miller and Stange are associated with the Center for Research in Family Practice and Primary Care, Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 18 practices, without whose participation this study would not have been possible. We also wish to thank Connie Gibbs and Jen Rouse, who spent many hours in the practices collecting data, and Diane Dodendorf and Jason Lebsack, who spent countless hours coordinating transcription and data management activities, for their dedicated work. We are especially indebted to Mary McAndrews, who transcribed hundreds of taped interviews and dictated field notes. The ongoing analyses that ensured the quality and comprehensiveness of the data were made possible through the dedicated work of Helen McIlvain, PhD; Jeffrey Susman, MD; Virginia Aita, PhD; Kristine McVea, MD; Elisabeth Backer, MD; Paul Turner, PhD; and Louis Pol, PhD. Finally, we thank the members of the advisory committee: Valerie Gilchrist, MD; Paul Nutting, MD, MPH; Carlos Jaén, MD, PhD; Kurt Stange, MD, PhD; William Miller, MD, MA; Reuben McDaniel, PhD; and Ruth Anderson, RN, PhD.

Clinicians, researchers, and policymakers now recognize that multiple competing demands1 and opportunities² are simultaneously affecting the physicians, staff, and patients within primary care practices. Our current understanding of outpatient practice is largely based on administrative databases, national surveys, and medical record reviews, with additional insights from surveys of patients or clinicians. These data generally are not designed to capture the richness of the content and context that is needed to better understand the realities and complexities of practice.^3-6 The underlying premise of The Prevention and Competing Demands in Primary Care (P&CD) Study is that efforts to change practice should be preceded by efforts to understand it.^2,7 The explicit goal of this study is to understand practice structure and process, including details of patients, physicians, staff, and clinical encounters; the practice as an organization; and its relationship to the larger community and health system.

In this paper we describe a dynamic observational multimethod assessment process (MAP) that can be used to understand the complex reality of primary care practice. MAP is based on a multimethod comparative case study design^8,9 that integrates elements of epidemiology with methods derived from the qualitative traditions of anthropology and sociology and relies most heavily on qualitative observation and interviewing methods. Studies of this type require an iterative data collection and analysis approach that evolves over time so that new methods can be introduced as the investigators gain a better understanding of important issues. A major strength of our study design was that it allowed hypotheses and insights gained from participants and from ongoing analyses to be integrated into the ongoing investigation.

The study’s primary research questions related to how practice characteristics affect preventive service delivery. Thus, the research design included: (1) an examination of the organizational contexts that support preventive services, (2) an examination of the competing demands imposed by carrying out clinical prevention and illness care in clinical encounters and in the practice, (3) a comparison of the approaches used by practices with high versus low intensity of preventive services delivered to eligible patients, and (4) an examination of approaches used to deliver different types of preventive services. Although the particular focus was on preventive services, the rich MAP allowed pursuit of other research topics that are presented in this issue of JFP.

This article describes the evolutionary methods of the P&CD study, focusing on how data were collected to ensure that sufficient details were available to understand a practice’s values, structures, and processes.*

Emergent research design

The P&CD study was conceived in 1994 to be an in-depth follow-up of insights from the Direct Observation of Primary Care (DOPC) Study that was just getting under way in northeastern Ohio.⁵ The DOPC Study provided a largely quantitative assessment of patients, physicians, encounters, and practices using patient questionnaires, physician surveys, medical record audits, and direct observation of clinical encounters using the Davis Observation Code.¹⁰ That study’s initial findings were presented in the May 1998 theme issue of JFP, and the study processes have recently been described.¹¹ Details of the DOPC methods have been published elsewhere.^4,5,12

Although the initial design allowed the DOPC research nurses to collect brief observational notes, the intensity of the quantitative data collection limited the scope of the study’s qualitative data for understanding details of the practice’s organization and the competing demands within clinical encounters. As a consequence, the P&CD study was designed to provide more in-depth description and understanding of the competing demands of family practice, and in particular, to evaluate factors affecting preventive services delivery using a comparative case study design and a MAP.

A key feature of the P&CD study design was an openness to the integration of emerging insights into the data collection protocol. For example, preliminary analyses of the DOPC data¹³ and other ongoing studies¹⁴ led to the discovery that complexity science was valuable for explaining the dynamics of office systems⁶ and needed to be incorporated into the design. (Complexity science is the study of systems that are characterized by nonlinear dynamics and emergent properties; it emphasizes the need to understand the interrelationships of the whole system and not just collect data about the parts.¹⁵) The investigators also developed new ways to display the relationships among physicians and staff in the practices using “practice genograms.”¹⁶ The practice genogram is a diagram of the functional and interpersonal relationships among the clinicians, support staff, and other people and organizations interacting with the practice. Throughout the project and consistent with the standards of qualitative research design,17 there were continued modifications and enhancements in the data collection and analysis strategies in response to insights that were emerging from ongoing analyses and interpretation of the data.

An important feature of the project was the development of an advisory committee of consultants and co-investigators that convened annually to provide multidisciplinary input, review results, and provide feedback. The advisory committee included academic representatives with expertise in nursing, health education, women’s health, minority health, and public policy. Two additional members were added to the project to provide expertise into the study of organizations as complex systems. The annual reviews by the advisory committee led to significant changes in the research design while the study was ongoing.

Practice Sample

Beginning in late 1996, we drew from 91 practices in Nebraska that had been randomly selected to participate in an earlier study on tobacco prevention and cessation.¹⁸ Initially a sample of 10 practices was purposefully selected^19,20 using an iterative process to represent a range in size (small and large), geographic location (urban, suburban, and rural), and rate of delivery of tobacco-related preventive services. Preliminary analyses of these 10 initial practices provided a summary of preventive health delivery strategies in primary care practices and a description of competing demands that enhanced or limited these strategies. To confirm or refute the emerging insights from the original 10 practices, 8 additional practices were selected for further data collection during the second and third years of the study. The sampling strategy in years 2 and 3 ensured that at least 2 practices each from several major regional hospital health systems were included and allowed us to assess emerging hypotheses about the importance of health system context for understanding community practices.

The practices were recruited by contacting one of the physicians to solicit participation; only those in which all family physicians in the practice agreed to participate were included in the study. Twenty-three practices were contacted; all physicians in 18 agreed to participate (78%).

Core Data Collection Methods

Data were collected by trained field researchers who spent 4 weeks or more taking notes at each practice while observing the practice and clinical encounters, conducting informal key informant interviews of staff, collecting office documents, and auditing charts of patients whose encounters were observed. Within each practice, data collection occurred in stages, with a short break after the initial week or 2 of observation to allow preliminary analyses to inform additional data collection.

Observations at the practice level were recorded in a combination of structured observational checklists, unstructured dictated field notes,^21-23 and key informant interviews.²⁴ Detailed floor plans of the practice were used to identify where particular activities occurred and where individual practice participants worked. Each day at the practice, the field researcher took short notes or “jottings” and dictated expanded field notes in the evening.²³ A template of topics was used periodically to ensure that important aspects of the practice were not being overlooked. The template included lists of features of the community, practice, staff, and patients that the researchers saw as important Figure W1.*¹³ A 3-page structured practice environment checklist was adapted from earlier work on the DOPC project¹¹ and included a wide range of practice characteristics and functions, including items such as the number and training of staff, counseling options offered, and management of telephone calls and referrals Figure W2a. This checklist also served as a detailed reminder to the research nurse of topics to be included in the field notes. Throughout the time the field researchers were recording field notes and filling in the checklist, they opportunistically asked clinicians or staff informal key informant questions for confirmation or clarification.

Consecutive patients for each clinician in a practice were approached with a goal of recruiting 30 patients who would consent to have the field researcher observe their visit. This generally required approaching 35 to 40 patients. Because some clinicians worked part-time or were not consistently in the practice, it was not always possible to observe 30 visits. After explaining the study and gaining signed informed consent from patients who agreed to participate, the field researchers observed the outpatient visit as unobtrusively as possible. A 1-page structured encounter checklist Figure W3 that was also modified from the DOPC study11 provided blanks for noting the reason for the visit, chief complaints, and final diagnoses, and for indicating whether any of approximately 100 preventive services were ordered or delivered. Space was provided at the bottom of the form for recording notes that were later used to dictate a re-creation of the encounter. At a later time, a chart audit was done on each observed patient’s medical record using a structure chart audit form Figure W4.

After the initial observational data were transcribed, a preliminary practice genogram¹⁶ was drawn, and an initial practice summary was written. The genogram of practice participants, roles, and relationships was initially diagrammed on a white board by a transdisciplinary research team by interviewing the field researcher about the current and past practice clinicians and staff and about the health system and community. The demographics of individuals were recorded, including age, sex, years with the practice, percentage of work effort, and job responsibilities. Additional details included functional and emotional relationships observed in the practice, such as who worked together and any obvious conflicts among members of the practice or with health system or community affiliations. This process enabled the investigators to identify areas of incomplete data so the field researchers could return to fill in missing details.

In addition to the key informant interviews done as part of the observation activities, more formal semistructured individual depth interviews were arranged with each clinician and many of the office staff.²⁵ These interviews consisted of a 30-minute to 1-hour narrative interview in which the respondent was asked open-ended questions designed to elicit in-depth responses Figure W5. Although the major focus of these interviews was on the delivery of preventive services, more general questions were included to understand perspectives on practice process. For example, respondents were asked: “Could you describe for me a typical day for you in this practice?” and “If you believed a change was needed regarding some specific delivery of a service within this office, could you describe the process you would go through to try to get it implemented?” These interviews were audiotaped and transcribed verbatim.

To supplement the observational and interview data, field researchers gathered existing documents and artifacts from the practice. Items such as blank charts and flow sheets, patient schedules, personnel lists, samples of patient education materials and handouts, mission and vision statements, and annual reports were collected and compiled in binders. In some practices, particularly those affiliated with hospital health systems, materials were also available from Web sites. All transcribed interviews and dictated field notes from the practice and encounter observations were imported into FolioViews 4.11 (NextPage, Provo, Utah), a text-base management software program that facilitates coding, searching, and retrieval of large computerized text files.²⁶

Emerging Design Decisions

Midway through data collection at the first practice, the advisory committee met to review the data and discuss any concerns. The advisory committee identified a number of emerging hypotheses related to complexity science concepts that were used to guide subsequent data collection and management. It was deemed particularly important to identify “attractors”—factors in the practice and in the larger environment that influenced the structure and function of the practice as an organization.⁶ For example, an attractor might be a particular burning interest of one of the physicians, an expectation of the local hospital systems, or a dominant demographic characteristic of the patients being served.¹³ An expanded systemic model of primary care Figure 1 was articulated that characterized 6 core areas for data collection: (1) patient perceptions and behavior, (2) physician perceptions and behavior, (3) encounter structures and processes, (4) practice structures and processes, (5) community characteristics, and (6) the larger health system. This model identified the need for additional data on the community context and patient experience. Checklists were revised and field researchers were asked to spend more time gathering data about the community. It was also apparent that accurate calculation of certain preventive service delivery rates would require patient input and a larger sample size. For example, a patient exit card

Figure W6 was developed to ascertain self-reported tobacco use status (for all patients) and use of obstetrics and gynecology services and history of hysterectomy (for women). These data were used to determine a patient’s eligibility for tobacco cessation counseling and Papanicolaou tests, respectively. Observation of 30 encounters with each clinician was done to increase the stability for calculating rates for common preventive service recommendations.

A larger issue emerged with the discovery that, after contributing data over the course of weeks or months, members of the practices desired feedback in a timely manner. Practice clinicians and staff were very interested in how they were doing and asked when they would be receiving a report of our results. They did not want to wait 3 years for the completion of the study. Although ongoing analyses were anticipated, these had primarily been designed to ensure completeness of data and to provide feedback to field researchers on areas where clarifications were needed. In response to the emergent desire for feedback, the team generated rapid-turnaround summary reports for each practice. A summary report template was designed to present the descriptive details of the practice, including the practice genogram and a summary of the strengths and weaknesses of the practice’s prevention approach. The final page in the report provided the practice with a series of questions or points for self-reflection that often included process questions, such as “How can this organization become a team?” or “How can this practice deliver preventive services more consistently?” These reports were shared interactively with practices at a debriefing meeting within 2 to 3 months of completing data collection at each practice.

The feedback meetings provided an important opportunity to check the validity of the researchers’ analyses by comparing them with the perspectives of the practice participants. In all the practices, the response was a strong overall validation of the research team’s interpretation of the practice and its structures and processes. During the feedback presentations, the practice physicians and staff consistently made comments like, “Wow, did you ever get us,” or “This is like looking in a mirror.” In a number of sessions, the participants mentioned that the report raised issues about which they were vaguely aware and that the findings were stimulating considerable self-reflection. In several practices, the physicians disclosed that they would be taking actions in the future to modify some of the deficiencies the reports uncovered.

The next modification came as the data were being collected simultaneously in the second and third practices. We realized that despite our efforts to be nondisruptive, participating in the project required extra effort on the part of the practice. Each practice therefore received partial compensation in the form of a $500 certificate for the purchase of books or equipment.

After completing data collection at several practices, it was apparent that patients’ perspectives were still under-represented and that this was limiting the understanding of the practice. To gain further insights into patients’ experiences, beginning with the sixth practice we adopted the patient path approach described by Pommerenke and Dietrich,²⁷ following patients from the time they walk into the practice until the time they leave, using a patient path form for recording activities at different stages of the visit Figure W7. Additional brief open-ended interviews were conducted in the waiting room or examination rooms while patients were waiting.

Although we had asked the research nurses to be more thorough in their descriptions of the community, data from the community and health system were still incomplete. This became even more pronounced when studying practices that were part of health systems. Therefore, beginning with the sixth practice, we included in-depth interviews with individuals from health systems (eg, regional managers and medical directors). A further refinement came with the use of community key informant trees, a systematic process of identifying and interviewing members of the local community surrounding the practice.^24,28 These interviews of patients, church leaders, and other individuals from the community began with the ninth practice.

Once all the modifications were incorporated, the final case study design provided data at each of the 6 levels as shown in the Table 1. Particularly detailed data were available at the clinician, encounter, and practice levels. For example, at the clinician level the data included perceptions of roles as ascertained through the in-depth interviews, as well as actual behaviors recorded in the encounter field notes and chart audits. Insights on the structures and process of the practice were obtained through unstructured observations of the practice, structured checklists, written documents, and interviews. Supporting data were collected on patients’ perspectives, the community, and the health system that provided contexts for the practice case studies.

Discussion

The complexity of primary care practices is best understood from multiple perspectives,²⁹ a principle that guided the initial selection of a multimethod comparative case study design for this investigation. The MAP that emerged from this comparative case study design has a number of strengths and weaknesses. A particular asset of the design was our ability to investigate specific phenomena within their context rather than isolated from it.²⁹ The design also encouraged the investigators to pursue emerging insights, thus informing multiple perspectives that might not have otherwise been considered, although this may be somewhat limited by the purposeful sampling strategy that focuses on maximizing information about a particular topic.

Limitations

A limitation for broader implementation of this research design is the intensity of data collection and analysis, which are difficult to accomplish without considerable resources and a research team with diverse skills. There might also be some concerns about the ways that the data collection process alters practice behavior; however, the prolonged observational time frame and the multiple data sources for “triangulation” are designed to limit any potential Hawthorne effect. That is, by collecting and systematically comparing data from multiple sources, including direct observation, different forms of interviews, and existing documents, the investigators were able to identify inconsistencies in patterns of behavior.^17,19

Although the data collection, analysis, and feedback process appeared to increase a practice’s self-reflection, our study limited the input of patient and practice participants in the design, analysis, and interpretation and thus does not approach the participatory research paradigm espoused by McCauley and colleagues.³⁰ Still, our study moved from being primarily observational descriptive research into a more collaborative and interventional project, in part at the request of the participants. This suggests a model and method for future research in the arena of health care process and outcome improvement with the practices as collaborators. The MAP characterized in this paper offers a means for simultaneously describing, understanding, and improving the richly complex and varied processes and outcomes of primary care. By more actively engaging the practices in the research process, the MAP also points toward a new more collaborative model of practice-based research.

Conclusions

The comprehensive data in the P&CD study provide a unique opportunity to understand and describe multiple perspectives from the clinician, patient, encounter, practice, community, and health system spheres. Each of the papers in this issue of JFP used some of these comprehensive data to study one or more of these spheres. For example, the encounter field notes were the primary source of data for exploring how “family” presents in encounters³¹Several of the authors used subsets of patients, including patients presenting with acute respiratory track infections,³³ smokers,³⁴ and frequent attenders.³⁵ These authors each supplemented the encounter field notes with data from the medical record reviews, medical record field notes, and patient exit cards. The complete data set including practice field notes, practice genograms, physician and staff interviews, office environment checklists, and encounter field notes were used to describe staff training, roles, and functions.³⁶ This is only a part of the research adventure available in this type of data. We hope many others will join in the excitement.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Research Center grant from the American Academy of Family Physicians. Drs Crabtree, Miller and Stange are associated with the Center for Research in Family Practice and Primary Care, Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 18 practices, without whose participation this study would not have been possible. We also wish to thank Connie Gibbs and Jen Rouse, who spent many hours in the practices collecting data, and Diane Dodendorf and Jason Lebsack, who spent countless hours coordinating transcription and data management activities, for their dedicated work. We are especially indebted to Mary McAndrews, who transcribed hundreds of taped interviews and dictated field notes. The ongoing analyses that ensured the quality and comprehensiveness of the data were made possible through the dedicated work of Helen McIlvain, PhD; Jeffrey Susman, MD; Virginia Aita, PhD; Kristine McVea, MD; Elisabeth Backer, MD; Paul Turner, PhD; and Louis Pol, PhD. Finally, we thank the members of the advisory committee: Valerie Gilchrist, MD; Paul Nutting, MD, MPH; Carlos Jaén, MD, PhD; Kurt Stange, MD, PhD; William Miller, MD, MA; Reuben McDaniel, PhD; and Ruth Anderson, RN, PhD.

Clinicians, researchers, and policymakers now recognize that multiple competing demands1 and opportunities² are simultaneously affecting the physicians, staff, and patients within primary care practices. Our current understanding of outpatient practice is largely based on administrative databases, national surveys, and medical record reviews, with additional insights from surveys of patients or clinicians. These data generally are not designed to capture the richness of the content and context that is needed to better understand the realities and complexities of practice.^3-6 The underlying premise of The Prevention and Competing Demands in Primary Care (P&CD) Study is that efforts to change practice should be preceded by efforts to understand it.^2,7 The explicit goal of this study is to understand practice structure and process, including details of patients, physicians, staff, and clinical encounters; the practice as an organization; and its relationship to the larger community and health system.

In this paper we describe a dynamic observational multimethod assessment process (MAP) that can be used to understand the complex reality of primary care practice. MAP is based on a multimethod comparative case study design^8,9 that integrates elements of epidemiology with methods derived from the qualitative traditions of anthropology and sociology and relies most heavily on qualitative observation and interviewing methods. Studies of this type require an iterative data collection and analysis approach that evolves over time so that new methods can be introduced as the investigators gain a better understanding of important issues. A major strength of our study design was that it allowed hypotheses and insights gained from participants and from ongoing analyses to be integrated into the ongoing investigation.

The study’s primary research questions related to how practice characteristics affect preventive service delivery. Thus, the research design included: (1) an examination of the organizational contexts that support preventive services, (2) an examination of the competing demands imposed by carrying out clinical prevention and illness care in clinical encounters and in the practice, (3) a comparison of the approaches used by practices with high versus low intensity of preventive services delivered to eligible patients, and (4) an examination of approaches used to deliver different types of preventive services. Although the particular focus was on preventive services, the rich MAP allowed pursuit of other research topics that are presented in this issue of JFP.

This article describes the evolutionary methods of the P&CD study, focusing on how data were collected to ensure that sufficient details were available to understand a practice’s values, structures, and processes.*

Emergent research design

The P&CD study was conceived in 1994 to be an in-depth follow-up of insights from the Direct Observation of Primary Care (DOPC) Study that was just getting under way in northeastern Ohio.⁵ The DOPC Study provided a largely quantitative assessment of patients, physicians, encounters, and practices using patient questionnaires, physician surveys, medical record audits, and direct observation of clinical encounters using the Davis Observation Code.¹⁰ That study’s initial findings were presented in the May 1998 theme issue of JFP, and the study processes have recently been described.¹¹ Details of the DOPC methods have been published elsewhere.^4,5,12

Although the initial design allowed the DOPC research nurses to collect brief observational notes, the intensity of the quantitative data collection limited the scope of the study’s qualitative data for understanding details of the practice’s organization and the competing demands within clinical encounters. As a consequence, the P&CD study was designed to provide more in-depth description and understanding of the competing demands of family practice, and in particular, to evaluate factors affecting preventive services delivery using a comparative case study design and a MAP.

A key feature of the P&CD study design was an openness to the integration of emerging insights into the data collection protocol. For example, preliminary analyses of the DOPC data¹³ and other ongoing studies¹⁴ led to the discovery that complexity science was valuable for explaining the dynamics of office systems⁶ and needed to be incorporated into the design. (Complexity science is the study of systems that are characterized by nonlinear dynamics and emergent properties; it emphasizes the need to understand the interrelationships of the whole system and not just collect data about the parts.¹⁵) The investigators also developed new ways to display the relationships among physicians and staff in the practices using “practice genograms.”¹⁶ The practice genogram is a diagram of the functional and interpersonal relationships among the clinicians, support staff, and other people and organizations interacting with the practice. Throughout the project and consistent with the standards of qualitative research design,17 there were continued modifications and enhancements in the data collection and analysis strategies in response to insights that were emerging from ongoing analyses and interpretation of the data.

An important feature of the project was the development of an advisory committee of consultants and co-investigators that convened annually to provide multidisciplinary input, review results, and provide feedback. The advisory committee included academic representatives with expertise in nursing, health education, women’s health, minority health, and public policy. Two additional members were added to the project to provide expertise into the study of organizations as complex systems. The annual reviews by the advisory committee led to significant changes in the research design while the study was ongoing.

Practice Sample

Beginning in late 1996, we drew from 91 practices in Nebraska that had been randomly selected to participate in an earlier study on tobacco prevention and cessation.¹⁸ Initially a sample of 10 practices was purposefully selected^19,20 using an iterative process to represent a range in size (small and large), geographic location (urban, suburban, and rural), and rate of delivery of tobacco-related preventive services. Preliminary analyses of these 10 initial practices provided a summary of preventive health delivery strategies in primary care practices and a description of competing demands that enhanced or limited these strategies. To confirm or refute the emerging insights from the original 10 practices, 8 additional practices were selected for further data collection during the second and third years of the study. The sampling strategy in years 2 and 3 ensured that at least 2 practices each from several major regional hospital health systems were included and allowed us to assess emerging hypotheses about the importance of health system context for understanding community practices.

The practices were recruited by contacting one of the physicians to solicit participation; only those in which all family physicians in the practice agreed to participate were included in the study. Twenty-three practices were contacted; all physicians in 18 agreed to participate (78%).

Core Data Collection Methods

Data were collected by trained field researchers who spent 4 weeks or more taking notes at each practice while observing the practice and clinical encounters, conducting informal key informant interviews of staff, collecting office documents, and auditing charts of patients whose encounters were observed. Within each practice, data collection occurred in stages, with a short break after the initial week or 2 of observation to allow preliminary analyses to inform additional data collection.

Observations at the practice level were recorded in a combination of structured observational checklists, unstructured dictated field notes,^21-23 and key informant interviews.²⁴ Detailed floor plans of the practice were used to identify where particular activities occurred and where individual practice participants worked. Each day at the practice, the field researcher took short notes or “jottings” and dictated expanded field notes in the evening.²³ A template of topics was used periodically to ensure that important aspects of the practice were not being overlooked. The template included lists of features of the community, practice, staff, and patients that the researchers saw as important Figure W1.*¹³ A 3-page structured practice environment checklist was adapted from earlier work on the DOPC project¹¹ and included a wide range of practice characteristics and functions, including items such as the number and training of staff, counseling options offered, and management of telephone calls and referrals Figure W2a. This checklist also served as a detailed reminder to the research nurse of topics to be included in the field notes. Throughout the time the field researchers were recording field notes and filling in the checklist, they opportunistically asked clinicians or staff informal key informant questions for confirmation or clarification.

Consecutive patients for each clinician in a practice were approached with a goal of recruiting 30 patients who would consent to have the field researcher observe their visit. This generally required approaching 35 to 40 patients. Because some clinicians worked part-time or were not consistently in the practice, it was not always possible to observe 30 visits. After explaining the study and gaining signed informed consent from patients who agreed to participate, the field researchers observed the outpatient visit as unobtrusively as possible. A 1-page structured encounter checklist Figure W3 that was also modified from the DOPC study11 provided blanks for noting the reason for the visit, chief complaints, and final diagnoses, and for indicating whether any of approximately 100 preventive services were ordered or delivered. Space was provided at the bottom of the form for recording notes that were later used to dictate a re-creation of the encounter. At a later time, a chart audit was done on each observed patient’s medical record using a structure chart audit form Figure W4.

After the initial observational data were transcribed, a preliminary practice genogram¹⁶ was drawn, and an initial practice summary was written. The genogram of practice participants, roles, and relationships was initially diagrammed on a white board by a transdisciplinary research team by interviewing the field researcher about the current and past practice clinicians and staff and about the health system and community. The demographics of individuals were recorded, including age, sex, years with the practice, percentage of work effort, and job responsibilities. Additional details included functional and emotional relationships observed in the practice, such as who worked together and any obvious conflicts among members of the practice or with health system or community affiliations. This process enabled the investigators to identify areas of incomplete data so the field researchers could return to fill in missing details.

In addition to the key informant interviews done as part of the observation activities, more formal semistructured individual depth interviews were arranged with each clinician and many of the office staff.²⁵ These interviews consisted of a 30-minute to 1-hour narrative interview in which the respondent was asked open-ended questions designed to elicit in-depth responses Figure W5. Although the major focus of these interviews was on the delivery of preventive services, more general questions were included to understand perspectives on practice process. For example, respondents were asked: “Could you describe for me a typical day for you in this practice?” and “If you believed a change was needed regarding some specific delivery of a service within this office, could you describe the process you would go through to try to get it implemented?” These interviews were audiotaped and transcribed verbatim.

To supplement the observational and interview data, field researchers gathered existing documents and artifacts from the practice. Items such as blank charts and flow sheets, patient schedules, personnel lists, samples of patient education materials and handouts, mission and vision statements, and annual reports were collected and compiled in binders. In some practices, particularly those affiliated with hospital health systems, materials were also available from Web sites. All transcribed interviews and dictated field notes from the practice and encounter observations were imported into FolioViews 4.11 (NextPage, Provo, Utah), a text-base management software program that facilitates coding, searching, and retrieval of large computerized text files.²⁶

Emerging Design Decisions

Midway through data collection at the first practice, the advisory committee met to review the data and discuss any concerns. The advisory committee identified a number of emerging hypotheses related to complexity science concepts that were used to guide subsequent data collection and management. It was deemed particularly important to identify “attractors”—factors in the practice and in the larger environment that influenced the structure and function of the practice as an organization.⁶ For example, an attractor might be a particular burning interest of one of the physicians, an expectation of the local hospital systems, or a dominant demographic characteristic of the patients being served.¹³ An expanded systemic model of primary care Figure 1 was articulated that characterized 6 core areas for data collection: (1) patient perceptions and behavior, (2) physician perceptions and behavior, (3) encounter structures and processes, (4) practice structures and processes, (5) community characteristics, and (6) the larger health system. This model identified the need for additional data on the community context and patient experience. Checklists were revised and field researchers were asked to spend more time gathering data about the community. It was also apparent that accurate calculation of certain preventive service delivery rates would require patient input and a larger sample size. For example, a patient exit card

Figure W6 was developed to ascertain self-reported tobacco use status (for all patients) and use of obstetrics and gynecology services and history of hysterectomy (for women). These data were used to determine a patient’s eligibility for tobacco cessation counseling and Papanicolaou tests, respectively. Observation of 30 encounters with each clinician was done to increase the stability for calculating rates for common preventive service recommendations.

A larger issue emerged with the discovery that, after contributing data over the course of weeks or months, members of the practices desired feedback in a timely manner. Practice clinicians and staff were very interested in how they were doing and asked when they would be receiving a report of our results. They did not want to wait 3 years for the completion of the study. Although ongoing analyses were anticipated, these had primarily been designed to ensure completeness of data and to provide feedback to field researchers on areas where clarifications were needed. In response to the emergent desire for feedback, the team generated rapid-turnaround summary reports for each practice. A summary report template was designed to present the descriptive details of the practice, including the practice genogram and a summary of the strengths and weaknesses of the practice’s prevention approach. The final page in the report provided the practice with a series of questions or points for self-reflection that often included process questions, such as “How can this organization become a team?” or “How can this practice deliver preventive services more consistently?” These reports were shared interactively with practices at a debriefing meeting within 2 to 3 months of completing data collection at each practice.

The feedback meetings provided an important opportunity to check the validity of the researchers’ analyses by comparing them with the perspectives of the practice participants. In all the practices, the response was a strong overall validation of the research team’s interpretation of the practice and its structures and processes. During the feedback presentations, the practice physicians and staff consistently made comments like, “Wow, did you ever get us,” or “This is like looking in a mirror.” In a number of sessions, the participants mentioned that the report raised issues about which they were vaguely aware and that the findings were stimulating considerable self-reflection. In several practices, the physicians disclosed that they would be taking actions in the future to modify some of the deficiencies the reports uncovered.

The next modification came as the data were being collected simultaneously in the second and third practices. We realized that despite our efforts to be nondisruptive, participating in the project required extra effort on the part of the practice. Each practice therefore received partial compensation in the form of a $500 certificate for the purchase of books or equipment.

After completing data collection at several practices, it was apparent that patients’ perspectives were still under-represented and that this was limiting the understanding of the practice. To gain further insights into patients’ experiences, beginning with the sixth practice we adopted the patient path approach described by Pommerenke and Dietrich,²⁷ following patients from the time they walk into the practice until the time they leave, using a patient path form for recording activities at different stages of the visit Figure W7. Additional brief open-ended interviews were conducted in the waiting room or examination rooms while patients were waiting.

Although we had asked the research nurses to be more thorough in their descriptions of the community, data from the community and health system were still incomplete. This became even more pronounced when studying practices that were part of health systems. Therefore, beginning with the sixth practice, we included in-depth interviews with individuals from health systems (eg, regional managers and medical directors). A further refinement came with the use of community key informant trees, a systematic process of identifying and interviewing members of the local community surrounding the practice.^24,28 These interviews of patients, church leaders, and other individuals from the community began with the ninth practice.

Once all the modifications were incorporated, the final case study design provided data at each of the 6 levels as shown in the Table 1. Particularly detailed data were available at the clinician, encounter, and practice levels. For example, at the clinician level the data included perceptions of roles as ascertained through the in-depth interviews, as well as actual behaviors recorded in the encounter field notes and chart audits. Insights on the structures and process of the practice were obtained through unstructured observations of the practice, structured checklists, written documents, and interviews. Supporting data were collected on patients’ perspectives, the community, and the health system that provided contexts for the practice case studies.

Discussion

The complexity of primary care practices is best understood from multiple perspectives,²⁹ a principle that guided the initial selection of a multimethod comparative case study design for this investigation. The MAP that emerged from this comparative case study design has a number of strengths and weaknesses. A particular asset of the design was our ability to investigate specific phenomena within their context rather than isolated from it.²⁹ The design also encouraged the investigators to pursue emerging insights, thus informing multiple perspectives that might not have otherwise been considered, although this may be somewhat limited by the purposeful sampling strategy that focuses on maximizing information about a particular topic.

Limitations

A limitation for broader implementation of this research design is the intensity of data collection and analysis, which are difficult to accomplish without considerable resources and a research team with diverse skills. There might also be some concerns about the ways that the data collection process alters practice behavior; however, the prolonged observational time frame and the multiple data sources for “triangulation” are designed to limit any potential Hawthorne effect. That is, by collecting and systematically comparing data from multiple sources, including direct observation, different forms of interviews, and existing documents, the investigators were able to identify inconsistencies in patterns of behavior.^17,19

Although the data collection, analysis, and feedback process appeared to increase a practice’s self-reflection, our study limited the input of patient and practice participants in the design, analysis, and interpretation and thus does not approach the participatory research paradigm espoused by McCauley and colleagues.³⁰ Still, our study moved from being primarily observational descriptive research into a more collaborative and interventional project, in part at the request of the participants. This suggests a model and method for future research in the arena of health care process and outcome improvement with the practices as collaborators. The MAP characterized in this paper offers a means for simultaneously describing, understanding, and improving the richly complex and varied processes and outcomes of primary care. By more actively engaging the practices in the research process, the MAP also points toward a new more collaborative model of practice-based research.

Conclusions

The comprehensive data in the P&CD study provide a unique opportunity to understand and describe multiple perspectives from the clinician, patient, encounter, practice, community, and health system spheres. Each of the papers in this issue of JFP used some of these comprehensive data to study one or more of these spheres. For example, the encounter field notes were the primary source of data for exploring how “family” presents in encounters³¹Several of the authors used subsets of patients, including patients presenting with acute respiratory track infections,³³ smokers,³⁴ and frequent attenders.³⁵ These authors each supplemented the encounter field notes with data from the medical record reviews, medical record field notes, and patient exit cards. The complete data set including practice field notes, practice genograms, physician and staff interviews, office environment checklists, and encounter field notes were used to describe staff training, roles, and functions.³⁶ This is only a part of the research adventure available in this type of data. We hope many others will join in the excitement.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Research Center grant from the American Academy of Family Physicians. Drs Crabtree, Miller and Stange are associated with the Center for Research in Family Practice and Primary Care, Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 18 practices, without whose participation this study would not have been possible. We also wish to thank Connie Gibbs and Jen Rouse, who spent many hours in the practices collecting data, and Diane Dodendorf and Jason Lebsack, who spent countless hours coordinating transcription and data management activities, for their dedicated work. We are especially indebted to Mary McAndrews, who transcribed hundreds of taped interviews and dictated field notes. The ongoing analyses that ensured the quality and comprehensiveness of the data were made possible through the dedicated work of Helen McIlvain, PhD; Jeffrey Susman, MD; Virginia Aita, PhD; Kristine McVea, MD; Elisabeth Backer, MD; Paul Turner, PhD; and Louis Pol, PhD. Finally, we thank the members of the advisory committee: Valerie Gilchrist, MD; Paul Nutting, MD, MPH; Carlos Jaén, MD, PhD; Kurt Stange, MD, PhD; William Miller, MD, MA; Reuben McDaniel, PhD; and Ruth Anderson, RN, PhD.