Benjamin F. Crabtree, PhD

“Everyday Primary Care,” a popular, urban 3-physician family medicine office, has served mostly middle- and working-class people for more than 25 years. Most of the patients have grown older with Drs. Newman and Cope and now have a substantial chronic disease burden. Dr. Varimore, Dr. Cope’s son, has recently joined the practice. He replaced a long-time partner who left in frustration to do emergency medicine.

On a typical day, Dr. Cope enters the crowded waiting room, sighs, and walks quickly toward the nurses’ station where her third scheduled patient has just arrived; her first 2 patients are already waiting in examining rooms. In her tiny office, stacks of charts, phone messages, and forms await her attention. Phones ring constantly. Rushing to see her first patient, Dr. Cope squeezes past her nursing assistant in the narrow hallway. She catches a glimpse of her partner, Dr. Newman, at the end of the corridor. They grunt a word of greeting, but say nothing more. In fact, the physicians and their staff have barely spoken to each other in days.

The 2 older physicians were hopeful that Dr. Varimore would infuse fresh energy into the practice, but the only thing that has changed with his arrival is an increase in the number of patients they see and the expenses of running the office. When the door finally closes at the end of a long day, everyone leaves feeling exhausted and alone.

A toxic atmosphere

The situation at Everyday Primary Care is not unusual.¹ These are unhealthy times for most primary care practices. Despite the critical role that primary care is expected to play in health care reform, there is tremendous uncertainty about the future viability of primary care practice.^1-6 An alarming number of primary care physicians are leaving practice or taking early retirement as frustration and exhaustion move deeply into our community.^1,7,8 Staff turnover is high and disruptive. Primary care physicians feel buffeted by conflicting patient demands, insurance coverage restrictions, inadequate Medicare reimbursement, multiple and often inconsistent practice guidelines, and onerous government regulations. Primary care practices suffer from a culture of despair that impedes decision-making. These practices—and the physicians who struggle to keep them viable—need to develop resilience to survive in this hostile climate and improve the quality of care they provide.

Research-based strategies. This article suggests strategies for primary care practices to move forward—whatever proposed reforms emerge from the current debate. The strategies we propose derive from specific, concrete observations gathered during a 15-year program of research that included nearly 500 primary care offices.^9-16 (In fact, Everyday Primary Care is an actual practice that participated in 1 of our studies, though we’ve changed its name and the names of the physicians.) Our research was funded by the National Institutes of Health (NIH) and included both descriptive and intervention projects. Our studies provided in-depth descriptions of a wide variety of primary care practices, as well as new models for describing change.^10,14,15 The practices varied in how they delivered preventive services, in their cancer-related prevention and screening activities, and in the way they managed chronic disease.^11,14,17-19 Yet across all these variations, we found a pattern in which educated, well-trained professionals and staff wanted to provide good care, but found themselves thwarted in their efforts to succeed.

What’s going on here? We sought to understand what was really happening in these primary care practices and to formulate strategies to help them become better for patients, staff, and clinicians.

We came up with 2 fundamental insights:

• Practices that focus on building strong internal relationships are better able to deal with surprise and uncertainty.
• Practices that are proactive in interacting with the changing environment will find multiple ways to achieve effective health care delivery.

Work on building those relationships

In our research, we repeatedly observed that careful attention to the relationships among all the people (clinical and nonclinical staff) working within each practice is critical to improving practice processes and outcomes.²⁰ We wanted to learn why relationships mattered so much and how they could be improved. What we found can best be explained by taking another look at Everyday Primary Care.

The physician-owners of everyday Primary Care, feeling stressed out and recognizing that “things are not good here,” signed up to participate in 1 of our studies. Participation required allowing an outside facilitator to observe practice operations and conduct open-ended interviews with physicians and staff over a 2-week period, followed by a series of 12 weekly meetings. In addition, physicians and staff agreed to fill out multiple surveys during the study process and allow researchers to audit the charts of randomly selected patient samples.

One year after Everyday Primary Care signed up, the office space was still cramped, the financial situation was no better, and environmental pressures were continuing to mount. And yet, the practice felt like a different place, one filled with energy and hope. What had happened?

RAP, huddles, effective teams. Most importantly, the quality and types of relationships within the practice had changed. At our suggestion, the practice formed a RAP (reflective adaptive process) team under the guidance of a facilitator—a nurse we trained in basic facilitation skills, including effective meeting strategies, brainstorming, and conflict resolution. The team consisted of physician leaders (both Drs. Cope and Varimore attended all meetings), the practice manager, representatives from each part of the practice (billing, front desk, nursing staff, insurance clerk), and a patient.¹⁵ The RAP intervention was designed to provide members with time and space to reflect and opportunities to learn the value of communication, respectful interaction, and listening to diverse opinions and perspectives.²⁰ The team met with the facilitator for 1 hour every week, reviewed the practice’s vision, and developed and implemented strategies for solving prioritized practice issues and problems.

Brainstorming helped identify recurrent problems. As the RAP meetings progressed, it became clear that despite the close quarters, each part of the practice was isolated from the others and all team members were frustrated by their inability to influence the lead physician, Dr. Cope. Over time, the RAP meetings changed the relationship patterns and the quality of communication, thus helping the practice move forward and get unstuck. Dr. Cope repeatedly commented, “I didn’t know that,” as staff shared their concerns and challenges. For example, Dr. Cope was amazed when the front desk described the amount of time and degree of disruption caused by drug reps constantly coming into the office. Together, the team was able to come up with a solution—setting aside a special time for drug reps, rather than allowing them to arrive whenever they chose—that worked for physicians and staff alike.

Our current project notes from Everyday Primary Care reflect a very different and vibrant practice, in which the atmosphere is charged with hope and everyone reports being more relaxed—though just as busy. Office processes have improved and space is less cluttered. Chart audit scores reveal improved quality of chronic care and preventive services. Because practice members have learned to communicate across the barriers of job classification and hierarchy, they are able to solve problems as they arise without allowing things to fester. These improved relationships led to an enhanced understanding of complex issues like patient triage and scheduling and more numerous and accurate memories of how the practice has operated over the years.^21-23

Our research has taught us that practices that pay attention to building strong relationships are better able to deal with the surprise and uncertainty that characterize modern health care delivery.^24-26 The primary care management literature has highlighted a number of practical strategies for enhancing relationships and communication, including the use of RAP teams, huddles, effective team meetings, and high-performing clinical teams.^15,27-29 In addition, we refer the reader to The Team Handbook, 3rd ed., by Peter R. Scholtes, Brian L. Joiner, and Barbara J. Streibel. The handbook contains a wide range of practical teambuilding strategies in an easily accessible style.³⁰ FIGURE 1 summarizes 5 tips for building critical relationships in your own practice.

FIGURE 1
5 tips for building critical relationships

Interact with the “local fitness landscape”

Our second insight is that practices must learn to interact with what we call the “local fitness landscape.”^31-33 To understand what that term implies, imagine your hometown with multiple primary care offices of different sizes, a variety of specialty practices, 2 or 3 competing hospital systems, multiple insurance options, businesses, housing clusters representing different social classes, schools, banks, scattered farms, industries, waterways, animals and plants, transportation systems, and political and religious institutions. The totality of all these elements is the local fitness landscape.

The landscape is a dynamic, fluid system within which the component parts respond to and influence each other. Everyday Primary Care is embedded in such a landscape, acting on and being acted upon by other parts of the system. Unfortunately, like most practices we observed, Everyday tended to ignore or resist the local fitness landscape rather than trying to understand and adapt to it. The physicians felt trapped by environmental constraints and frustrated by the turbulence they observed.

What constraints does Everyday Primary Care face? When we first visited this practice, we could see that the facility was too small for the growing volume of patients. The physician-owners knew the space wasn’t conducive to optimum patient care, but told us they could not afford to pay higher rent for larger quarters. Similarly, they understood the potential of electronic medical records (EMRs), but hadn’t been able to find time or money to support the transition. Rising overhead expenses were outpacing practice productivity, as measured in the number of patients seen per day. What was worse, the need to see so many patients was making it more difficult to address the needs of their aging and medically complex patient population.

Looking outward can help

Despite these constraints, internal conversations generated through RAP sessions led practice staff to reach out to other physicians and physician organizations for information. They compared notes with other practices on questions like how their computerized billing system functions, or how to word a letter to patients announcing a new policy on prescription refills. These external conversations expanded the practice’s notions of what was possible and gave them opportunities to share information and learn of new approaches other practices were developing. The result was a newfound level of energy and hope within the practice and exposure to new ideas from the outside.

Learning from the landscape. Numerous conversations with physician organizations, neighboring practices, and a local hospital system yielded new solutions for recalcitrant problems: How to make better use of existing office space, for example, and where to find support for long-range strategic planning. These contacts exposed Everyday to the experiences of other practices with EMRs, and the practice’s physicians have now selected and implemented their own system. The practice was finally able to address the inevitable retirement of 1 of the physicians and now has a succession plan in place. In sum, Everyday learned how to interact and adjust to the changing environment and no longer worried about survival.

Practices co-evolve with all the other systems in a constantly changing fitness landscape. As practice members navigate the local fitness landscapes, they make decisions among competing demands and priorities to maintain their own financial viability and internal stability. What seems to characterize innovative primary care practices is that they don’t wait to react to the next environmental change. Rather, by paying attention to local relationships, they improve the chances that co-evolution will move the practice in desired ways.

Making much-needed connections

There are a number of ways that practices can engage their fitness landscapes, but perhaps the most powerful is creating the time and space to meet with colleagues—either locally or regionally. The most effective approaches are likely to be those that allow sharing experiences and ideas over time, rather than one-time, opportunistic conversations that occur, say, at national and state academy meetings. Practices can participate in activities of regional Practice-Based Research Networks, local residency programs, or even form their own local support group.^34,35 To learn how you can connect with a regional Practice-Based Research Network, go to the AHRQ website (http://pbrn.ahrq.gov/portal/server.pt). FIGURE 2 summarizes 4 strategies for reaching out to your local landscape.

FIGURE 2
4 strategies for reaching out to your local landscape

One size doesn’t fit all: Strategic alternatives

When practices build critical relationships and pay attention to their local fitness landscape, they co-evolve improvements that make sense in the context of their unique characteristics and circumstances. Our research shows that practices use a range of alternative strategies to meet the needs of patients, their communities, and themselves. For example, while we have observed primary care offices using EMRs that have achieved high levels of adherence to diabetes guidelines, we have also found high adherence rates in practices that use paper charts.¹⁹ We have seen different, successful approaches to the delivery of preventive health services.¹¹ Some practices involve staff in assuring protocol adherence and others don’t. Some use reminder systems and others don’t. Several practices with higher rates of preventive service delivery use none of these. A recent evaluation of 15 case studies of family practices using teams to implement the chronic care model showed the value of different types of teams in different practices.³⁶

Variability and standardization. The emergence of processes and outcome measures designed to meet the needs of a particular local setting (fitness landscape) appeals to our sense of equity and common sense. Yet variations like these fly in the face of prevailing models and guidelines that emphasize standardized processes. Many health plans and provider organizations insist on evidence-based “best practices” and “optimized models” for delivering primary care.^37-39 They assume that if we know the goals, there is a best way to get everyone to achieve them.

A better strategy is to determine when variability and tailoring are more appropriate and then use standardization to help create more time for those processes that require variation. Thus, the practice can use a standardized protocol to turn over immunizations to staff in order to free clinicians to spend more time interacting directly with patients.

Multiple pathways to excellence. Medical practice is full of surprises and complexities. We used to believe that the right tools in the hands of accountable individuals using good management systems would produce best practice outcomes. But we have learned that no single right tool or individual management strategy works consistently in primary care.

We now believe that the relationship system within the practice is a critical element in creating an optimal healing environment. Practices with improved relationship systems exhibit more resilience in weathering a hostile environment, while discovering their own unique model of successful primary care. Such practices can thrive, provide improved quality of patient-centered care, and find professional satisfaction and joy in daily work. We hope that the health care reform plans now being debated in Congress will be informed by these insights and provide space for multiple models of care delivery to emerge.

CORRESPONDENCE
Benjamin F. Crabtree, PhD, Department of Family Medicine, Robert Wood Johnson Medical school, 1 World’s Fair Drive, First Floor, Somerset, NJ 08873; [email protected]

“Everyday Primary Care,” a popular, urban 3-physician family medicine office, has served mostly middle- and working-class people for more than 25 years. Most of the patients have grown older with Drs. Newman and Cope and now have a substantial chronic disease burden. Dr. Varimore, Dr. Cope’s son, has recently joined the practice. He replaced a long-time partner who left in frustration to do emergency medicine.

On a typical day, Dr. Cope enters the crowded waiting room, sighs, and walks quickly toward the nurses’ station where her third scheduled patient has just arrived; her first 2 patients are already waiting in examining rooms. In her tiny office, stacks of charts, phone messages, and forms await her attention. Phones ring constantly. Rushing to see her first patient, Dr. Cope squeezes past her nursing assistant in the narrow hallway. She catches a glimpse of her partner, Dr. Newman, at the end of the corridor. They grunt a word of greeting, but say nothing more. In fact, the physicians and their staff have barely spoken to each other in days.

The 2 older physicians were hopeful that Dr. Varimore would infuse fresh energy into the practice, but the only thing that has changed with his arrival is an increase in the number of patients they see and the expenses of running the office. When the door finally closes at the end of a long day, everyone leaves feeling exhausted and alone.

A toxic atmosphere

The situation at Everyday Primary Care is not unusual.¹ These are unhealthy times for most primary care practices. Despite the critical role that primary care is expected to play in health care reform, there is tremendous uncertainty about the future viability of primary care practice.^1-6 An alarming number of primary care physicians are leaving practice or taking early retirement as frustration and exhaustion move deeply into our community.^1,7,8 Staff turnover is high and disruptive. Primary care physicians feel buffeted by conflicting patient demands, insurance coverage restrictions, inadequate Medicare reimbursement, multiple and often inconsistent practice guidelines, and onerous government regulations. Primary care practices suffer from a culture of despair that impedes decision-making. These practices—and the physicians who struggle to keep them viable—need to develop resilience to survive in this hostile climate and improve the quality of care they provide.

Research-based strategies. This article suggests strategies for primary care practices to move forward—whatever proposed reforms emerge from the current debate. The strategies we propose derive from specific, concrete observations gathered during a 15-year program of research that included nearly 500 primary care offices.^9-16 (In fact, Everyday Primary Care is an actual practice that participated in 1 of our studies, though we’ve changed its name and the names of the physicians.) Our research was funded by the National Institutes of Health (NIH) and included both descriptive and intervention projects. Our studies provided in-depth descriptions of a wide variety of primary care practices, as well as new models for describing change.^10,14,15 The practices varied in how they delivered preventive services, in their cancer-related prevention and screening activities, and in the way they managed chronic disease.^11,14,17-19 Yet across all these variations, we found a pattern in which educated, well-trained professionals and staff wanted to provide good care, but found themselves thwarted in their efforts to succeed.

What’s going on here? We sought to understand what was really happening in these primary care practices and to formulate strategies to help them become better for patients, staff, and clinicians.

We came up with 2 fundamental insights:

• Practices that focus on building strong internal relationships are better able to deal with surprise and uncertainty.
• Practices that are proactive in interacting with the changing environment will find multiple ways to achieve effective health care delivery.

Work on building those relationships

In our research, we repeatedly observed that careful attention to the relationships among all the people (clinical and nonclinical staff) working within each practice is critical to improving practice processes and outcomes.²⁰ We wanted to learn why relationships mattered so much and how they could be improved. What we found can best be explained by taking another look at Everyday Primary Care.

The physician-owners of everyday Primary Care, feeling stressed out and recognizing that “things are not good here,” signed up to participate in 1 of our studies. Participation required allowing an outside facilitator to observe practice operations and conduct open-ended interviews with physicians and staff over a 2-week period, followed by a series of 12 weekly meetings. In addition, physicians and staff agreed to fill out multiple surveys during the study process and allow researchers to audit the charts of randomly selected patient samples.

One year after Everyday Primary Care signed up, the office space was still cramped, the financial situation was no better, and environmental pressures were continuing to mount. And yet, the practice felt like a different place, one filled with energy and hope. What had happened?

RAP, huddles, effective teams. Most importantly, the quality and types of relationships within the practice had changed. At our suggestion, the practice formed a RAP (reflective adaptive process) team under the guidance of a facilitator—a nurse we trained in basic facilitation skills, including effective meeting strategies, brainstorming, and conflict resolution. The team consisted of physician leaders (both Drs. Cope and Varimore attended all meetings), the practice manager, representatives from each part of the practice (billing, front desk, nursing staff, insurance clerk), and a patient.¹⁵ The RAP intervention was designed to provide members with time and space to reflect and opportunities to learn the value of communication, respectful interaction, and listening to diverse opinions and perspectives.²⁰ The team met with the facilitator for 1 hour every week, reviewed the practice’s vision, and developed and implemented strategies for solving prioritized practice issues and problems.

Brainstorming helped identify recurrent problems. As the RAP meetings progressed, it became clear that despite the close quarters, each part of the practice was isolated from the others and all team members were frustrated by their inability to influence the lead physician, Dr. Cope. Over time, the RAP meetings changed the relationship patterns and the quality of communication, thus helping the practice move forward and get unstuck. Dr. Cope repeatedly commented, “I didn’t know that,” as staff shared their concerns and challenges. For example, Dr. Cope was amazed when the front desk described the amount of time and degree of disruption caused by drug reps constantly coming into the office. Together, the team was able to come up with a solution—setting aside a special time for drug reps, rather than allowing them to arrive whenever they chose—that worked for physicians and staff alike.

Our current project notes from Everyday Primary Care reflect a very different and vibrant practice, in which the atmosphere is charged with hope and everyone reports being more relaxed—though just as busy. Office processes have improved and space is less cluttered. Chart audit scores reveal improved quality of chronic care and preventive services. Because practice members have learned to communicate across the barriers of job classification and hierarchy, they are able to solve problems as they arise without allowing things to fester. These improved relationships led to an enhanced understanding of complex issues like patient triage and scheduling and more numerous and accurate memories of how the practice has operated over the years.^21-23

Our research has taught us that practices that pay attention to building strong relationships are better able to deal with the surprise and uncertainty that characterize modern health care delivery.^24-26 The primary care management literature has highlighted a number of practical strategies for enhancing relationships and communication, including the use of RAP teams, huddles, effective team meetings, and high-performing clinical teams.^15,27-29 In addition, we refer the reader to The Team Handbook, 3rd ed., by Peter R. Scholtes, Brian L. Joiner, and Barbara J. Streibel. The handbook contains a wide range of practical teambuilding strategies in an easily accessible style.³⁰ FIGURE 1 summarizes 5 tips for building critical relationships in your own practice.

FIGURE 1
5 tips for building critical relationships

Interact with the “local fitness landscape”

Our second insight is that practices must learn to interact with what we call the “local fitness landscape.”^31-33 To understand what that term implies, imagine your hometown with multiple primary care offices of different sizes, a variety of specialty practices, 2 or 3 competing hospital systems, multiple insurance options, businesses, housing clusters representing different social classes, schools, banks, scattered farms, industries, waterways, animals and plants, transportation systems, and political and religious institutions. The totality of all these elements is the local fitness landscape.

The landscape is a dynamic, fluid system within which the component parts respond to and influence each other. Everyday Primary Care is embedded in such a landscape, acting on and being acted upon by other parts of the system. Unfortunately, like most practices we observed, Everyday tended to ignore or resist the local fitness landscape rather than trying to understand and adapt to it. The physicians felt trapped by environmental constraints and frustrated by the turbulence they observed.

What constraints does Everyday Primary Care face? When we first visited this practice, we could see that the facility was too small for the growing volume of patients. The physician-owners knew the space wasn’t conducive to optimum patient care, but told us they could not afford to pay higher rent for larger quarters. Similarly, they understood the potential of electronic medical records (EMRs), but hadn’t been able to find time or money to support the transition. Rising overhead expenses were outpacing practice productivity, as measured in the number of patients seen per day. What was worse, the need to see so many patients was making it more difficult to address the needs of their aging and medically complex patient population.

Looking outward can help

Despite these constraints, internal conversations generated through RAP sessions led practice staff to reach out to other physicians and physician organizations for information. They compared notes with other practices on questions like how their computerized billing system functions, or how to word a letter to patients announcing a new policy on prescription refills. These external conversations expanded the practice’s notions of what was possible and gave them opportunities to share information and learn of new approaches other practices were developing. The result was a newfound level of energy and hope within the practice and exposure to new ideas from the outside.

Learning from the landscape. Numerous conversations with physician organizations, neighboring practices, and a local hospital system yielded new solutions for recalcitrant problems: How to make better use of existing office space, for example, and where to find support for long-range strategic planning. These contacts exposed Everyday to the experiences of other practices with EMRs, and the practice’s physicians have now selected and implemented their own system. The practice was finally able to address the inevitable retirement of 1 of the physicians and now has a succession plan in place. In sum, Everyday learned how to interact and adjust to the changing environment and no longer worried about survival.

Practices co-evolve with all the other systems in a constantly changing fitness landscape. As practice members navigate the local fitness landscapes, they make decisions among competing demands and priorities to maintain their own financial viability and internal stability. What seems to characterize innovative primary care practices is that they don’t wait to react to the next environmental change. Rather, by paying attention to local relationships, they improve the chances that co-evolution will move the practice in desired ways.

Making much-needed connections

There are a number of ways that practices can engage their fitness landscapes, but perhaps the most powerful is creating the time and space to meet with colleagues—either locally or regionally. The most effective approaches are likely to be those that allow sharing experiences and ideas over time, rather than one-time, opportunistic conversations that occur, say, at national and state academy meetings. Practices can participate in activities of regional Practice-Based Research Networks, local residency programs, or even form their own local support group.^34,35 To learn how you can connect with a regional Practice-Based Research Network, go to the AHRQ website (http://pbrn.ahrq.gov/portal/server.pt). FIGURE 2 summarizes 4 strategies for reaching out to your local landscape.

FIGURE 2
4 strategies for reaching out to your local landscape

One size doesn’t fit all: Strategic alternatives

When practices build critical relationships and pay attention to their local fitness landscape, they co-evolve improvements that make sense in the context of their unique characteristics and circumstances. Our research shows that practices use a range of alternative strategies to meet the needs of patients, their communities, and themselves. For example, while we have observed primary care offices using EMRs that have achieved high levels of adherence to diabetes guidelines, we have also found high adherence rates in practices that use paper charts.¹⁹ We have seen different, successful approaches to the delivery of preventive health services.¹¹ Some practices involve staff in assuring protocol adherence and others don’t. Some use reminder systems and others don’t. Several practices with higher rates of preventive service delivery use none of these. A recent evaluation of 15 case studies of family practices using teams to implement the chronic care model showed the value of different types of teams in different practices.³⁶

Variability and standardization. The emergence of processes and outcome measures designed to meet the needs of a particular local setting (fitness landscape) appeals to our sense of equity and common sense. Yet variations like these fly in the face of prevailing models and guidelines that emphasize standardized processes. Many health plans and provider organizations insist on evidence-based “best practices” and “optimized models” for delivering primary care.^37-39 They assume that if we know the goals, there is a best way to get everyone to achieve them.

A better strategy is to determine when variability and tailoring are more appropriate and then use standardization to help create more time for those processes that require variation. Thus, the practice can use a standardized protocol to turn over immunizations to staff in order to free clinicians to spend more time interacting directly with patients.

Multiple pathways to excellence. Medical practice is full of surprises and complexities. We used to believe that the right tools in the hands of accountable individuals using good management systems would produce best practice outcomes. But we have learned that no single right tool or individual management strategy works consistently in primary care.

We now believe that the relationship system within the practice is a critical element in creating an optimal healing environment. Practices with improved relationship systems exhibit more resilience in weathering a hostile environment, while discovering their own unique model of successful primary care. Such practices can thrive, provide improved quality of patient-centered care, and find professional satisfaction and joy in daily work. We hope that the health care reform plans now being debated in Congress will be informed by these insights and provide space for multiple models of care delivery to emerge.

CORRESPONDENCE
Benjamin F. Crabtree, PhD, Department of Family Medicine, Robert Wood Johnson Medical school, 1 World’s Fair Drive, First Floor, Somerset, NJ 08873; [email protected]

“Everyday Primary Care,” a popular, urban 3-physician family medicine office, has served mostly middle- and working-class people for more than 25 years. Most of the patients have grown older with Drs. Newman and Cope and now have a substantial chronic disease burden. Dr. Varimore, Dr. Cope’s son, has recently joined the practice. He replaced a long-time partner who left in frustration to do emergency medicine.

On a typical day, Dr. Cope enters the crowded waiting room, sighs, and walks quickly toward the nurses’ station where her third scheduled patient has just arrived; her first 2 patients are already waiting in examining rooms. In her tiny office, stacks of charts, phone messages, and forms await her attention. Phones ring constantly. Rushing to see her first patient, Dr. Cope squeezes past her nursing assistant in the narrow hallway. She catches a glimpse of her partner, Dr. Newman, at the end of the corridor. They grunt a word of greeting, but say nothing more. In fact, the physicians and their staff have barely spoken to each other in days.

The 2 older physicians were hopeful that Dr. Varimore would infuse fresh energy into the practice, but the only thing that has changed with his arrival is an increase in the number of patients they see and the expenses of running the office. When the door finally closes at the end of a long day, everyone leaves feeling exhausted and alone.

A toxic atmosphere

The situation at Everyday Primary Care is not unusual.¹ These are unhealthy times for most primary care practices. Despite the critical role that primary care is expected to play in health care reform, there is tremendous uncertainty about the future viability of primary care practice.^1-6 An alarming number of primary care physicians are leaving practice or taking early retirement as frustration and exhaustion move deeply into our community.^1,7,8 Staff turnover is high and disruptive. Primary care physicians feel buffeted by conflicting patient demands, insurance coverage restrictions, inadequate Medicare reimbursement, multiple and often inconsistent practice guidelines, and onerous government regulations. Primary care practices suffer from a culture of despair that impedes decision-making. These practices—and the physicians who struggle to keep them viable—need to develop resilience to survive in this hostile climate and improve the quality of care they provide.

Research-based strategies. This article suggests strategies for primary care practices to move forward—whatever proposed reforms emerge from the current debate. The strategies we propose derive from specific, concrete observations gathered during a 15-year program of research that included nearly 500 primary care offices.^9-16 (In fact, Everyday Primary Care is an actual practice that participated in 1 of our studies, though we’ve changed its name and the names of the physicians.) Our research was funded by the National Institutes of Health (NIH) and included both descriptive and intervention projects. Our studies provided in-depth descriptions of a wide variety of primary care practices, as well as new models for describing change.^10,14,15 The practices varied in how they delivered preventive services, in their cancer-related prevention and screening activities, and in the way they managed chronic disease.^11,14,17-19 Yet across all these variations, we found a pattern in which educated, well-trained professionals and staff wanted to provide good care, but found themselves thwarted in their efforts to succeed.

What’s going on here? We sought to understand what was really happening in these primary care practices and to formulate strategies to help them become better for patients, staff, and clinicians.

We came up with 2 fundamental insights:

• Practices that focus on building strong internal relationships are better able to deal with surprise and uncertainty.
• Practices that are proactive in interacting with the changing environment will find multiple ways to achieve effective health care delivery.

Work on building those relationships

In our research, we repeatedly observed that careful attention to the relationships among all the people (clinical and nonclinical staff) working within each practice is critical to improving practice processes and outcomes.²⁰ We wanted to learn why relationships mattered so much and how they could be improved. What we found can best be explained by taking another look at Everyday Primary Care.

The physician-owners of everyday Primary Care, feeling stressed out and recognizing that “things are not good here,” signed up to participate in 1 of our studies. Participation required allowing an outside facilitator to observe practice operations and conduct open-ended interviews with physicians and staff over a 2-week period, followed by a series of 12 weekly meetings. In addition, physicians and staff agreed to fill out multiple surveys during the study process and allow researchers to audit the charts of randomly selected patient samples.

One year after Everyday Primary Care signed up, the office space was still cramped, the financial situation was no better, and environmental pressures were continuing to mount. And yet, the practice felt like a different place, one filled with energy and hope. What had happened?

RAP, huddles, effective teams. Most importantly, the quality and types of relationships within the practice had changed. At our suggestion, the practice formed a RAP (reflective adaptive process) team under the guidance of a facilitator—a nurse we trained in basic facilitation skills, including effective meeting strategies, brainstorming, and conflict resolution. The team consisted of physician leaders (both Drs. Cope and Varimore attended all meetings), the practice manager, representatives from each part of the practice (billing, front desk, nursing staff, insurance clerk), and a patient.¹⁵ The RAP intervention was designed to provide members with time and space to reflect and opportunities to learn the value of communication, respectful interaction, and listening to diverse opinions and perspectives.²⁰ The team met with the facilitator for 1 hour every week, reviewed the practice’s vision, and developed and implemented strategies for solving prioritized practice issues and problems.

Brainstorming helped identify recurrent problems. As the RAP meetings progressed, it became clear that despite the close quarters, each part of the practice was isolated from the others and all team members were frustrated by their inability to influence the lead physician, Dr. Cope. Over time, the RAP meetings changed the relationship patterns and the quality of communication, thus helping the practice move forward and get unstuck. Dr. Cope repeatedly commented, “I didn’t know that,” as staff shared their concerns and challenges. For example, Dr. Cope was amazed when the front desk described the amount of time and degree of disruption caused by drug reps constantly coming into the office. Together, the team was able to come up with a solution—setting aside a special time for drug reps, rather than allowing them to arrive whenever they chose—that worked for physicians and staff alike.

Our current project notes from Everyday Primary Care reflect a very different and vibrant practice, in which the atmosphere is charged with hope and everyone reports being more relaxed—though just as busy. Office processes have improved and space is less cluttered. Chart audit scores reveal improved quality of chronic care and preventive services. Because practice members have learned to communicate across the barriers of job classification and hierarchy, they are able to solve problems as they arise without allowing things to fester. These improved relationships led to an enhanced understanding of complex issues like patient triage and scheduling and more numerous and accurate memories of how the practice has operated over the years.^21-23

Our research has taught us that practices that pay attention to building strong relationships are better able to deal with the surprise and uncertainty that characterize modern health care delivery.^24-26 The primary care management literature has highlighted a number of practical strategies for enhancing relationships and communication, including the use of RAP teams, huddles, effective team meetings, and high-performing clinical teams.^15,27-29 In addition, we refer the reader to The Team Handbook, 3rd ed., by Peter R. Scholtes, Brian L. Joiner, and Barbara J. Streibel. The handbook contains a wide range of practical teambuilding strategies in an easily accessible style.³⁰ FIGURE 1 summarizes 5 tips for building critical relationships in your own practice.

FIGURE 1
5 tips for building critical relationships

Interact with the “local fitness landscape”

Our second insight is that practices must learn to interact with what we call the “local fitness landscape.”^31-33 To understand what that term implies, imagine your hometown with multiple primary care offices of different sizes, a variety of specialty practices, 2 or 3 competing hospital systems, multiple insurance options, businesses, housing clusters representing different social classes, schools, banks, scattered farms, industries, waterways, animals and plants, transportation systems, and political and religious institutions. The totality of all these elements is the local fitness landscape.

The landscape is a dynamic, fluid system within which the component parts respond to and influence each other. Everyday Primary Care is embedded in such a landscape, acting on and being acted upon by other parts of the system. Unfortunately, like most practices we observed, Everyday tended to ignore or resist the local fitness landscape rather than trying to understand and adapt to it. The physicians felt trapped by environmental constraints and frustrated by the turbulence they observed.

What constraints does Everyday Primary Care face? When we first visited this practice, we could see that the facility was too small for the growing volume of patients. The physician-owners knew the space wasn’t conducive to optimum patient care, but told us they could not afford to pay higher rent for larger quarters. Similarly, they understood the potential of electronic medical records (EMRs), but hadn’t been able to find time or money to support the transition. Rising overhead expenses were outpacing practice productivity, as measured in the number of patients seen per day. What was worse, the need to see so many patients was making it more difficult to address the needs of their aging and medically complex patient population.

Looking outward can help

Despite these constraints, internal conversations generated through RAP sessions led practice staff to reach out to other physicians and physician organizations for information. They compared notes with other practices on questions like how their computerized billing system functions, or how to word a letter to patients announcing a new policy on prescription refills. These external conversations expanded the practice’s notions of what was possible and gave them opportunities to share information and learn of new approaches other practices were developing. The result was a newfound level of energy and hope within the practice and exposure to new ideas from the outside.

Learning from the landscape. Numerous conversations with physician organizations, neighboring practices, and a local hospital system yielded new solutions for recalcitrant problems: How to make better use of existing office space, for example, and where to find support for long-range strategic planning. These contacts exposed Everyday to the experiences of other practices with EMRs, and the practice’s physicians have now selected and implemented their own system. The practice was finally able to address the inevitable retirement of 1 of the physicians and now has a succession plan in place. In sum, Everyday learned how to interact and adjust to the changing environment and no longer worried about survival.

Practices co-evolve with all the other systems in a constantly changing fitness landscape. As practice members navigate the local fitness landscapes, they make decisions among competing demands and priorities to maintain their own financial viability and internal stability. What seems to characterize innovative primary care practices is that they don’t wait to react to the next environmental change. Rather, by paying attention to local relationships, they improve the chances that co-evolution will move the practice in desired ways.

Making much-needed connections

There are a number of ways that practices can engage their fitness landscapes, but perhaps the most powerful is creating the time and space to meet with colleagues—either locally or regionally. The most effective approaches are likely to be those that allow sharing experiences and ideas over time, rather than one-time, opportunistic conversations that occur, say, at national and state academy meetings. Practices can participate in activities of regional Practice-Based Research Networks, local residency programs, or even form their own local support group.^34,35 To learn how you can connect with a regional Practice-Based Research Network, go to the AHRQ website (http://pbrn.ahrq.gov/portal/server.pt). FIGURE 2 summarizes 4 strategies for reaching out to your local landscape.

FIGURE 2
4 strategies for reaching out to your local landscape

One size doesn’t fit all: Strategic alternatives

When practices build critical relationships and pay attention to their local fitness landscape, they co-evolve improvements that make sense in the context of their unique characteristics and circumstances. Our research shows that practices use a range of alternative strategies to meet the needs of patients, their communities, and themselves. For example, while we have observed primary care offices using EMRs that have achieved high levels of adherence to diabetes guidelines, we have also found high adherence rates in practices that use paper charts.¹⁹ We have seen different, successful approaches to the delivery of preventive health services.¹¹ Some practices involve staff in assuring protocol adherence and others don’t. Some use reminder systems and others don’t. Several practices with higher rates of preventive service delivery use none of these. A recent evaluation of 15 case studies of family practices using teams to implement the chronic care model showed the value of different types of teams in different practices.³⁶

Variability and standardization. The emergence of processes and outcome measures designed to meet the needs of a particular local setting (fitness landscape) appeals to our sense of equity and common sense. Yet variations like these fly in the face of prevailing models and guidelines that emphasize standardized processes. Many health plans and provider organizations insist on evidence-based “best practices” and “optimized models” for delivering primary care.^37-39 They assume that if we know the goals, there is a best way to get everyone to achieve them.

A better strategy is to determine when variability and tailoring are more appropriate and then use standardization to help create more time for those processes that require variation. Thus, the practice can use a standardized protocol to turn over immunizations to staff in order to free clinicians to spend more time interacting directly with patients.

Multiple pathways to excellence. Medical practice is full of surprises and complexities. We used to believe that the right tools in the hands of accountable individuals using good management systems would produce best practice outcomes. But we have learned that no single right tool or individual management strategy works consistently in primary care.

We now believe that the relationship system within the practice is a critical element in creating an optimal healing environment. Practices with improved relationship systems exhibit more resilience in weathering a hostile environment, while discovering their own unique model of successful primary care. Such practices can thrive, provide improved quality of patient-centered care, and find professional satisfaction and joy in daily work. We hope that the health care reform plans now being debated in Congress will be informed by these insights and provide space for multiple models of care delivery to emerge.

CORRESPONDENCE
Benjamin F. Crabtree, PhD, Department of Family Medicine, Robert Wood Johnson Medical school, 1 World’s Fair Drive, First Floor, Somerset, NJ 08873; [email protected]

Clinicians, researchers, and policymakers now recognize that multiple competing demands1 and opportunities² are simultaneously affecting the physicians, staff, and patients within primary care practices. Our current understanding of outpatient practice is largely based on administrative databases, national surveys, and medical record reviews, with additional insights from surveys of patients or clinicians. These data generally are not designed to capture the richness of the content and context that is needed to better understand the realities and complexities of practice.^3-6 The underlying premise of The Prevention and Competing Demands in Primary Care (P&CD) Study is that efforts to change practice should be preceded by efforts to understand it.^2,7 The explicit goal of this study is to understand practice structure and process, including details of patients, physicians, staff, and clinical encounters; the practice as an organization; and its relationship to the larger community and health system.

In this paper we describe a dynamic observational multimethod assessment process (MAP) that can be used to understand the complex reality of primary care practice. MAP is based on a multimethod comparative case study design^8,9 that integrates elements of epidemiology with methods derived from the qualitative traditions of anthropology and sociology and relies most heavily on qualitative observation and interviewing methods. Studies of this type require an iterative data collection and analysis approach that evolves over time so that new methods can be introduced as the investigators gain a better understanding of important issues. A major strength of our study design was that it allowed hypotheses and insights gained from participants and from ongoing analyses to be integrated into the ongoing investigation.

The study’s primary research questions related to how practice characteristics affect preventive service delivery. Thus, the research design included: (1) an examination of the organizational contexts that support preventive services, (2) an examination of the competing demands imposed by carrying out clinical prevention and illness care in clinical encounters and in the practice, (3) a comparison of the approaches used by practices with high versus low intensity of preventive services delivered to eligible patients, and (4) an examination of approaches used to deliver different types of preventive services. Although the particular focus was on preventive services, the rich MAP allowed pursuit of other research topics that are presented in this issue of JFP.

This article describes the evolutionary methods of the P&CD study, focusing on how data were collected to ensure that sufficient details were available to understand a practice’s values, structures, and processes.*

Emergent research design

The P&CD study was conceived in 1994 to be an in-depth follow-up of insights from the Direct Observation of Primary Care (DOPC) Study that was just getting under way in northeastern Ohio.⁵ The DOPC Study provided a largely quantitative assessment of patients, physicians, encounters, and practices using patient questionnaires, physician surveys, medical record audits, and direct observation of clinical encounters using the Davis Observation Code.¹⁰ That study’s initial findings were presented in the May 1998 theme issue of JFP, and the study processes have recently been described.¹¹ Details of the DOPC methods have been published elsewhere.^4,5,12

Although the initial design allowed the DOPC research nurses to collect brief observational notes, the intensity of the quantitative data collection limited the scope of the study’s qualitative data for understanding details of the practice’s organization and the competing demands within clinical encounters. As a consequence, the P&CD study was designed to provide more in-depth description and understanding of the competing demands of family practice, and in particular, to evaluate factors affecting preventive services delivery using a comparative case study design and a MAP.

A key feature of the P&CD study design was an openness to the integration of emerging insights into the data collection protocol. For example, preliminary analyses of the DOPC data¹³ and other ongoing studies¹⁴ led to the discovery that complexity science was valuable for explaining the dynamics of office systems⁶ and needed to be incorporated into the design. (Complexity science is the study of systems that are characterized by nonlinear dynamics and emergent properties; it emphasizes the need to understand the interrelationships of the whole system and not just collect data about the parts.¹⁵) The investigators also developed new ways to display the relationships among physicians and staff in the practices using “practice genograms.”¹⁶ The practice genogram is a diagram of the functional and interpersonal relationships among the clinicians, support staff, and other people and organizations interacting with the practice. Throughout the project and consistent with the standards of qualitative research design,17 there were continued modifications and enhancements in the data collection and analysis strategies in response to insights that were emerging from ongoing analyses and interpretation of the data.

An important feature of the project was the development of an advisory committee of consultants and co-investigators that convened annually to provide multidisciplinary input, review results, and provide feedback. The advisory committee included academic representatives with expertise in nursing, health education, women’s health, minority health, and public policy. Two additional members were added to the project to provide expertise into the study of organizations as complex systems. The annual reviews by the advisory committee led to significant changes in the research design while the study was ongoing.

Practice Sample

Beginning in late 1996, we drew from 91 practices in Nebraska that had been randomly selected to participate in an earlier study on tobacco prevention and cessation.¹⁸ Initially a sample of 10 practices was purposefully selected^19,20 using an iterative process to represent a range in size (small and large), geographic location (urban, suburban, and rural), and rate of delivery of tobacco-related preventive services. Preliminary analyses of these 10 initial practices provided a summary of preventive health delivery strategies in primary care practices and a description of competing demands that enhanced or limited these strategies. To confirm or refute the emerging insights from the original 10 practices, 8 additional practices were selected for further data collection during the second and third years of the study. The sampling strategy in years 2 and 3 ensured that at least 2 practices each from several major regional hospital health systems were included and allowed us to assess emerging hypotheses about the importance of health system context for understanding community practices.

The practices were recruited by contacting one of the physicians to solicit participation; only those in which all family physicians in the practice agreed to participate were included in the study. Twenty-three practices were contacted; all physicians in 18 agreed to participate (78%).

Core Data Collection Methods

Data were collected by trained field researchers who spent 4 weeks or more taking notes at each practice while observing the practice and clinical encounters, conducting informal key informant interviews of staff, collecting office documents, and auditing charts of patients whose encounters were observed. Within each practice, data collection occurred in stages, with a short break after the initial week or 2 of observation to allow preliminary analyses to inform additional data collection.

Observations at the practice level were recorded in a combination of structured observational checklists, unstructured dictated field notes,^21-23 and key informant interviews.²⁴ Detailed floor plans of the practice were used to identify where particular activities occurred and where individual practice participants worked. Each day at the practice, the field researcher took short notes or “jottings” and dictated expanded field notes in the evening.²³ A template of topics was used periodically to ensure that important aspects of the practice were not being overlooked. The template included lists of features of the community, practice, staff, and patients that the researchers saw as important Figure W1.*¹³ A 3-page structured practice environment checklist was adapted from earlier work on the DOPC project¹¹ and included a wide range of practice characteristics and functions, including items such as the number and training of staff, counseling options offered, and management of telephone calls and referrals Figure W2a. This checklist also served as a detailed reminder to the research nurse of topics to be included in the field notes. Throughout the time the field researchers were recording field notes and filling in the checklist, they opportunistically asked clinicians or staff informal key informant questions for confirmation or clarification.

Consecutive patients for each clinician in a practice were approached with a goal of recruiting 30 patients who would consent to have the field researcher observe their visit. This generally required approaching 35 to 40 patients. Because some clinicians worked part-time or were not consistently in the practice, it was not always possible to observe 30 visits. After explaining the study and gaining signed informed consent from patients who agreed to participate, the field researchers observed the outpatient visit as unobtrusively as possible. A 1-page structured encounter checklist Figure W3 that was also modified from the DOPC study11 provided blanks for noting the reason for the visit, chief complaints, and final diagnoses, and for indicating whether any of approximately 100 preventive services were ordered or delivered. Space was provided at the bottom of the form for recording notes that were later used to dictate a re-creation of the encounter. At a later time, a chart audit was done on each observed patient’s medical record using a structure chart audit form Figure W4.

After the initial observational data were transcribed, a preliminary practice genogram¹⁶ was drawn, and an initial practice summary was written. The genogram of practice participants, roles, and relationships was initially diagrammed on a white board by a transdisciplinary research team by interviewing the field researcher about the current and past practice clinicians and staff and about the health system and community. The demographics of individuals were recorded, including age, sex, years with the practice, percentage of work effort, and job responsibilities. Additional details included functional and emotional relationships observed in the practice, such as who worked together and any obvious conflicts among members of the practice or with health system or community affiliations. This process enabled the investigators to identify areas of incomplete data so the field researchers could return to fill in missing details.

In addition to the key informant interviews done as part of the observation activities, more formal semistructured individual depth interviews were arranged with each clinician and many of the office staff.²⁵ These interviews consisted of a 30-minute to 1-hour narrative interview in which the respondent was asked open-ended questions designed to elicit in-depth responses Figure W5. Although the major focus of these interviews was on the delivery of preventive services, more general questions were included to understand perspectives on practice process. For example, respondents were asked: “Could you describe for me a typical day for you in this practice?” and “If you believed a change was needed regarding some specific delivery of a service within this office, could you describe the process you would go through to try to get it implemented?” These interviews were audiotaped and transcribed verbatim.

To supplement the observational and interview data, field researchers gathered existing documents and artifacts from the practice. Items such as blank charts and flow sheets, patient schedules, personnel lists, samples of patient education materials and handouts, mission and vision statements, and annual reports were collected and compiled in binders. In some practices, particularly those affiliated with hospital health systems, materials were also available from Web sites. All transcribed interviews and dictated field notes from the practice and encounter observations were imported into FolioViews 4.11 (NextPage, Provo, Utah), a text-base management software program that facilitates coding, searching, and retrieval of large computerized text files.²⁶

Emerging Design Decisions

Midway through data collection at the first practice, the advisory committee met to review the data and discuss any concerns. The advisory committee identified a number of emerging hypotheses related to complexity science concepts that were used to guide subsequent data collection and management. It was deemed particularly important to identify “attractors”—factors in the practice and in the larger environment that influenced the structure and function of the practice as an organization.⁶ For example, an attractor might be a particular burning interest of one of the physicians, an expectation of the local hospital systems, or a dominant demographic characteristic of the patients being served.¹³ An expanded systemic model of primary care Figure 1 was articulated that characterized 6 core areas for data collection: (1) patient perceptions and behavior, (2) physician perceptions and behavior, (3) encounter structures and processes, (4) practice structures and processes, (5) community characteristics, and (6) the larger health system. This model identified the need for additional data on the community context and patient experience. Checklists were revised and field researchers were asked to spend more time gathering data about the community. It was also apparent that accurate calculation of certain preventive service delivery rates would require patient input and a larger sample size. For example, a patient exit card

Figure W6 was developed to ascertain self-reported tobacco use status (for all patients) and use of obstetrics and gynecology services and history of hysterectomy (for women). These data were used to determine a patient’s eligibility for tobacco cessation counseling and Papanicolaou tests, respectively. Observation of 30 encounters with each clinician was done to increase the stability for calculating rates for common preventive service recommendations.

A larger issue emerged with the discovery that, after contributing data over the course of weeks or months, members of the practices desired feedback in a timely manner. Practice clinicians and staff were very interested in how they were doing and asked when they would be receiving a report of our results. They did not want to wait 3 years for the completion of the study. Although ongoing analyses were anticipated, these had primarily been designed to ensure completeness of data and to provide feedback to field researchers on areas where clarifications were needed. In response to the emergent desire for feedback, the team generated rapid-turnaround summary reports for each practice. A summary report template was designed to present the descriptive details of the practice, including the practice genogram and a summary of the strengths and weaknesses of the practice’s prevention approach. The final page in the report provided the practice with a series of questions or points for self-reflection that often included process questions, such as “How can this organization become a team?” or “How can this practice deliver preventive services more consistently?” These reports were shared interactively with practices at a debriefing meeting within 2 to 3 months of completing data collection at each practice.

The feedback meetings provided an important opportunity to check the validity of the researchers’ analyses by comparing them with the perspectives of the practice participants. In all the practices, the response was a strong overall validation of the research team’s interpretation of the practice and its structures and processes. During the feedback presentations, the practice physicians and staff consistently made comments like, “Wow, did you ever get us,” or “This is like looking in a mirror.” In a number of sessions, the participants mentioned that the report raised issues about which they were vaguely aware and that the findings were stimulating considerable self-reflection. In several practices, the physicians disclosed that they would be taking actions in the future to modify some of the deficiencies the reports uncovered.

The next modification came as the data were being collected simultaneously in the second and third practices. We realized that despite our efforts to be nondisruptive, participating in the project required extra effort on the part of the practice. Each practice therefore received partial compensation in the form of a $500 certificate for the purchase of books or equipment.

After completing data collection at several practices, it was apparent that patients’ perspectives were still under-represented and that this was limiting the understanding of the practice. To gain further insights into patients’ experiences, beginning with the sixth practice we adopted the patient path approach described by Pommerenke and Dietrich,²⁷ following patients from the time they walk into the practice until the time they leave, using a patient path form for recording activities at different stages of the visit Figure W7. Additional brief open-ended interviews were conducted in the waiting room or examination rooms while patients were waiting.

Although we had asked the research nurses to be more thorough in their descriptions of the community, data from the community and health system were still incomplete. This became even more pronounced when studying practices that were part of health systems. Therefore, beginning with the sixth practice, we included in-depth interviews with individuals from health systems (eg, regional managers and medical directors). A further refinement came with the use of community key informant trees, a systematic process of identifying and interviewing members of the local community surrounding the practice.^24,28 These interviews of patients, church leaders, and other individuals from the community began with the ninth practice.

Once all the modifications were incorporated, the final case study design provided data at each of the 6 levels as shown in the Table 1. Particularly detailed data were available at the clinician, encounter, and practice levels. For example, at the clinician level the data included perceptions of roles as ascertained through the in-depth interviews, as well as actual behaviors recorded in the encounter field notes and chart audits. Insights on the structures and process of the practice were obtained through unstructured observations of the practice, structured checklists, written documents, and interviews. Supporting data were collected on patients’ perspectives, the community, and the health system that provided contexts for the practice case studies.

Discussion

The complexity of primary care practices is best understood from multiple perspectives,²⁹ a principle that guided the initial selection of a multimethod comparative case study design for this investigation. The MAP that emerged from this comparative case study design has a number of strengths and weaknesses. A particular asset of the design was our ability to investigate specific phenomena within their context rather than isolated from it.²⁹ The design also encouraged the investigators to pursue emerging insights, thus informing multiple perspectives that might not have otherwise been considered, although this may be somewhat limited by the purposeful sampling strategy that focuses on maximizing information about a particular topic.

Limitations

A limitation for broader implementation of this research design is the intensity of data collection and analysis, which are difficult to accomplish without considerable resources and a research team with diverse skills. There might also be some concerns about the ways that the data collection process alters practice behavior; however, the prolonged observational time frame and the multiple data sources for “triangulation” are designed to limit any potential Hawthorne effect. That is, by collecting and systematically comparing data from multiple sources, including direct observation, different forms of interviews, and existing documents, the investigators were able to identify inconsistencies in patterns of behavior.^17,19

Although the data collection, analysis, and feedback process appeared to increase a practice’s self-reflection, our study limited the input of patient and practice participants in the design, analysis, and interpretation and thus does not approach the participatory research paradigm espoused by McCauley and colleagues.³⁰ Still, our study moved from being primarily observational descriptive research into a more collaborative and interventional project, in part at the request of the participants. This suggests a model and method for future research in the arena of health care process and outcome improvement with the practices as collaborators. The MAP characterized in this paper offers a means for simultaneously describing, understanding, and improving the richly complex and varied processes and outcomes of primary care. By more actively engaging the practices in the research process, the MAP also points toward a new more collaborative model of practice-based research.

Conclusions

The comprehensive data in the P&CD study provide a unique opportunity to understand and describe multiple perspectives from the clinician, patient, encounter, practice, community, and health system spheres. Each of the papers in this issue of JFP used some of these comprehensive data to study one or more of these spheres. For example, the encounter field notes were the primary source of data for exploring how “family” presents in encounters³¹Several of the authors used subsets of patients, including patients presenting with acute respiratory track infections,³³ smokers,³⁴ and frequent attenders.³⁵ These authors each supplemented the encounter field notes with data from the medical record reviews, medical record field notes, and patient exit cards. The complete data set including practice field notes, practice genograms, physician and staff interviews, office environment checklists, and encounter field notes were used to describe staff training, roles, and functions.³⁶ This is only a part of the research adventure available in this type of data. We hope many others will join in the excitement.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Research Center grant from the American Academy of Family Physicians. Drs Crabtree, Miller and Stange are associated with the Center for Research in Family Practice and Primary Care, Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 18 practices, without whose participation this study would not have been possible. We also wish to thank Connie Gibbs and Jen Rouse, who spent many hours in the practices collecting data, and Diane Dodendorf and Jason Lebsack, who spent countless hours coordinating transcription and data management activities, for their dedicated work. We are especially indebted to Mary McAndrews, who transcribed hundreds of taped interviews and dictated field notes. The ongoing analyses that ensured the quality and comprehensiveness of the data were made possible through the dedicated work of Helen McIlvain, PhD; Jeffrey Susman, MD; Virginia Aita, PhD; Kristine McVea, MD; Elisabeth Backer, MD; Paul Turner, PhD; and Louis Pol, PhD. Finally, we thank the members of the advisory committee: Valerie Gilchrist, MD; Paul Nutting, MD, MPH; Carlos Jaén, MD, PhD; Kurt Stange, MD, PhD; William Miller, MD, MA; Reuben McDaniel, PhD; and Ruth Anderson, RN, PhD.

Clinicians, researchers, and policymakers now recognize that multiple competing demands1 and opportunities² are simultaneously affecting the physicians, staff, and patients within primary care practices. Our current understanding of outpatient practice is largely based on administrative databases, national surveys, and medical record reviews, with additional insights from surveys of patients or clinicians. These data generally are not designed to capture the richness of the content and context that is needed to better understand the realities and complexities of practice.^3-6 The underlying premise of The Prevention and Competing Demands in Primary Care (P&CD) Study is that efforts to change practice should be preceded by efforts to understand it.^2,7 The explicit goal of this study is to understand practice structure and process, including details of patients, physicians, staff, and clinical encounters; the practice as an organization; and its relationship to the larger community and health system.

In this paper we describe a dynamic observational multimethod assessment process (MAP) that can be used to understand the complex reality of primary care practice. MAP is based on a multimethod comparative case study design^8,9 that integrates elements of epidemiology with methods derived from the qualitative traditions of anthropology and sociology and relies most heavily on qualitative observation and interviewing methods. Studies of this type require an iterative data collection and analysis approach that evolves over time so that new methods can be introduced as the investigators gain a better understanding of important issues. A major strength of our study design was that it allowed hypotheses and insights gained from participants and from ongoing analyses to be integrated into the ongoing investigation.

The study’s primary research questions related to how practice characteristics affect preventive service delivery. Thus, the research design included: (1) an examination of the organizational contexts that support preventive services, (2) an examination of the competing demands imposed by carrying out clinical prevention and illness care in clinical encounters and in the practice, (3) a comparison of the approaches used by practices with high versus low intensity of preventive services delivered to eligible patients, and (4) an examination of approaches used to deliver different types of preventive services. Although the particular focus was on preventive services, the rich MAP allowed pursuit of other research topics that are presented in this issue of JFP.

This article describes the evolutionary methods of the P&CD study, focusing on how data were collected to ensure that sufficient details were available to understand a practice’s values, structures, and processes.*

Emergent research design

The P&CD study was conceived in 1994 to be an in-depth follow-up of insights from the Direct Observation of Primary Care (DOPC) Study that was just getting under way in northeastern Ohio.⁵ The DOPC Study provided a largely quantitative assessment of patients, physicians, encounters, and practices using patient questionnaires, physician surveys, medical record audits, and direct observation of clinical encounters using the Davis Observation Code.¹⁰ That study’s initial findings were presented in the May 1998 theme issue of JFP, and the study processes have recently been described.¹¹ Details of the DOPC methods have been published elsewhere.^4,5,12

Although the initial design allowed the DOPC research nurses to collect brief observational notes, the intensity of the quantitative data collection limited the scope of the study’s qualitative data for understanding details of the practice’s organization and the competing demands within clinical encounters. As a consequence, the P&CD study was designed to provide more in-depth description and understanding of the competing demands of family practice, and in particular, to evaluate factors affecting preventive services delivery using a comparative case study design and a MAP.

A key feature of the P&CD study design was an openness to the integration of emerging insights into the data collection protocol. For example, preliminary analyses of the DOPC data¹³ and other ongoing studies¹⁴ led to the discovery that complexity science was valuable for explaining the dynamics of office systems⁶ and needed to be incorporated into the design. (Complexity science is the study of systems that are characterized by nonlinear dynamics and emergent properties; it emphasizes the need to understand the interrelationships of the whole system and not just collect data about the parts.¹⁵) The investigators also developed new ways to display the relationships among physicians and staff in the practices using “practice genograms.”¹⁶ The practice genogram is a diagram of the functional and interpersonal relationships among the clinicians, support staff, and other people and organizations interacting with the practice. Throughout the project and consistent with the standards of qualitative research design,17 there were continued modifications and enhancements in the data collection and analysis strategies in response to insights that were emerging from ongoing analyses and interpretation of the data.

An important feature of the project was the development of an advisory committee of consultants and co-investigators that convened annually to provide multidisciplinary input, review results, and provide feedback. The advisory committee included academic representatives with expertise in nursing, health education, women’s health, minority health, and public policy. Two additional members were added to the project to provide expertise into the study of organizations as complex systems. The annual reviews by the advisory committee led to significant changes in the research design while the study was ongoing.

Practice Sample

Beginning in late 1996, we drew from 91 practices in Nebraska that had been randomly selected to participate in an earlier study on tobacco prevention and cessation.¹⁸ Initially a sample of 10 practices was purposefully selected^19,20 using an iterative process to represent a range in size (small and large), geographic location (urban, suburban, and rural), and rate of delivery of tobacco-related preventive services. Preliminary analyses of these 10 initial practices provided a summary of preventive health delivery strategies in primary care practices and a description of competing demands that enhanced or limited these strategies. To confirm or refute the emerging insights from the original 10 practices, 8 additional practices were selected for further data collection during the second and third years of the study. The sampling strategy in years 2 and 3 ensured that at least 2 practices each from several major regional hospital health systems were included and allowed us to assess emerging hypotheses about the importance of health system context for understanding community practices.

The practices were recruited by contacting one of the physicians to solicit participation; only those in which all family physicians in the practice agreed to participate were included in the study. Twenty-three practices were contacted; all physicians in 18 agreed to participate (78%).

Core Data Collection Methods

Data were collected by trained field researchers who spent 4 weeks or more taking notes at each practice while observing the practice and clinical encounters, conducting informal key informant interviews of staff, collecting office documents, and auditing charts of patients whose encounters were observed. Within each practice, data collection occurred in stages, with a short break after the initial week or 2 of observation to allow preliminary analyses to inform additional data collection.

Observations at the practice level were recorded in a combination of structured observational checklists, unstructured dictated field notes,^21-23 and key informant interviews.²⁴ Detailed floor plans of the practice were used to identify where particular activities occurred and where individual practice participants worked. Each day at the practice, the field researcher took short notes or “jottings” and dictated expanded field notes in the evening.²³ A template of topics was used periodically to ensure that important aspects of the practice were not being overlooked. The template included lists of features of the community, practice, staff, and patients that the researchers saw as important Figure W1.*¹³ A 3-page structured practice environment checklist was adapted from earlier work on the DOPC project¹¹ and included a wide range of practice characteristics and functions, including items such as the number and training of staff, counseling options offered, and management of telephone calls and referrals Figure W2a. This checklist also served as a detailed reminder to the research nurse of topics to be included in the field notes. Throughout the time the field researchers were recording field notes and filling in the checklist, they opportunistically asked clinicians or staff informal key informant questions for confirmation or clarification.

Consecutive patients for each clinician in a practice were approached with a goal of recruiting 30 patients who would consent to have the field researcher observe their visit. This generally required approaching 35 to 40 patients. Because some clinicians worked part-time or were not consistently in the practice, it was not always possible to observe 30 visits. After explaining the study and gaining signed informed consent from patients who agreed to participate, the field researchers observed the outpatient visit as unobtrusively as possible. A 1-page structured encounter checklist Figure W3 that was also modified from the DOPC study11 provided blanks for noting the reason for the visit, chief complaints, and final diagnoses, and for indicating whether any of approximately 100 preventive services were ordered or delivered. Space was provided at the bottom of the form for recording notes that were later used to dictate a re-creation of the encounter. At a later time, a chart audit was done on each observed patient’s medical record using a structure chart audit form Figure W4.

After the initial observational data were transcribed, a preliminary practice genogram¹⁶ was drawn, and an initial practice summary was written. The genogram of practice participants, roles, and relationships was initially diagrammed on a white board by a transdisciplinary research team by interviewing the field researcher about the current and past practice clinicians and staff and about the health system and community. The demographics of individuals were recorded, including age, sex, years with the practice, percentage of work effort, and job responsibilities. Additional details included functional and emotional relationships observed in the practice, such as who worked together and any obvious conflicts among members of the practice or with health system or community affiliations. This process enabled the investigators to identify areas of incomplete data so the field researchers could return to fill in missing details.

In addition to the key informant interviews done as part of the observation activities, more formal semistructured individual depth interviews were arranged with each clinician and many of the office staff.²⁵ These interviews consisted of a 30-minute to 1-hour narrative interview in which the respondent was asked open-ended questions designed to elicit in-depth responses Figure W5. Although the major focus of these interviews was on the delivery of preventive services, more general questions were included to understand perspectives on practice process. For example, respondents were asked: “Could you describe for me a typical day for you in this practice?” and “If you believed a change was needed regarding some specific delivery of a service within this office, could you describe the process you would go through to try to get it implemented?” These interviews were audiotaped and transcribed verbatim.

To supplement the observational and interview data, field researchers gathered existing documents and artifacts from the practice. Items such as blank charts and flow sheets, patient schedules, personnel lists, samples of patient education materials and handouts, mission and vision statements, and annual reports were collected and compiled in binders. In some practices, particularly those affiliated with hospital health systems, materials were also available from Web sites. All transcribed interviews and dictated field notes from the practice and encounter observations were imported into FolioViews 4.11 (NextPage, Provo, Utah), a text-base management software program that facilitates coding, searching, and retrieval of large computerized text files.²⁶

Emerging Design Decisions

Midway through data collection at the first practice, the advisory committee met to review the data and discuss any concerns. The advisory committee identified a number of emerging hypotheses related to complexity science concepts that were used to guide subsequent data collection and management. It was deemed particularly important to identify “attractors”—factors in the practice and in the larger environment that influenced the structure and function of the practice as an organization.⁶ For example, an attractor might be a particular burning interest of one of the physicians, an expectation of the local hospital systems, or a dominant demographic characteristic of the patients being served.¹³ An expanded systemic model of primary care Figure 1 was articulated that characterized 6 core areas for data collection: (1) patient perceptions and behavior, (2) physician perceptions and behavior, (3) encounter structures and processes, (4) practice structures and processes, (5) community characteristics, and (6) the larger health system. This model identified the need for additional data on the community context and patient experience. Checklists were revised and field researchers were asked to spend more time gathering data about the community. It was also apparent that accurate calculation of certain preventive service delivery rates would require patient input and a larger sample size. For example, a patient exit card

Figure W6 was developed to ascertain self-reported tobacco use status (for all patients) and use of obstetrics and gynecology services and history of hysterectomy (for women). These data were used to determine a patient’s eligibility for tobacco cessation counseling and Papanicolaou tests, respectively. Observation of 30 encounters with each clinician was done to increase the stability for calculating rates for common preventive service recommendations.

A larger issue emerged with the discovery that, after contributing data over the course of weeks or months, members of the practices desired feedback in a timely manner. Practice clinicians and staff were very interested in how they were doing and asked when they would be receiving a report of our results. They did not want to wait 3 years for the completion of the study. Although ongoing analyses were anticipated, these had primarily been designed to ensure completeness of data and to provide feedback to field researchers on areas where clarifications were needed. In response to the emergent desire for feedback, the team generated rapid-turnaround summary reports for each practice. A summary report template was designed to present the descriptive details of the practice, including the practice genogram and a summary of the strengths and weaknesses of the practice’s prevention approach. The final page in the report provided the practice with a series of questions or points for self-reflection that often included process questions, such as “How can this organization become a team?” or “How can this practice deliver preventive services more consistently?” These reports were shared interactively with practices at a debriefing meeting within 2 to 3 months of completing data collection at each practice.

The feedback meetings provided an important opportunity to check the validity of the researchers’ analyses by comparing them with the perspectives of the practice participants. In all the practices, the response was a strong overall validation of the research team’s interpretation of the practice and its structures and processes. During the feedback presentations, the practice physicians and staff consistently made comments like, “Wow, did you ever get us,” or “This is like looking in a mirror.” In a number of sessions, the participants mentioned that the report raised issues about which they were vaguely aware and that the findings were stimulating considerable self-reflection. In several practices, the physicians disclosed that they would be taking actions in the future to modify some of the deficiencies the reports uncovered.

The next modification came as the data were being collected simultaneously in the second and third practices. We realized that despite our efforts to be nondisruptive, participating in the project required extra effort on the part of the practice. Each practice therefore received partial compensation in the form of a $500 certificate for the purchase of books or equipment.

After completing data collection at several practices, it was apparent that patients’ perspectives were still under-represented and that this was limiting the understanding of the practice. To gain further insights into patients’ experiences, beginning with the sixth practice we adopted the patient path approach described by Pommerenke and Dietrich,²⁷ following patients from the time they walk into the practice until the time they leave, using a patient path form for recording activities at different stages of the visit Figure W7. Additional brief open-ended interviews were conducted in the waiting room or examination rooms while patients were waiting.

Although we had asked the research nurses to be more thorough in their descriptions of the community, data from the community and health system were still incomplete. This became even more pronounced when studying practices that were part of health systems. Therefore, beginning with the sixth practice, we included in-depth interviews with individuals from health systems (eg, regional managers and medical directors). A further refinement came with the use of community key informant trees, a systematic process of identifying and interviewing members of the local community surrounding the practice.^24,28 These interviews of patients, church leaders, and other individuals from the community began with the ninth practice.

Once all the modifications were incorporated, the final case study design provided data at each of the 6 levels as shown in the Table 1. Particularly detailed data were available at the clinician, encounter, and practice levels. For example, at the clinician level the data included perceptions of roles as ascertained through the in-depth interviews, as well as actual behaviors recorded in the encounter field notes and chart audits. Insights on the structures and process of the practice were obtained through unstructured observations of the practice, structured checklists, written documents, and interviews. Supporting data were collected on patients’ perspectives, the community, and the health system that provided contexts for the practice case studies.

Discussion

The complexity of primary care practices is best understood from multiple perspectives,²⁹ a principle that guided the initial selection of a multimethod comparative case study design for this investigation. The MAP that emerged from this comparative case study design has a number of strengths and weaknesses. A particular asset of the design was our ability to investigate specific phenomena within their context rather than isolated from it.²⁹ The design also encouraged the investigators to pursue emerging insights, thus informing multiple perspectives that might not have otherwise been considered, although this may be somewhat limited by the purposeful sampling strategy that focuses on maximizing information about a particular topic.

Limitations

A limitation for broader implementation of this research design is the intensity of data collection and analysis, which are difficult to accomplish without considerable resources and a research team with diverse skills. There might also be some concerns about the ways that the data collection process alters practice behavior; however, the prolonged observational time frame and the multiple data sources for “triangulation” are designed to limit any potential Hawthorne effect. That is, by collecting and systematically comparing data from multiple sources, including direct observation, different forms of interviews, and existing documents, the investigators were able to identify inconsistencies in patterns of behavior.^17,19

Although the data collection, analysis, and feedback process appeared to increase a practice’s self-reflection, our study limited the input of patient and practice participants in the design, analysis, and interpretation and thus does not approach the participatory research paradigm espoused by McCauley and colleagues.³⁰ Still, our study moved from being primarily observational descriptive research into a more collaborative and interventional project, in part at the request of the participants. This suggests a model and method for future research in the arena of health care process and outcome improvement with the practices as collaborators. The MAP characterized in this paper offers a means for simultaneously describing, understanding, and improving the richly complex and varied processes and outcomes of primary care. By more actively engaging the practices in the research process, the MAP also points toward a new more collaborative model of practice-based research.

Conclusions

The comprehensive data in the P&CD study provide a unique opportunity to understand and describe multiple perspectives from the clinician, patient, encounter, practice, community, and health system spheres. Each of the papers in this issue of JFP used some of these comprehensive data to study one or more of these spheres. For example, the encounter field notes were the primary source of data for exploring how “family” presents in encounters³¹Several of the authors used subsets of patients, including patients presenting with acute respiratory track infections,³³ smokers,³⁴ and frequent attenders.³⁵ These authors each supplemented the encounter field notes with data from the medical record reviews, medical record field notes, and patient exit cards. The complete data set including practice field notes, practice genograms, physician and staff interviews, office environment checklists, and encounter field notes were used to describe staff training, roles, and functions.³⁶ This is only a part of the research adventure available in this type of data. We hope many others will join in the excitement.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Research Center grant from the American Academy of Family Physicians. Drs Crabtree, Miller and Stange are associated with the Center for Research in Family Practice and Primary Care, Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 18 practices, without whose participation this study would not have been possible. We also wish to thank Connie Gibbs and Jen Rouse, who spent many hours in the practices collecting data, and Diane Dodendorf and Jason Lebsack, who spent countless hours coordinating transcription and data management activities, for their dedicated work. We are especially indebted to Mary McAndrews, who transcribed hundreds of taped interviews and dictated field notes. The ongoing analyses that ensured the quality and comprehensiveness of the data were made possible through the dedicated work of Helen McIlvain, PhD; Jeffrey Susman, MD; Virginia Aita, PhD; Kristine McVea, MD; Elisabeth Backer, MD; Paul Turner, PhD; and Louis Pol, PhD. Finally, we thank the members of the advisory committee: Valerie Gilchrist, MD; Paul Nutting, MD, MPH; Carlos Jaén, MD, PhD; Kurt Stange, MD, PhD; William Miller, MD, MA; Reuben McDaniel, PhD; and Ruth Anderson, RN, PhD.

Clinicians, researchers, and policymakers now recognize that multiple competing demands1 and opportunities² are simultaneously affecting the physicians, staff, and patients within primary care practices. Our current understanding of outpatient practice is largely based on administrative databases, national surveys, and medical record reviews, with additional insights from surveys of patients or clinicians. These data generally are not designed to capture the richness of the content and context that is needed to better understand the realities and complexities of practice.^3-6 The underlying premise of The Prevention and Competing Demands in Primary Care (P&CD) Study is that efforts to change practice should be preceded by efforts to understand it.^2,7 The explicit goal of this study is to understand practice structure and process, including details of patients, physicians, staff, and clinical encounters; the practice as an organization; and its relationship to the larger community and health system.

In this paper we describe a dynamic observational multimethod assessment process (MAP) that can be used to understand the complex reality of primary care practice. MAP is based on a multimethod comparative case study design^8,9 that integrates elements of epidemiology with methods derived from the qualitative traditions of anthropology and sociology and relies most heavily on qualitative observation and interviewing methods. Studies of this type require an iterative data collection and analysis approach that evolves over time so that new methods can be introduced as the investigators gain a better understanding of important issues. A major strength of our study design was that it allowed hypotheses and insights gained from participants and from ongoing analyses to be integrated into the ongoing investigation.

The study’s primary research questions related to how practice characteristics affect preventive service delivery. Thus, the research design included: (1) an examination of the organizational contexts that support preventive services, (2) an examination of the competing demands imposed by carrying out clinical prevention and illness care in clinical encounters and in the practice, (3) a comparison of the approaches used by practices with high versus low intensity of preventive services delivered to eligible patients, and (4) an examination of approaches used to deliver different types of preventive services. Although the particular focus was on preventive services, the rich MAP allowed pursuit of other research topics that are presented in this issue of JFP.

This article describes the evolutionary methods of the P&CD study, focusing on how data were collected to ensure that sufficient details were available to understand a practice’s values, structures, and processes.*

Emergent research design

The P&CD study was conceived in 1994 to be an in-depth follow-up of insights from the Direct Observation of Primary Care (DOPC) Study that was just getting under way in northeastern Ohio.⁵ The DOPC Study provided a largely quantitative assessment of patients, physicians, encounters, and practices using patient questionnaires, physician surveys, medical record audits, and direct observation of clinical encounters using the Davis Observation Code.¹⁰ That study’s initial findings were presented in the May 1998 theme issue of JFP, and the study processes have recently been described.¹¹ Details of the DOPC methods have been published elsewhere.^4,5,12

Although the initial design allowed the DOPC research nurses to collect brief observational notes, the intensity of the quantitative data collection limited the scope of the study’s qualitative data for understanding details of the practice’s organization and the competing demands within clinical encounters. As a consequence, the P&CD study was designed to provide more in-depth description and understanding of the competing demands of family practice, and in particular, to evaluate factors affecting preventive services delivery using a comparative case study design and a MAP.

A key feature of the P&CD study design was an openness to the integration of emerging insights into the data collection protocol. For example, preliminary analyses of the DOPC data¹³ and other ongoing studies¹⁴ led to the discovery that complexity science was valuable for explaining the dynamics of office systems⁶ and needed to be incorporated into the design. (Complexity science is the study of systems that are characterized by nonlinear dynamics and emergent properties; it emphasizes the need to understand the interrelationships of the whole system and not just collect data about the parts.¹⁵) The investigators also developed new ways to display the relationships among physicians and staff in the practices using “practice genograms.”¹⁶ The practice genogram is a diagram of the functional and interpersonal relationships among the clinicians, support staff, and other people and organizations interacting with the practice. Throughout the project and consistent with the standards of qualitative research design,17 there were continued modifications and enhancements in the data collection and analysis strategies in response to insights that were emerging from ongoing analyses and interpretation of the data.

An important feature of the project was the development of an advisory committee of consultants and co-investigators that convened annually to provide multidisciplinary input, review results, and provide feedback. The advisory committee included academic representatives with expertise in nursing, health education, women’s health, minority health, and public policy. Two additional members were added to the project to provide expertise into the study of organizations as complex systems. The annual reviews by the advisory committee led to significant changes in the research design while the study was ongoing.

Practice Sample

Beginning in late 1996, we drew from 91 practices in Nebraska that had been randomly selected to participate in an earlier study on tobacco prevention and cessation.¹⁸ Initially a sample of 10 practices was purposefully selected^19,20 using an iterative process to represent a range in size (small and large), geographic location (urban, suburban, and rural), and rate of delivery of tobacco-related preventive services. Preliminary analyses of these 10 initial practices provided a summary of preventive health delivery strategies in primary care practices and a description of competing demands that enhanced or limited these strategies. To confirm or refute the emerging insights from the original 10 practices, 8 additional practices were selected for further data collection during the second and third years of the study. The sampling strategy in years 2 and 3 ensured that at least 2 practices each from several major regional hospital health systems were included and allowed us to assess emerging hypotheses about the importance of health system context for understanding community practices.

The practices were recruited by contacting one of the physicians to solicit participation; only those in which all family physicians in the practice agreed to participate were included in the study. Twenty-three practices were contacted; all physicians in 18 agreed to participate (78%).

Core Data Collection Methods

Data were collected by trained field researchers who spent 4 weeks or more taking notes at each practice while observing the practice and clinical encounters, conducting informal key informant interviews of staff, collecting office documents, and auditing charts of patients whose encounters were observed. Within each practice, data collection occurred in stages, with a short break after the initial week or 2 of observation to allow preliminary analyses to inform additional data collection.

Observations at the practice level were recorded in a combination of structured observational checklists, unstructured dictated field notes,^21-23 and key informant interviews.²⁴ Detailed floor plans of the practice were used to identify where particular activities occurred and where individual practice participants worked. Each day at the practice, the field researcher took short notes or “jottings” and dictated expanded field notes in the evening.²³ A template of topics was used periodically to ensure that important aspects of the practice were not being overlooked. The template included lists of features of the community, practice, staff, and patients that the researchers saw as important Figure W1.*¹³ A 3-page structured practice environment checklist was adapted from earlier work on the DOPC project¹¹ and included a wide range of practice characteristics and functions, including items such as the number and training of staff, counseling options offered, and management of telephone calls and referrals Figure W2a. This checklist also served as a detailed reminder to the research nurse of topics to be included in the field notes. Throughout the time the field researchers were recording field notes and filling in the checklist, they opportunistically asked clinicians or staff informal key informant questions for confirmation or clarification.

Consecutive patients for each clinician in a practice were approached with a goal of recruiting 30 patients who would consent to have the field researcher observe their visit. This generally required approaching 35 to 40 patients. Because some clinicians worked part-time or were not consistently in the practice, it was not always possible to observe 30 visits. After explaining the study and gaining signed informed consent from patients who agreed to participate, the field researchers observed the outpatient visit as unobtrusively as possible. A 1-page structured encounter checklist Figure W3 that was also modified from the DOPC study11 provided blanks for noting the reason for the visit, chief complaints, and final diagnoses, and for indicating whether any of approximately 100 preventive services were ordered or delivered. Space was provided at the bottom of the form for recording notes that were later used to dictate a re-creation of the encounter. At a later time, a chart audit was done on each observed patient’s medical record using a structure chart audit form Figure W4.

After the initial observational data were transcribed, a preliminary practice genogram¹⁶ was drawn, and an initial practice summary was written. The genogram of practice participants, roles, and relationships was initially diagrammed on a white board by a transdisciplinary research team by interviewing the field researcher about the current and past practice clinicians and staff and about the health system and community. The demographics of individuals were recorded, including age, sex, years with the practice, percentage of work effort, and job responsibilities. Additional details included functional and emotional relationships observed in the practice, such as who worked together and any obvious conflicts among members of the practice or with health system or community affiliations. This process enabled the investigators to identify areas of incomplete data so the field researchers could return to fill in missing details.

In addition to the key informant interviews done as part of the observation activities, more formal semistructured individual depth interviews were arranged with each clinician and many of the office staff.²⁵ These interviews consisted of a 30-minute to 1-hour narrative interview in which the respondent was asked open-ended questions designed to elicit in-depth responses Figure W5. Although the major focus of these interviews was on the delivery of preventive services, more general questions were included to understand perspectives on practice process. For example, respondents were asked: “Could you describe for me a typical day for you in this practice?” and “If you believed a change was needed regarding some specific delivery of a service within this office, could you describe the process you would go through to try to get it implemented?” These interviews were audiotaped and transcribed verbatim.

To supplement the observational and interview data, field researchers gathered existing documents and artifacts from the practice. Items such as blank charts and flow sheets, patient schedules, personnel lists, samples of patient education materials and handouts, mission and vision statements, and annual reports were collected and compiled in binders. In some practices, particularly those affiliated with hospital health systems, materials were also available from Web sites. All transcribed interviews and dictated field notes from the practice and encounter observations were imported into FolioViews 4.11 (NextPage, Provo, Utah), a text-base management software program that facilitates coding, searching, and retrieval of large computerized text files.²⁶

Emerging Design Decisions

Midway through data collection at the first practice, the advisory committee met to review the data and discuss any concerns. The advisory committee identified a number of emerging hypotheses related to complexity science concepts that were used to guide subsequent data collection and management. It was deemed particularly important to identify “attractors”—factors in the practice and in the larger environment that influenced the structure and function of the practice as an organization.⁶ For example, an attractor might be a particular burning interest of one of the physicians, an expectation of the local hospital systems, or a dominant demographic characteristic of the patients being served.¹³ An expanded systemic model of primary care Figure 1 was articulated that characterized 6 core areas for data collection: (1) patient perceptions and behavior, (2) physician perceptions and behavior, (3) encounter structures and processes, (4) practice structures and processes, (5) community characteristics, and (6) the larger health system. This model identified the need for additional data on the community context and patient experience. Checklists were revised and field researchers were asked to spend more time gathering data about the community. It was also apparent that accurate calculation of certain preventive service delivery rates would require patient input and a larger sample size. For example, a patient exit card

Figure W6 was developed to ascertain self-reported tobacco use status (for all patients) and use of obstetrics and gynecology services and history of hysterectomy (for women). These data were used to determine a patient’s eligibility for tobacco cessation counseling and Papanicolaou tests, respectively. Observation of 30 encounters with each clinician was done to increase the stability for calculating rates for common preventive service recommendations.

A larger issue emerged with the discovery that, after contributing data over the course of weeks or months, members of the practices desired feedback in a timely manner. Practice clinicians and staff were very interested in how they were doing and asked when they would be receiving a report of our results. They did not want to wait 3 years for the completion of the study. Although ongoing analyses were anticipated, these had primarily been designed to ensure completeness of data and to provide feedback to field researchers on areas where clarifications were needed. In response to the emergent desire for feedback, the team generated rapid-turnaround summary reports for each practice. A summary report template was designed to present the descriptive details of the practice, including the practice genogram and a summary of the strengths and weaknesses of the practice’s prevention approach. The final page in the report provided the practice with a series of questions or points for self-reflection that often included process questions, such as “How can this organization become a team?” or “How can this practice deliver preventive services more consistently?” These reports were shared interactively with practices at a debriefing meeting within 2 to 3 months of completing data collection at each practice.

The feedback meetings provided an important opportunity to check the validity of the researchers’ analyses by comparing them with the perspectives of the practice participants. In all the practices, the response was a strong overall validation of the research team’s interpretation of the practice and its structures and processes. During the feedback presentations, the practice physicians and staff consistently made comments like, “Wow, did you ever get us,” or “This is like looking in a mirror.” In a number of sessions, the participants mentioned that the report raised issues about which they were vaguely aware and that the findings were stimulating considerable self-reflection. In several practices, the physicians disclosed that they would be taking actions in the future to modify some of the deficiencies the reports uncovered.

The next modification came as the data were being collected simultaneously in the second and third practices. We realized that despite our efforts to be nondisruptive, participating in the project required extra effort on the part of the practice. Each practice therefore received partial compensation in the form of a $500 certificate for the purchase of books or equipment.

After completing data collection at several practices, it was apparent that patients’ perspectives were still under-represented and that this was limiting the understanding of the practice. To gain further insights into patients’ experiences, beginning with the sixth practice we adopted the patient path approach described by Pommerenke and Dietrich,²⁷ following patients from the time they walk into the practice until the time they leave, using a patient path form for recording activities at different stages of the visit Figure W7. Additional brief open-ended interviews were conducted in the waiting room or examination rooms while patients were waiting.

Although we had asked the research nurses to be more thorough in their descriptions of the community, data from the community and health system were still incomplete. This became even more pronounced when studying practices that were part of health systems. Therefore, beginning with the sixth practice, we included in-depth interviews with individuals from health systems (eg, regional managers and medical directors). A further refinement came with the use of community key informant trees, a systematic process of identifying and interviewing members of the local community surrounding the practice.^24,28 These interviews of patients, church leaders, and other individuals from the community began with the ninth practice.

Once all the modifications were incorporated, the final case study design provided data at each of the 6 levels as shown in the Table 1. Particularly detailed data were available at the clinician, encounter, and practice levels. For example, at the clinician level the data included perceptions of roles as ascertained through the in-depth interviews, as well as actual behaviors recorded in the encounter field notes and chart audits. Insights on the structures and process of the practice were obtained through unstructured observations of the practice, structured checklists, written documents, and interviews. Supporting data were collected on patients’ perspectives, the community, and the health system that provided contexts for the practice case studies.

Discussion

The complexity of primary care practices is best understood from multiple perspectives,²⁹ a principle that guided the initial selection of a multimethod comparative case study design for this investigation. The MAP that emerged from this comparative case study design has a number of strengths and weaknesses. A particular asset of the design was our ability to investigate specific phenomena within their context rather than isolated from it.²⁹ The design also encouraged the investigators to pursue emerging insights, thus informing multiple perspectives that might not have otherwise been considered, although this may be somewhat limited by the purposeful sampling strategy that focuses on maximizing information about a particular topic.

Limitations

A limitation for broader implementation of this research design is the intensity of data collection and analysis, which are difficult to accomplish without considerable resources and a research team with diverse skills. There might also be some concerns about the ways that the data collection process alters practice behavior; however, the prolonged observational time frame and the multiple data sources for “triangulation” are designed to limit any potential Hawthorne effect. That is, by collecting and systematically comparing data from multiple sources, including direct observation, different forms of interviews, and existing documents, the investigators were able to identify inconsistencies in patterns of behavior.^17,19

Although the data collection, analysis, and feedback process appeared to increase a practice’s self-reflection, our study limited the input of patient and practice participants in the design, analysis, and interpretation and thus does not approach the participatory research paradigm espoused by McCauley and colleagues.³⁰ Still, our study moved from being primarily observational descriptive research into a more collaborative and interventional project, in part at the request of the participants. This suggests a model and method for future research in the arena of health care process and outcome improvement with the practices as collaborators. The MAP characterized in this paper offers a means for simultaneously describing, understanding, and improving the richly complex and varied processes and outcomes of primary care. By more actively engaging the practices in the research process, the MAP also points toward a new more collaborative model of practice-based research.

Conclusions

The comprehensive data in the P&CD study provide a unique opportunity to understand and describe multiple perspectives from the clinician, patient, encounter, practice, community, and health system spheres. Each of the papers in this issue of JFP used some of these comprehensive data to study one or more of these spheres. For example, the encounter field notes were the primary source of data for exploring how “family” presents in encounters³¹Several of the authors used subsets of patients, including patients presenting with acute respiratory track infections,³³ smokers,³⁴ and frequent attenders.³⁵ These authors each supplemented the encounter field notes with data from the medical record reviews, medical record field notes, and patient exit cards. The complete data set including practice field notes, practice genograms, physician and staff interviews, office environment checklists, and encounter field notes were used to describe staff training, roles, and functions.³⁶ This is only a part of the research adventure available in this type of data. We hope many others will join in the excitement.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Research Center grant from the American Academy of Family Physicians. Drs Crabtree, Miller and Stange are associated with the Center for Research in Family Practice and Primary Care, Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 18 practices, without whose participation this study would not have been possible. We also wish to thank Connie Gibbs and Jen Rouse, who spent many hours in the practices collecting data, and Diane Dodendorf and Jason Lebsack, who spent countless hours coordinating transcription and data management activities, for their dedicated work. We are especially indebted to Mary McAndrews, who transcribed hundreds of taped interviews and dictated field notes. The ongoing analyses that ensured the quality and comprehensiveness of the data were made possible through the dedicated work of Helen McIlvain, PhD; Jeffrey Susman, MD; Virginia Aita, PhD; Kristine McVea, MD; Elisabeth Backer, MD; Paul Turner, PhD; and Louis Pol, PhD. Finally, we thank the members of the advisory committee: Valerie Gilchrist, MD; Paul Nutting, MD, MPH; Carlos Jaén, MD, PhD; Kurt Stange, MD, PhD; William Miller, MD, MA; Reuben McDaniel, PhD; and Ruth Anderson, RN, PhD.