Original Research

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.

Approximately 17 million smokers attempt to stop smoking for more than 24 hours every year; only 1.2 million are successful.¹ There is strong evidence that smokers attempting to quit could at least double their chances of success if they were assisted by clinicians using effective behavioral and pharmacologic interventions.² Because 7 of 10 smokers will see a physician each year³ and the majority of these visits are made to primary care physicians,⁴ these physicians have multiple opportunities to assist smokers in their attempts to quit.

Clinicians should follow the “5 A’s” (ask, advise, assess, assist, and arrange) whenever appropriate. The current US Public Health Service smoking cessation clinical practice guideline offers specific directions for clinician intervention for all smokers, recommending a minimum of 3A’s (ask, advise, and assess) at every visit. That is, all smokers should be asked about their current smoking status, advised to quit, and assessed regarding their readiness to change. For smokers willing to quit, 2 additional A’s (assist and arrange follow-up) should be implemented; for smokers not willing to quit, a brief motivational intervention is recommended.²

Although there is a high level of agreement among primary care physicians about their responsibility to assist in tobacco cessation,^5,6 there are significant gaps in practice.^7-9 Reports of physicians’ rates of smoking cessation advice range from 21% to 78%,^7-12 falling short of recommended levels.¹³

A recent direct observation study of community family physicians found that, on average, 25% of smokers were advised to stop smoking.¹⁴ The study also showed that smoking cessation advice was offered during 55% of well care visits and in 32% of chronic illness visits for tobacco-related problems. The average duration of smoking cessation advice was less than 90 seconds. Although the study’s authors were able to assess whether smoking cessation advice occurred during an encounter, limits of the data made it impossible to examine how the particular content of smoking cessation advice was delivered. Similar results were found in a study of direct observation of Australian physicians.¹⁵

For this study, we used direct observation of outpatient visits by smokers to describe the extent of tobacco counseling and the processes by which it was provided. The analyses also explore the contextual factors that influence the provision of smoking cessation counseling. We hypothesized that the low rates of smoking counseling reported in the literature were in part due to the competing demands brought on by the complex agenda of patients presenting with undifferentiated problems.^16,17 We also hypothesized that the current care included missed opportunities to integrate tobacco counseling into the broad primary care agenda.

Methods

The data used for this analysis were collected as part of The Prevention and Competing Demands in Primary Care Study, an in-depth observational study that examined the organizational and clinical structures and process of community-based family practices.Each of 18 purposefully selected practices was studied using a multimethod comparative case study design that involved extensive direct observation of clinical encounters and office systems by field researchers who spent 4 weeks or more in each practice. Field researchers directly observed approximately 30 patient encounters with each of more than 50 clinicians, dictated descriptions of the visits, and audited the medical records of each of these patients. Detailed descriptive field notes documented day-to-day practice operations. Individual depth interviews with each clinician, many of the practice staff, and members of the community were used to obtain different perspectives on the practice. Details of the sampling and data collection are available elsewhere in this issue of JFP.¹⁸

From the exit survey administered to patients, 239 current cigarette smokers (14.7% of the study population) were identified from the 1624 encounters. To minimize observer variation in encounter content, only the narratives of a single research nurse were examined, reducing the sample of current smokers to 123. Only encounters with physicians were selected for analysis, further reducing the sample size to 91.

The research team included 6 members representing a broad range of perspectives, including family medicine, health services research, epidemiology, psychology, anthropology, and sociology. We used an iterative analysis and interpretation process that evolved over time as the team became more familiar with the data.¹⁹ Two immediate objectives were identified: (1) to develop a classification system that could be used to describe how physicians address smoking cessation, and (2) to identify factors that may enhance or impede the degree of adherence to the clinical smoking cessation guideline.² First, the team selected 18 encounters for reading and discussion by all research team members. For each of these encounters, one team member read the narrative out loud, and then the team discussed at length their understanding and assessment of what had taken place. Narrative data from the chart audit and physician interviews were considered as the discussion proceeded. During these discussions, preliminary schemes for classifying and assessing the encounters were developed.

The team was then divided into 3 groups of 2, and each group was assigned approximately 10 encounters for reading and for further development of the initial schemes. To ensure that each group member’s evaluation was independent, each member wrote a description and evaluation of each encounter without having read what the other member had written. The classifications and evaluations were then shared with the other member and the entire research team. Multiple team discussions were used to address differences in interpretation and to identify salient patterns within the data.

After discussing the initial 48 encounters, the remaining 43 encounters were analyzed. The same process of intragroup blind review was followed, and at this point, a nearly complete list of patterns and other important features seen within the encounters was established. Analysis and discussion by the entire research team led to agreement on the classification and evaluation of each of the 91 encounters.

To test the possibility that a single observer may introduce observer bias, the research team analyzed 51 additional clinical encounters with 9 family physicians in 5 different practices by a different research nurse. The 3 teams used the same blinded iterative process. These encounters were reviewed, looking for new patterns of smoking cessation counseling or confirmation of the patterns previously identified.

Results

We observed between 2 and 7 encounters of 20 family physicians in 7 practices Table 1. Five clear patterns were discernable according to the level of tobacco counseling and the type of visit. They represent a hierarchy that ranges from optimal smoking cessation counseling during visits when it was appropriate, to visits during which other agendas were appropriately given higher priority, to deficient missed opportunities. No additional patterns of interaction of smoking cessation counseling were identified among the 51 additional encounters audited.

In nearly half of the visits physicians either followed recommendations (21%), or competing priorities within the encounter reasonably overrode tobacco counseling (24%). In the other encounters tobacco cessation counseling fell short of recommendations, including visits among patients being seen for acute respiratory illnesses or other smoking-related illnesses. This failure often occurred despite the presence of a reminder system that identified the patient as a smoker. In 9% (8 cases) the physicians explicitly told the observing research nurse that they would not address tobacco with a specific patient because of a preconception that the patient would not respond.

Patterns of Tobacco Counseling

Good counseling
Good quality cessation counseling occurred in 21% of the encounters, during which physicians offered appropriate brief interventions depending on patients’ willingness to quit at that visit. Three levels of intervention were discernible within this first pattern. The 5A’s occurred when patients requested help, emphatically said “yes” when asked if they were interested in quitting, or when they responded positively to the physician’s description of pharmacologic options to help quit smoking. Patients were offered only 3A’s if they indicated they were not ready to quit by explicitly saying so or by staying quiet after an inquiry about their willingness to quit. Eleven physicians (55%) had at least 1 encounter with a smoker in which the physicians demonstrated good quality smoking cessation intervention, indicating that they had the knowledge and skill to provide recommended smoking cessation strategies.

Competing demands
Another common pattern was when a smoking cessation agenda was appropriately overridden by higher priorities. This occurred in 24% of the encounters. These were visits during which the physician-patient interaction was less straightforward than simply history taking, diagnosis, and treatment. In 10 encounters the top priority was alleviation of acute pain. Examples included abdominal pain, chest pain, back pain related to pyelonephritis, and severe rib pain after trauma. During 6 encounters patients were experiencing psychological distress, including anxiety attack, anger, a hypomanic breakdown, and depression. In some encounters it became clear that higher-priority competing demands took precedence as a result of a patient-driven agenda (eg, a discussion about care from multiple consultants or a lengthy discussion about multiple medications) or a physician-driven agenda (eg, a first visit for a patient with a complex medical problem squeezed into an acute visit time slot). In reviewing these encounters, the research team agreed that the competing priorities were appropriately important to reasonably not expect discussion of tobacco cessation.

Failure in non–smoking-related visit
A third common pattern was seen in 27% of encounters in which the physician failed to address smoking cessation in a non–smoking-related illness visit during which competing demands were low. In the vast majority of these (14 of 20), failure occurred despite having a reminder system for smoking cessation in place. Examples of visits in this pattern included consults for skin conditions (eg, boil or rash) or follow-up of stable back pain.

Failure in smoking-related visit
Although a smoking related-illness often triggered counseling, another common pattern was for physicians to fail to address smoking in patients presenting with acute respiratory illnesses or other chronic conditions related to smoking. This occurred in 22% of cases, including 10 encounters in which the physician failed to even ask the patient’s smoking status. In 7 of 17 encounters the physician did ask the patient if he or she smoked; in 3 they advised patients to stop smoking, but did not follow though with assessing readiness to change or offering assistance to help the patient quit smoking. Most visits (12 cases) following this pattern failed to address tobacco use for acute upper respiratory symptoms (eg, sore throat, nasal congestion, “sinus,” severe cough).

Failure in health maintenance visit
Finally, a fifth pattern emerged when smoking cessation was not fully addressed in health maintenance visits. In the 2 encounters where this occurred, the physician did ask about smoking status as part of the history taking but did not assess the patient’s readiness to change or offer assistance. It should be noted that 3 of the 5 health maintenance examinations were of good quality tobacco counseling.

Discussion

Our study confirms previous reports of poor compliance with a smoking cessation practice guideline that recommends assessment and consideration of counseling at every visit.^7-12 We found that reliance on a reminder system to identify smokers was often not sufficient to prompt smoking cessation interventions, even during visits for tobacco-related problems.²⁰ In our study, however, more than one half of the physicians demonstrated that they have the skills needed to provide good quality brief intervention for smoking cessation,² and one fourth of the smokers received good quality tobacco counseling.

An important new finding in our study is the documentation of competing demands and priorities during encounters with smokers in primary care practices. In almost 25% of visits by smokers the smoking cessation agenda was appropriately overridden by competing demands (eg, acute pain, acute psychological distress, and other important demands). This finding shows that guidelines that recommend assessment and counseling at every visit are unrealistic, and if followed may not lead to optimal integration and individualization of primary care services.¹⁷ However, the finding of “appropriately missed opportunities” makes it imperative that tobacco cessation counseling be reliably integrated during all other visits with smokers when these competing demands are not present. Visits for well care and tobacco-related illnesses represent teachable moments that should not be missed.

Limitations

Although our study provides important and novel insights into the delivery of tobacco interventions in primary care, it has limitations. The physicians and practices represented here were purposely selected from the larger Prevention and Competing Demands Study and are not representative of the universe of family practices in Nebraska or the United States. Because the study relied on descriptions recorded by an observer, it is possible that subtle communication nuances between the patient and physician may have been missed. Nevertheless, the observer was specifically focused on preventive service delivery, so important details of the encounter are likely to have been captured. We explored the possibility of observer bias by a single observer by expanding an audit of encounters to other practices, physicians, and observers, and we failed to detect additional patterns of delivery. Finally, these patient encounters are only a cross-sectional window into these physicians’ smoking cessation practices.

Conclusions

Our study has important implications for improving delivery of tobacco cessation services in primary care practices. Although many physicians demonstrated basic skills for delivering brief smoking cessation interventions, it is clear that most have not adopted the model of tobacco use disorder as a chronic disease that needs to be addressed at every visit.² Reliance on guidelines and office system tools without the adoption of this model is unlikely to result in higher rates of tobacco cessation. Thus, there is a need to develop interventions that encourage the adoption of this illness model and to develop systems to support tobacco counseling during visits that don’t include overriding important competing opportunities.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Family Practice Research Center grant from the American Academy of Family Physicians. Drs Jaén, Flocke, and Crabtree are associated with the Center for Research in Family Practice and Primary Care Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 12 practices, without whose participation this study would not have been possible. We also wish to acknowledge the dedicated work of Angela Henke from the Department of Family Medicine of the State University of New York at Buffalo, who provided coordination support for the analyses and collated the data tables. Evangeline Rodriguez from the Department of Family and Community Medicine at the University of Texas Health Science Center at San Antonio assisted with manuscript preparation. Kurt C. Stange, MD, PhD, provided helpful comments on earlier drafts of this paper.

Related Resources

• The Virtual Office of the Surgeon General http://www.surgeongeneral.gov/tobacco/ This site contains several PDF files of patient-oriented materials based on the Public Health Service Clinical practice guideline.
• U.S. Centers for Disease Control and Prevention—Tobacco Information and Prevention Source (TIPS) http://www.cdc.gov/tobacco Tips for adults, clinicians and youths about how to treat and prevent tobacco use.
• QuitNet " target="_blank">http://www.quitnet.com/BR> QuitNet offers an online support community, forums moderated by counselors, and individually tailored advice to help smokers kick their nicotine addiction.
• California Smokers’ Helpline http://www.nobutts.ucsd.edu/ This site was created to be both fun and informative. A must for patients who are ready to quit or just thinking about it.

Approximately 17 million smokers attempt to stop smoking for more than 24 hours every year; only 1.2 million are successful.¹ There is strong evidence that smokers attempting to quit could at least double their chances of success if they were assisted by clinicians using effective behavioral and pharmacologic interventions.² Because 7 of 10 smokers will see a physician each year³ and the majority of these visits are made to primary care physicians,⁴ these physicians have multiple opportunities to assist smokers in their attempts to quit.

Clinicians should follow the “5 A’s” (ask, advise, assess, assist, and arrange) whenever appropriate. The current US Public Health Service smoking cessation clinical practice guideline offers specific directions for clinician intervention for all smokers, recommending a minimum of 3A’s (ask, advise, and assess) at every visit. That is, all smokers should be asked about their current smoking status, advised to quit, and assessed regarding their readiness to change. For smokers willing to quit, 2 additional A’s (assist and arrange follow-up) should be implemented; for smokers not willing to quit, a brief motivational intervention is recommended.²

Although there is a high level of agreement among primary care physicians about their responsibility to assist in tobacco cessation,^5,6 there are significant gaps in practice.^7-9 Reports of physicians’ rates of smoking cessation advice range from 21% to 78%,^7-12 falling short of recommended levels.¹³

A recent direct observation study of community family physicians found that, on average, 25% of smokers were advised to stop smoking.¹⁴ The study also showed that smoking cessation advice was offered during 55% of well care visits and in 32% of chronic illness visits for tobacco-related problems. The average duration of smoking cessation advice was less than 90 seconds. Although the study’s authors were able to assess whether smoking cessation advice occurred during an encounter, limits of the data made it impossible to examine how the particular content of smoking cessation advice was delivered. Similar results were found in a study of direct observation of Australian physicians.¹⁵

For this study, we used direct observation of outpatient visits by smokers to describe the extent of tobacco counseling and the processes by which it was provided. The analyses also explore the contextual factors that influence the provision of smoking cessation counseling. We hypothesized that the low rates of smoking counseling reported in the literature were in part due to the competing demands brought on by the complex agenda of patients presenting with undifferentiated problems.^16,17 We also hypothesized that the current care included missed opportunities to integrate tobacco counseling into the broad primary care agenda.

Methods

The data used for this analysis were collected as part of The Prevention and Competing Demands in Primary Care Study, an in-depth observational study that examined the organizational and clinical structures and process of community-based family practices.Each of 18 purposefully selected practices was studied using a multimethod comparative case study design that involved extensive direct observation of clinical encounters and office systems by field researchers who spent 4 weeks or more in each practice. Field researchers directly observed approximately 30 patient encounters with each of more than 50 clinicians, dictated descriptions of the visits, and audited the medical records of each of these patients. Detailed descriptive field notes documented day-to-day practice operations. Individual depth interviews with each clinician, many of the practice staff, and members of the community were used to obtain different perspectives on the practice. Details of the sampling and data collection are available elsewhere in this issue of JFP.¹⁸

From the exit survey administered to patients, 239 current cigarette smokers (14.7% of the study population) were identified from the 1624 encounters. To minimize observer variation in encounter content, only the narratives of a single research nurse were examined, reducing the sample of current smokers to 123. Only encounters with physicians were selected for analysis, further reducing the sample size to 91.

The research team included 6 members representing a broad range of perspectives, including family medicine, health services research, epidemiology, psychology, anthropology, and sociology. We used an iterative analysis and interpretation process that evolved over time as the team became more familiar with the data.¹⁹ Two immediate objectives were identified: (1) to develop a classification system that could be used to describe how physicians address smoking cessation, and (2) to identify factors that may enhance or impede the degree of adherence to the clinical smoking cessation guideline.² First, the team selected 18 encounters for reading and discussion by all research team members. For each of these encounters, one team member read the narrative out loud, and then the team discussed at length their understanding and assessment of what had taken place. Narrative data from the chart audit and physician interviews were considered as the discussion proceeded. During these discussions, preliminary schemes for classifying and assessing the encounters were developed.

The team was then divided into 3 groups of 2, and each group was assigned approximately 10 encounters for reading and for further development of the initial schemes. To ensure that each group member’s evaluation was independent, each member wrote a description and evaluation of each encounter without having read what the other member had written. The classifications and evaluations were then shared with the other member and the entire research team. Multiple team discussions were used to address differences in interpretation and to identify salient patterns within the data.

After discussing the initial 48 encounters, the remaining 43 encounters were analyzed. The same process of intragroup blind review was followed, and at this point, a nearly complete list of patterns and other important features seen within the encounters was established. Analysis and discussion by the entire research team led to agreement on the classification and evaluation of each of the 91 encounters.

To test the possibility that a single observer may introduce observer bias, the research team analyzed 51 additional clinical encounters with 9 family physicians in 5 different practices by a different research nurse. The 3 teams used the same blinded iterative process. These encounters were reviewed, looking for new patterns of smoking cessation counseling or confirmation of the patterns previously identified.

Results

We observed between 2 and 7 encounters of 20 family physicians in 7 practices Table 1. Five clear patterns were discernable according to the level of tobacco counseling and the type of visit. They represent a hierarchy that ranges from optimal smoking cessation counseling during visits when it was appropriate, to visits during which other agendas were appropriately given higher priority, to deficient missed opportunities. No additional patterns of interaction of smoking cessation counseling were identified among the 51 additional encounters audited.

In nearly half of the visits physicians either followed recommendations (21%), or competing priorities within the encounter reasonably overrode tobacco counseling (24%). In the other encounters tobacco cessation counseling fell short of recommendations, including visits among patients being seen for acute respiratory illnesses or other smoking-related illnesses. This failure often occurred despite the presence of a reminder system that identified the patient as a smoker. In 9% (8 cases) the physicians explicitly told the observing research nurse that they would not address tobacco with a specific patient because of a preconception that the patient would not respond.

Patterns of Tobacco Counseling

Good counseling
Good quality cessation counseling occurred in 21% of the encounters, during which physicians offered appropriate brief interventions depending on patients’ willingness to quit at that visit. Three levels of intervention were discernible within this first pattern. The 5A’s occurred when patients requested help, emphatically said “yes” when asked if they were interested in quitting, or when they responded positively to the physician’s description of pharmacologic options to help quit smoking. Patients were offered only 3A’s if they indicated they were not ready to quit by explicitly saying so or by staying quiet after an inquiry about their willingness to quit. Eleven physicians (55%) had at least 1 encounter with a smoker in which the physicians demonstrated good quality smoking cessation intervention, indicating that they had the knowledge and skill to provide recommended smoking cessation strategies.

Competing demands
Another common pattern was when a smoking cessation agenda was appropriately overridden by higher priorities. This occurred in 24% of the encounters. These were visits during which the physician-patient interaction was less straightforward than simply history taking, diagnosis, and treatment. In 10 encounters the top priority was alleviation of acute pain. Examples included abdominal pain, chest pain, back pain related to pyelonephritis, and severe rib pain after trauma. During 6 encounters patients were experiencing psychological distress, including anxiety attack, anger, a hypomanic breakdown, and depression. In some encounters it became clear that higher-priority competing demands took precedence as a result of a patient-driven agenda (eg, a discussion about care from multiple consultants or a lengthy discussion about multiple medications) or a physician-driven agenda (eg, a first visit for a patient with a complex medical problem squeezed into an acute visit time slot). In reviewing these encounters, the research team agreed that the competing priorities were appropriately important to reasonably not expect discussion of tobacco cessation.

Failure in non–smoking-related visit
A third common pattern was seen in 27% of encounters in which the physician failed to address smoking cessation in a non–smoking-related illness visit during which competing demands were low. In the vast majority of these (14 of 20), failure occurred despite having a reminder system for smoking cessation in place. Examples of visits in this pattern included consults for skin conditions (eg, boil or rash) or follow-up of stable back pain.

Failure in smoking-related visit
Although a smoking related-illness often triggered counseling, another common pattern was for physicians to fail to address smoking in patients presenting with acute respiratory illnesses or other chronic conditions related to smoking. This occurred in 22% of cases, including 10 encounters in which the physician failed to even ask the patient’s smoking status. In 7 of 17 encounters the physician did ask the patient if he or she smoked; in 3 they advised patients to stop smoking, but did not follow though with assessing readiness to change or offering assistance to help the patient quit smoking. Most visits (12 cases) following this pattern failed to address tobacco use for acute upper respiratory symptoms (eg, sore throat, nasal congestion, “sinus,” severe cough).

Failure in health maintenance visit
Finally, a fifth pattern emerged when smoking cessation was not fully addressed in health maintenance visits. In the 2 encounters where this occurred, the physician did ask about smoking status as part of the history taking but did not assess the patient’s readiness to change or offer assistance. It should be noted that 3 of the 5 health maintenance examinations were of good quality tobacco counseling.

Discussion

Our study confirms previous reports of poor compliance with a smoking cessation practice guideline that recommends assessment and consideration of counseling at every visit.^7-12 We found that reliance on a reminder system to identify smokers was often not sufficient to prompt smoking cessation interventions, even during visits for tobacco-related problems.²⁰ In our study, however, more than one half of the physicians demonstrated that they have the skills needed to provide good quality brief intervention for smoking cessation,² and one fourth of the smokers received good quality tobacco counseling.

An important new finding in our study is the documentation of competing demands and priorities during encounters with smokers in primary care practices. In almost 25% of visits by smokers the smoking cessation agenda was appropriately overridden by competing demands (eg, acute pain, acute psychological distress, and other important demands). This finding shows that guidelines that recommend assessment and counseling at every visit are unrealistic, and if followed may not lead to optimal integration and individualization of primary care services.¹⁷ However, the finding of “appropriately missed opportunities” makes it imperative that tobacco cessation counseling be reliably integrated during all other visits with smokers when these competing demands are not present. Visits for well care and tobacco-related illnesses represent teachable moments that should not be missed.

Limitations

Although our study provides important and novel insights into the delivery of tobacco interventions in primary care, it has limitations. The physicians and practices represented here were purposely selected from the larger Prevention and Competing Demands Study and are not representative of the universe of family practices in Nebraska or the United States. Because the study relied on descriptions recorded by an observer, it is possible that subtle communication nuances between the patient and physician may have been missed. Nevertheless, the observer was specifically focused on preventive service delivery, so important details of the encounter are likely to have been captured. We explored the possibility of observer bias by a single observer by expanding an audit of encounters to other practices, physicians, and observers, and we failed to detect additional patterns of delivery. Finally, these patient encounters are only a cross-sectional window into these physicians’ smoking cessation practices.

Conclusions

Our study has important implications for improving delivery of tobacco cessation services in primary care practices. Although many physicians demonstrated basic skills for delivering brief smoking cessation interventions, it is clear that most have not adopted the model of tobacco use disorder as a chronic disease that needs to be addressed at every visit.² Reliance on guidelines and office system tools without the adoption of this model is unlikely to result in higher rates of tobacco cessation. Thus, there is a need to develop interventions that encourage the adoption of this illness model and to develop systems to support tobacco counseling during visits that don’t include overriding important competing opportunities.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Family Practice Research Center grant from the American Academy of Family Physicians. Drs Jaén, Flocke, and Crabtree are associated with the Center for Research in Family Practice and Primary Care Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 12 practices, without whose participation this study would not have been possible. We also wish to acknowledge the dedicated work of Angela Henke from the Department of Family Medicine of the State University of New York at Buffalo, who provided coordination support for the analyses and collated the data tables. Evangeline Rodriguez from the Department of Family and Community Medicine at the University of Texas Health Science Center at San Antonio assisted with manuscript preparation. Kurt C. Stange, MD, PhD, provided helpful comments on earlier drafts of this paper.

Related Resources

• The Virtual Office of the Surgeon General http://www.surgeongeneral.gov/tobacco/ This site contains several PDF files of patient-oriented materials based on the Public Health Service Clinical practice guideline.
• U.S. Centers for Disease Control and Prevention—Tobacco Information and Prevention Source (TIPS) http://www.cdc.gov/tobacco Tips for adults, clinicians and youths about how to treat and prevent tobacco use.
• QuitNet " target="_blank">http://www.quitnet.com/BR> QuitNet offers an online support community, forums moderated by counselors, and individually tailored advice to help smokers kick their nicotine addiction.
• California Smokers’ Helpline http://www.nobutts.ucsd.edu/ This site was created to be both fun and informative. A must for patients who are ready to quit or just thinking about it.

Approximately 17 million smokers attempt to stop smoking for more than 24 hours every year; only 1.2 million are successful.¹ There is strong evidence that smokers attempting to quit could at least double their chances of success if they were assisted by clinicians using effective behavioral and pharmacologic interventions.² Because 7 of 10 smokers will see a physician each year³ and the majority of these visits are made to primary care physicians,⁴ these physicians have multiple opportunities to assist smokers in their attempts to quit.

Clinicians should follow the “5 A’s” (ask, advise, assess, assist, and arrange) whenever appropriate. The current US Public Health Service smoking cessation clinical practice guideline offers specific directions for clinician intervention for all smokers, recommending a minimum of 3A’s (ask, advise, and assess) at every visit. That is, all smokers should be asked about their current smoking status, advised to quit, and assessed regarding their readiness to change. For smokers willing to quit, 2 additional A’s (assist and arrange follow-up) should be implemented; for smokers not willing to quit, a brief motivational intervention is recommended.²

Although there is a high level of agreement among primary care physicians about their responsibility to assist in tobacco cessation,^5,6 there are significant gaps in practice.^7-9 Reports of physicians’ rates of smoking cessation advice range from 21% to 78%,^7-12 falling short of recommended levels.¹³

A recent direct observation study of community family physicians found that, on average, 25% of smokers were advised to stop smoking.¹⁴ The study also showed that smoking cessation advice was offered during 55% of well care visits and in 32% of chronic illness visits for tobacco-related problems. The average duration of smoking cessation advice was less than 90 seconds. Although the study’s authors were able to assess whether smoking cessation advice occurred during an encounter, limits of the data made it impossible to examine how the particular content of smoking cessation advice was delivered. Similar results were found in a study of direct observation of Australian physicians.¹⁵

For this study, we used direct observation of outpatient visits by smokers to describe the extent of tobacco counseling and the processes by which it was provided. The analyses also explore the contextual factors that influence the provision of smoking cessation counseling. We hypothesized that the low rates of smoking counseling reported in the literature were in part due to the competing demands brought on by the complex agenda of patients presenting with undifferentiated problems.^16,17 We also hypothesized that the current care included missed opportunities to integrate tobacco counseling into the broad primary care agenda.

Methods

The data used for this analysis were collected as part of The Prevention and Competing Demands in Primary Care Study, an in-depth observational study that examined the organizational and clinical structures and process of community-based family practices.Each of 18 purposefully selected practices was studied using a multimethod comparative case study design that involved extensive direct observation of clinical encounters and office systems by field researchers who spent 4 weeks or more in each practice. Field researchers directly observed approximately 30 patient encounters with each of more than 50 clinicians, dictated descriptions of the visits, and audited the medical records of each of these patients. Detailed descriptive field notes documented day-to-day practice operations. Individual depth interviews with each clinician, many of the practice staff, and members of the community were used to obtain different perspectives on the practice. Details of the sampling and data collection are available elsewhere in this issue of JFP.¹⁸

From the exit survey administered to patients, 239 current cigarette smokers (14.7% of the study population) were identified from the 1624 encounters. To minimize observer variation in encounter content, only the narratives of a single research nurse were examined, reducing the sample of current smokers to 123. Only encounters with physicians were selected for analysis, further reducing the sample size to 91.

The research team included 6 members representing a broad range of perspectives, including family medicine, health services research, epidemiology, psychology, anthropology, and sociology. We used an iterative analysis and interpretation process that evolved over time as the team became more familiar with the data.¹⁹ Two immediate objectives were identified: (1) to develop a classification system that could be used to describe how physicians address smoking cessation, and (2) to identify factors that may enhance or impede the degree of adherence to the clinical smoking cessation guideline.² First, the team selected 18 encounters for reading and discussion by all research team members. For each of these encounters, one team member read the narrative out loud, and then the team discussed at length their understanding and assessment of what had taken place. Narrative data from the chart audit and physician interviews were considered as the discussion proceeded. During these discussions, preliminary schemes for classifying and assessing the encounters were developed.

The team was then divided into 3 groups of 2, and each group was assigned approximately 10 encounters for reading and for further development of the initial schemes. To ensure that each group member’s evaluation was independent, each member wrote a description and evaluation of each encounter without having read what the other member had written. The classifications and evaluations were then shared with the other member and the entire research team. Multiple team discussions were used to address differences in interpretation and to identify salient patterns within the data.

After discussing the initial 48 encounters, the remaining 43 encounters were analyzed. The same process of intragroup blind review was followed, and at this point, a nearly complete list of patterns and other important features seen within the encounters was established. Analysis and discussion by the entire research team led to agreement on the classification and evaluation of each of the 91 encounters.

To test the possibility that a single observer may introduce observer bias, the research team analyzed 51 additional clinical encounters with 9 family physicians in 5 different practices by a different research nurse. The 3 teams used the same blinded iterative process. These encounters were reviewed, looking for new patterns of smoking cessation counseling or confirmation of the patterns previously identified.

Results

We observed between 2 and 7 encounters of 20 family physicians in 7 practices Table 1. Five clear patterns were discernable according to the level of tobacco counseling and the type of visit. They represent a hierarchy that ranges from optimal smoking cessation counseling during visits when it was appropriate, to visits during which other agendas were appropriately given higher priority, to deficient missed opportunities. No additional patterns of interaction of smoking cessation counseling were identified among the 51 additional encounters audited.

In nearly half of the visits physicians either followed recommendations (21%), or competing priorities within the encounter reasonably overrode tobacco counseling (24%). In the other encounters tobacco cessation counseling fell short of recommendations, including visits among patients being seen for acute respiratory illnesses or other smoking-related illnesses. This failure often occurred despite the presence of a reminder system that identified the patient as a smoker. In 9% (8 cases) the physicians explicitly told the observing research nurse that they would not address tobacco with a specific patient because of a preconception that the patient would not respond.

Patterns of Tobacco Counseling

Good counseling
Good quality cessation counseling occurred in 21% of the encounters, during which physicians offered appropriate brief interventions depending on patients’ willingness to quit at that visit. Three levels of intervention were discernible within this first pattern. The 5A’s occurred when patients requested help, emphatically said “yes” when asked if they were interested in quitting, or when they responded positively to the physician’s description of pharmacologic options to help quit smoking. Patients were offered only 3A’s if they indicated they were not ready to quit by explicitly saying so or by staying quiet after an inquiry about their willingness to quit. Eleven physicians (55%) had at least 1 encounter with a smoker in which the physicians demonstrated good quality smoking cessation intervention, indicating that they had the knowledge and skill to provide recommended smoking cessation strategies.

Competing demands
Another common pattern was when a smoking cessation agenda was appropriately overridden by higher priorities. This occurred in 24% of the encounters. These were visits during which the physician-patient interaction was less straightforward than simply history taking, diagnosis, and treatment. In 10 encounters the top priority was alleviation of acute pain. Examples included abdominal pain, chest pain, back pain related to pyelonephritis, and severe rib pain after trauma. During 6 encounters patients were experiencing psychological distress, including anxiety attack, anger, a hypomanic breakdown, and depression. In some encounters it became clear that higher-priority competing demands took precedence as a result of a patient-driven agenda (eg, a discussion about care from multiple consultants or a lengthy discussion about multiple medications) or a physician-driven agenda (eg, a first visit for a patient with a complex medical problem squeezed into an acute visit time slot). In reviewing these encounters, the research team agreed that the competing priorities were appropriately important to reasonably not expect discussion of tobacco cessation.

Failure in non–smoking-related visit
A third common pattern was seen in 27% of encounters in which the physician failed to address smoking cessation in a non–smoking-related illness visit during which competing demands were low. In the vast majority of these (14 of 20), failure occurred despite having a reminder system for smoking cessation in place. Examples of visits in this pattern included consults for skin conditions (eg, boil or rash) or follow-up of stable back pain.

Failure in smoking-related visit
Although a smoking related-illness often triggered counseling, another common pattern was for physicians to fail to address smoking in patients presenting with acute respiratory illnesses or other chronic conditions related to smoking. This occurred in 22% of cases, including 10 encounters in which the physician failed to even ask the patient’s smoking status. In 7 of 17 encounters the physician did ask the patient if he or she smoked; in 3 they advised patients to stop smoking, but did not follow though with assessing readiness to change or offering assistance to help the patient quit smoking. Most visits (12 cases) following this pattern failed to address tobacco use for acute upper respiratory symptoms (eg, sore throat, nasal congestion, “sinus,” severe cough).

Failure in health maintenance visit
Finally, a fifth pattern emerged when smoking cessation was not fully addressed in health maintenance visits. In the 2 encounters where this occurred, the physician did ask about smoking status as part of the history taking but did not assess the patient’s readiness to change or offer assistance. It should be noted that 3 of the 5 health maintenance examinations were of good quality tobacco counseling.

Discussion

Our study confirms previous reports of poor compliance with a smoking cessation practice guideline that recommends assessment and consideration of counseling at every visit.^7-12 We found that reliance on a reminder system to identify smokers was often not sufficient to prompt smoking cessation interventions, even during visits for tobacco-related problems.²⁰ In our study, however, more than one half of the physicians demonstrated that they have the skills needed to provide good quality brief intervention for smoking cessation,² and one fourth of the smokers received good quality tobacco counseling.

An important new finding in our study is the documentation of competing demands and priorities during encounters with smokers in primary care practices. In almost 25% of visits by smokers the smoking cessation agenda was appropriately overridden by competing demands (eg, acute pain, acute psychological distress, and other important demands). This finding shows that guidelines that recommend assessment and counseling at every visit are unrealistic, and if followed may not lead to optimal integration and individualization of primary care services.¹⁷ However, the finding of “appropriately missed opportunities” makes it imperative that tobacco cessation counseling be reliably integrated during all other visits with smokers when these competing demands are not present. Visits for well care and tobacco-related illnesses represent teachable moments that should not be missed.

Limitations

Although our study provides important and novel insights into the delivery of tobacco interventions in primary care, it has limitations. The physicians and practices represented here were purposely selected from the larger Prevention and Competing Demands Study and are not representative of the universe of family practices in Nebraska or the United States. Because the study relied on descriptions recorded by an observer, it is possible that subtle communication nuances between the patient and physician may have been missed. Nevertheless, the observer was specifically focused on preventive service delivery, so important details of the encounter are likely to have been captured. We explored the possibility of observer bias by a single observer by expanding an audit of encounters to other practices, physicians, and observers, and we failed to detect additional patterns of delivery. Finally, these patient encounters are only a cross-sectional window into these physicians’ smoking cessation practices.

Conclusions

Our study has important implications for improving delivery of tobacco cessation services in primary care practices. Although many physicians demonstrated basic skills for delivering brief smoking cessation interventions, it is clear that most have not adopted the model of tobacco use disorder as a chronic disease that needs to be addressed at every visit.² Reliance on guidelines and office system tools without the adoption of this model is unlikely to result in higher rates of tobacco cessation. Thus, there is a need to develop interventions that encourage the adoption of this illness model and to develop systems to support tobacco counseling during visits that don’t include overriding important competing opportunities.

Acknowledgments

Our study was supported by a grant from the Agency for Healthcare Research and Quality (R01 HS08776) and a Family Practice Research Center grant from the American Academy of Family Physicians. Drs Jaén, Flocke, and Crabtree are associated with the Center for Research in Family Practice and Primary Care Cleveland, New Brunswick, Allentown, and San Antonio. We are grateful to the physicians, staff, and patients from the 12 practices, without whose participation this study would not have been possible. We also wish to acknowledge the dedicated work of Angela Henke from the Department of Family Medicine of the State University of New York at Buffalo, who provided coordination support for the analyses and collated the data tables. Evangeline Rodriguez from the Department of Family and Community Medicine at the University of Texas Health Science Center at San Antonio assisted with manuscript preparation. Kurt C. Stange, MD, PhD, provided helpful comments on earlier drafts of this paper.

Related Resources

• The Virtual Office of the Surgeon General http://www.surgeongeneral.gov/tobacco/ This site contains several PDF files of patient-oriented materials based on the Public Health Service Clinical practice guideline.
• U.S. Centers for Disease Control and Prevention—Tobacco Information and Prevention Source (TIPS) http://www.cdc.gov/tobacco Tips for adults, clinicians and youths about how to treat and prevent tobacco use.
• QuitNet " target="_blank">http://www.quitnet.com/BR> QuitNet offers an online support community, forums moderated by counselors, and individually tailored advice to help smokers kick their nicotine addiction.
• California Smokers’ Helpline http://www.nobutts.ucsd.edu/ This site was created to be both fun and informative. A must for patients who are ready to quit or just thinking about it.

Recent smoking cessation guidelines¹ identify critical ways primary care physicians can intervene with their patients to improve cessation rates. These guidelines recommend that all patients be asked about their smoking status at every visit. Also at every visit all smokers should be advised to quit and should be assessed for their readiness to do so. The guidelines include recommendations for physicians to incorporate elements into their practices that will help them maximize smoking cessation rates. These elements include systems to routinely identify all smokers, reminder systems to encourage physicians to discuss smoking, tools for assisting their patients in quitting, and assignments for nonphysician personnel to assist in smoking cessation. Preliminary data suggest that few practices have adopted these types of systems^2,3 and that many smokers have not been advised by their physicians to quit.⁴

Examining counseling behaviors, such as smoking cessation, in a physician’s office can be challenging. Previous attempts to examine such efforts have relied primarily on physician self-report^5,6 and patient surveys.^7,8 A few studies have employed direct observation as a method of data collection.^3,9 Stange and colleagues¹⁰ have suggested that direct observation of clinical practices may be the gold standard for measuring counseling activities. Although patient reports appear to have a high degree of correlation with direct observation, the accuracy of these reports tend to deteriorate with time;^10,11 medical records are frequently incomplete; and physician reports typically overestimate counseling activities.¹⁰

For our study, medical students directly observed physician-patient encounters in primary care physicians’ offices in Kansas. Our objectives were to describe physician activities related to smoking cessation efforts and to identify physician and office characteristics that support these efforts.

Methods

Study Setting

We identified 38 family physicians in 38 separate practices in Kansas who agreed to precept students for 6 weeks during June and July of 1999; 89% of these practices were in non-metropolitan areas. These physicians had served as preceptors to medical students in previous years and were familiar with data collection efforts by students. Each family physician consented to have students observe preventive care practices in their offices during the rotation.

Medical Student Training and Support

Students collected data on a summer research elective between their first and second years of medical school. They received extensive training on the research study. During the next 8 weeks, the students worked with their assigned physicians. They submitted weekly reports of research activities to the study coordinator, who was in contact with the students through electronic mail and telephone calls throughout the course of our study.

Sample Selection

Physician-patient encounters were included in our study if the patient was aged at least 18 years, the physician saw the patient during normal office hours, and the student was present for the entire visit. Encounters were excluded from data collection if the office visit was for a critical acute complaint or a procedure, if the patient appeared to be in immediate emotional distress, if the patient suffered from dementia, if there were language difficulties that precluded observation of counseling behaviors, or if the student had previously observed an encounter with that patient. The Human Subjects Committee of the University of Kansas Medical Center approved the protocol.

Data Collection

After using the first week to identify any local problems in the data collection process, the students observed up to 40 consecutive eligible physician-patient encounters per week. They discontinued data collection after observing a total of 80 such encounters. Students recorded their observations on preprinted standardized observational assessment cards that were designed to facilitate recording data in as unobtrusive a manner as possible.

If tobacco use was discussed and the patient was a smoker, the student recorded additional information about the discussion. This included whether the physician asked if the patient wanted to quit smoker, advised the patient to quit smoking, offered assistance with smoking cessation, asked the patient to set a quit date, arranged follow-up for smoking cessation, or discussed either nicotine replacement or bupropion.

During the final week in the practice, students conducted a formal examination of the office to identify smoking policies, the designation of office personnel to handle smoking cessation efforts, the presence of smoking cessation materials and pharmaceutical samples in the office, patient follow-up procedures, and charting tools used to record or prompt discussion of tobacco use.

During the last few days of the rotation the students administered a survey to the physicians to obtain demographic data about the physician, recent training or education on smoking cessation, and perceived confidence in providing assistance with smoking cessation (used with permission of DePue and colleagues, unpublished).

Data Analysis

We examined the relationship between characteristics of the patient, the physician, and the physician’s office with the presence or absence of tobacco discussions during a physician-patient encounter. Simple chi-square tests were not appropriate for many of the analyses in our study, because of the clustering of multiple patients within individual office practices. For this reason, we used logistic regression with generalized estimating equations to determine the association of patient, physician, and office characteristics with the outcomes, while simultaneously controlling for the clustering of patients within practices.¹²

Results

We completed observations of 2963 physician-patient encounters. The mean age of the patients was 56 years (range = 18 to 99 years); 66% were women. New patient visits accounted for 130 (4.4%) of the observations.

Tobacco was mentioned or discussed in 633 (21%) visits, with 560 (88%) of these discussions initiated by the physician. The rate at which tobacco was discussed varied substantially among the practices Figure 1. In one practice, tobacco was not discussed during any of the patient encounters observed. Another practice, which designated a nurse to provide assistance with smoking cessation and follow-up of patients, addressed tobacco use during 90% of patient encounters.

Of the 633 patients with whom tobacco was discussed, 244 (39%) were identified as current smokers. The content of these tobacco-related discussions is shown in Table 1. The most common type of assistance given to smokers was pharmacotherapy. Physicians discussed bupropion and nicotine replacement therapy during 31% and 17% of encounters with smokers, respectively, with both agents discussed during 15% of encounters. Of the 24 practices in which tobacco was discussed with at least 5 smokers, the rate at which assistance was provided ranged from 0% to 100%.

The majority of physicians (68%) reported spending 1 to 6 hours during the past year developing knowledge or skills specific to smoking cessation. Using a Likert scale of 1 to 10 (where 10 = definitely confident and 1 = definitely not confident), an 8 or higher was reported by 58% of the physicians for their ability to incorporate smoking cessation strategies into regular office visits and by 34% for their ability to set up an office environment to support smoking cessation strategies.

Although all of the physicians maintained smoke-free offices, resources to support smoking cessation varied among the practices. Of the 38 offices, 26 (68%) had patient education materials; 22 (58%) maintained a standard location in the medical record to document the patient’s smoking status; 2 (5%) recorded the patient’s smoking status at every visit; and 6 (16%) had a staff person assigned to smoking cessation activities. Although pharmaceutical samples of bupropion were available in 35 (92%) offices, only 12 (32%) had samples of nicotine-replacement therapy.

Women physicians, physicians with 10 years or fewer in practice, and those practicing in offices with a form for recording smoking status in a standard location in the medical record were significantly more likely to discuss tobacco with their patients Table 1. The 2 patient characteristics associated with discussion of tobacco were being younger than 65 years and being a new patient.

When these factors were included in a multivariable logistic regression model, patient age, new patient status, and the presence of a form for recording smoking status were found to be important independent predictors of tobacco discussion Table 3. One variable that was not retained in the model was being in practice for 10 years or fewer (this variable was highly correlated with having a form for recording smoking status). An additional finding in the model was an interaction between patient sex and age, with women 65 years and older being the group least likely to have tobacco discussed during the visit.

Discussion

Our study shows that tobacco is a common issue in primary care that is discussed in more than 1 in 5 office visits. There is substantial variation, however, in the extent to which primary care physicians incorporate smoking cessation activities into their practices. We saw some practices in which tobacco was rarely, if ever, mentioned and 1 practice in which smoking was addressed during 90% of visits. This widespread variation illustrates an opportunity for improvement.¹³

It can be difficult to address behavioral problems such as nicotine addiction in a busy primary care practice. Other barriers include perceived patient attitudes about quitting,⁶ a lack of office support systems, time constraints, and the need to respond to other urgent health needs.¹⁴ When patients are presenting with a variety of acute and chronic ailments, it is easy to forget preventive care issues, such as nicotine dependence.

There is a large and growing body of evidence showing that changes in office systems can improve smoking cessation practices.^1,4,15,16 We showed that some of the variation between offices in smoking cessation practices can be explained by the presence of charting systems that routinely identify smokers. Although only a handful of offices in our study documented smoking status at each visit, those that did discussed tobacco 3 times as often as those that did not do so routinely. Although the physicians in our study reported that they were confident in their abilities to develop systems to support smoking cessation, most offices had not implemented the types of office systems described in published guidelines available at the time of the study.¹⁷

Further improvements and greater efficiencies can be obtained by delegating specific activities to nonphysician personnel in the office.^18-20 Our data showed a greater than 50% increased frequency of tobacco discussions in offices that assigned specific staff persons to address smoking cessation. This difference, however, was not statistically significant. This may have been because of the small number of offices that had such a dedicated staff person, resulting in a small percentage of the total patient encounters with this factor present and therefore a loss of power to detect differences.

Tobacco was more than twice as likely to be addressed during office visits with new patients, perhaps as part of a comprehensive health assessment. This is consistent with a recent report showing that when smoking status was recorded, it was usually on a health history form at the back of the chart.²¹

Consistent with previous studies,^22-24 we found that women physicians and physicians more recently trained were more likely to ask about smoking. These same physicians were more likely to have a standard form to record smoking status. It may be that newer physicians were more likely to be exposed to protocols or similar charting materials during their training; this is only speculation, however, since it appears that few medical schools routinely include smoking cessation training in their curricula.²⁵

One of the strategies recommended by Prochaska and Goldstein²⁶ and others^27,28 is to tailor smoking cessation strategies to a patient’s readiness to quit, yet assessments of readiness to quit were rarely seen in our study. Although it is possible that physicians had established readiness to quit during previous encounters with these patients, current guidelines recommend that this readiness be re-established at each visit. Because we do not know what proportion of smokers were ready to quit, we do not know what proportion of patients should have received assistance in smoking cessation, such as discussing pharmacotherapy, setting a quit date, or arranging follow-up.

In the 1995 National Ambulatory Medical Care Survey, nicotine-replacement therapy was prescribed during 1.3% of office visits with smokers.²⁹ (At that time, nicotine-replacement therapy was only available by prescription, and bupropion was not yet a standard treatment for nicotine addiction.) In contrast, our more recent data suggest that discussions of pharmacotherapy are a very common feature of physicians’ smoking cessation activities and that bupropion is being discussed more often than nicotine replacement.

Limitations

Direct observation of clinical practices has the advantage of reducing recall bias and increasing objectivity, yet there are limitations to this method as well. First, the use of separate observers precluded us from measuring the reliability of data collection. Second, we did not collect information regarding the reason for patient visits, which may include situations where the discussion of tobacco was not feasible or appropriate. Third, our study did not allow us to identify all smokers seen in the clinic and used volunteer physicians. Both of these factors could lead to an overestimate of the frequency in which assistance is provided. Although we found that assistance with smoking cessation was offered during 33% of visits with smokers, physicians participating in the National Ambulatory Medical Care Survey only reported offering assistance during 21% of visits. Finally, our study was limited to practices in Kansas and may not reflect those in other areas of the country.

Conclusions

Although smoking cessation is be a common topic in some physician-patient encounters, there are widespread variations in how it is addressed in primary care practices. More comprehensive and efficient management of nicotine dependence may be possible if physicians addressed the infrastructure in their offices that can support smoking cessation activities. Many physicians may not yet be considering changing their office systems to enhance smoking cessation activities. Further efforts will be needed to identify the barriers to system changes and to help physicians integrate effective and efficient smoking cessation systems into their practices.

Acknowledgments

Partial funding for this project was provided through the following grants: The Robert Wood Johnson Foundation Generalist Physicians Faculty Scholars award (#032686, J.S. Ahluwalia); Kansas Academy of Family Physicians (J. Gladden); J.H. Baker Trust of La Crosse, Kansas (J Gladden); Kansas Association for Medically Underserved (J. Gladden); and a Primary Care Physician Education grant from the Kansas Health Foundation. We would like to thank the family physicians who not only provided a valuable learning experience for the students but also allowed the data collection necessary for our paper. We appreciate the commitment of the students who collected the data. We would like to thank Kristin Hedberg, MA, for preliminary data analysis and Timothy P. Daaleman, DO, and Delwyn Catley, PhD, for their careful review of early versions of this manuscript.

Related Resources

For Patients:

• QuitNet http://www.quitnet.com/qn_main.jtml Developed by Boston University, this site provides information and tools to people trying to quit smoking. It contains peer support programs, information on pharmaceuticals, a directory of local smoking cessation programs, and the latest news from the tobacco front.
• Quitsmokingsupport.com http://www.quitsmokingsupport.com/intro.htm This advertiser-supported site has dozens of pages on weight control, methods, interactive chat/support rooms, quit smoking articles, smokers’ lungs, etc.

For physicians:

• Smoking cessation guidelines. http://www.surgeongeneral.gov/tobacco/default.htm Clinical practice guidelines from the Surgeon General’s Web site with full text and references for online retrieval. Patient education materials and posters for the doctor’s office are also available.
• PSNonline http://psnonline.org/Medical%20Management%20Program/Smoking% 20Cessation/smoking_cessation_front_page.htm This site provides a copy of the sticker or stamp that includes smoking cessation in the vital signs. Several studies have shown that this can significantly improve smoking cessation counseling in the doctor’s office.

Recent smoking cessation guidelines¹ identify critical ways primary care physicians can intervene with their patients to improve cessation rates. These guidelines recommend that all patients be asked about their smoking status at every visit. Also at every visit all smokers should be advised to quit and should be assessed for their readiness to do so. The guidelines include recommendations for physicians to incorporate elements into their practices that will help them maximize smoking cessation rates. These elements include systems to routinely identify all smokers, reminder systems to encourage physicians to discuss smoking, tools for assisting their patients in quitting, and assignments for nonphysician personnel to assist in smoking cessation. Preliminary data suggest that few practices have adopted these types of systems^2,3 and that many smokers have not been advised by their physicians to quit.⁴

Examining counseling behaviors, such as smoking cessation, in a physician’s office can be challenging. Previous attempts to examine such efforts have relied primarily on physician self-report^5,6 and patient surveys.^7,8 A few studies have employed direct observation as a method of data collection.^3,9 Stange and colleagues¹⁰ have suggested that direct observation of clinical practices may be the gold standard for measuring counseling activities. Although patient reports appear to have a high degree of correlation with direct observation, the accuracy of these reports tend to deteriorate with time;^10,11 medical records are frequently incomplete; and physician reports typically overestimate counseling activities.¹⁰

For our study, medical students directly observed physician-patient encounters in primary care physicians’ offices in Kansas. Our objectives were to describe physician activities related to smoking cessation efforts and to identify physician and office characteristics that support these efforts.

Methods

Study Setting

We identified 38 family physicians in 38 separate practices in Kansas who agreed to precept students for 6 weeks during June and July of 1999; 89% of these practices were in non-metropolitan areas. These physicians had served as preceptors to medical students in previous years and were familiar with data collection efforts by students. Each family physician consented to have students observe preventive care practices in their offices during the rotation.

Medical Student Training and Support

Students collected data on a summer research elective between their first and second years of medical school. They received extensive training on the research study. During the next 8 weeks, the students worked with their assigned physicians. They submitted weekly reports of research activities to the study coordinator, who was in contact with the students through electronic mail and telephone calls throughout the course of our study.

Sample Selection

Physician-patient encounters were included in our study if the patient was aged at least 18 years, the physician saw the patient during normal office hours, and the student was present for the entire visit. Encounters were excluded from data collection if the office visit was for a critical acute complaint or a procedure, if the patient appeared to be in immediate emotional distress, if the patient suffered from dementia, if there were language difficulties that precluded observation of counseling behaviors, or if the student had previously observed an encounter with that patient. The Human Subjects Committee of the University of Kansas Medical Center approved the protocol.

Data Collection

After using the first week to identify any local problems in the data collection process, the students observed up to 40 consecutive eligible physician-patient encounters per week. They discontinued data collection after observing a total of 80 such encounters. Students recorded their observations on preprinted standardized observational assessment cards that were designed to facilitate recording data in as unobtrusive a manner as possible.

If tobacco use was discussed and the patient was a smoker, the student recorded additional information about the discussion. This included whether the physician asked if the patient wanted to quit smoker, advised the patient to quit smoking, offered assistance with smoking cessation, asked the patient to set a quit date, arranged follow-up for smoking cessation, or discussed either nicotine replacement or bupropion.

During the final week in the practice, students conducted a formal examination of the office to identify smoking policies, the designation of office personnel to handle smoking cessation efforts, the presence of smoking cessation materials and pharmaceutical samples in the office, patient follow-up procedures, and charting tools used to record or prompt discussion of tobacco use.

During the last few days of the rotation the students administered a survey to the physicians to obtain demographic data about the physician, recent training or education on smoking cessation, and perceived confidence in providing assistance with smoking cessation (used with permission of DePue and colleagues, unpublished).

Data Analysis

We examined the relationship between characteristics of the patient, the physician, and the physician’s office with the presence or absence of tobacco discussions during a physician-patient encounter. Simple chi-square tests were not appropriate for many of the analyses in our study, because of the clustering of multiple patients within individual office practices. For this reason, we used logistic regression with generalized estimating equations to determine the association of patient, physician, and office characteristics with the outcomes, while simultaneously controlling for the clustering of patients within practices.¹²

Results

We completed observations of 2963 physician-patient encounters. The mean age of the patients was 56 years (range = 18 to 99 years); 66% were women. New patient visits accounted for 130 (4.4%) of the observations.

Tobacco was mentioned or discussed in 633 (21%) visits, with 560 (88%) of these discussions initiated by the physician. The rate at which tobacco was discussed varied substantially among the practices Figure 1. In one practice, tobacco was not discussed during any of the patient encounters observed. Another practice, which designated a nurse to provide assistance with smoking cessation and follow-up of patients, addressed tobacco use during 90% of patient encounters.

Of the 633 patients with whom tobacco was discussed, 244 (39%) were identified as current smokers. The content of these tobacco-related discussions is shown in Table 1. The most common type of assistance given to smokers was pharmacotherapy. Physicians discussed bupropion and nicotine replacement therapy during 31% and 17% of encounters with smokers, respectively, with both agents discussed during 15% of encounters. Of the 24 practices in which tobacco was discussed with at least 5 smokers, the rate at which assistance was provided ranged from 0% to 100%.

The majority of physicians (68%) reported spending 1 to 6 hours during the past year developing knowledge or skills specific to smoking cessation. Using a Likert scale of 1 to 10 (where 10 = definitely confident and 1 = definitely not confident), an 8 or higher was reported by 58% of the physicians for their ability to incorporate smoking cessation strategies into regular office visits and by 34% for their ability to set up an office environment to support smoking cessation strategies.

Although all of the physicians maintained smoke-free offices, resources to support smoking cessation varied among the practices. Of the 38 offices, 26 (68%) had patient education materials; 22 (58%) maintained a standard location in the medical record to document the patient’s smoking status; 2 (5%) recorded the patient’s smoking status at every visit; and 6 (16%) had a staff person assigned to smoking cessation activities. Although pharmaceutical samples of bupropion were available in 35 (92%) offices, only 12 (32%) had samples of nicotine-replacement therapy.

Women physicians, physicians with 10 years or fewer in practice, and those practicing in offices with a form for recording smoking status in a standard location in the medical record were significantly more likely to discuss tobacco with their patients Table 1. The 2 patient characteristics associated with discussion of tobacco were being younger than 65 years and being a new patient.

When these factors were included in a multivariable logistic regression model, patient age, new patient status, and the presence of a form for recording smoking status were found to be important independent predictors of tobacco discussion Table 3. One variable that was not retained in the model was being in practice for 10 years or fewer (this variable was highly correlated with having a form for recording smoking status). An additional finding in the model was an interaction between patient sex and age, with women 65 years and older being the group least likely to have tobacco discussed during the visit.

Discussion

Our study shows that tobacco is a common issue in primary care that is discussed in more than 1 in 5 office visits. There is substantial variation, however, in the extent to which primary care physicians incorporate smoking cessation activities into their practices. We saw some practices in which tobacco was rarely, if ever, mentioned and 1 practice in which smoking was addressed during 90% of visits. This widespread variation illustrates an opportunity for improvement.¹³

It can be difficult to address behavioral problems such as nicotine addiction in a busy primary care practice. Other barriers include perceived patient attitudes about quitting,⁶ a lack of office support systems, time constraints, and the need to respond to other urgent health needs.¹⁴ When patients are presenting with a variety of acute and chronic ailments, it is easy to forget preventive care issues, such as nicotine dependence.

There is a large and growing body of evidence showing that changes in office systems can improve smoking cessation practices.^1,4,15,16 We showed that some of the variation between offices in smoking cessation practices can be explained by the presence of charting systems that routinely identify smokers. Although only a handful of offices in our study documented smoking status at each visit, those that did discussed tobacco 3 times as often as those that did not do so routinely. Although the physicians in our study reported that they were confident in their abilities to develop systems to support smoking cessation, most offices had not implemented the types of office systems described in published guidelines available at the time of the study.¹⁷

Further improvements and greater efficiencies can be obtained by delegating specific activities to nonphysician personnel in the office.^18-20 Our data showed a greater than 50% increased frequency of tobacco discussions in offices that assigned specific staff persons to address smoking cessation. This difference, however, was not statistically significant. This may have been because of the small number of offices that had such a dedicated staff person, resulting in a small percentage of the total patient encounters with this factor present and therefore a loss of power to detect differences.

Tobacco was more than twice as likely to be addressed during office visits with new patients, perhaps as part of a comprehensive health assessment. This is consistent with a recent report showing that when smoking status was recorded, it was usually on a health history form at the back of the chart.²¹

Consistent with previous studies,^22-24 we found that women physicians and physicians more recently trained were more likely to ask about smoking. These same physicians were more likely to have a standard form to record smoking status. It may be that newer physicians were more likely to be exposed to protocols or similar charting materials during their training; this is only speculation, however, since it appears that few medical schools routinely include smoking cessation training in their curricula.²⁵

One of the strategies recommended by Prochaska and Goldstein²⁶ and others^27,28 is to tailor smoking cessation strategies to a patient’s readiness to quit, yet assessments of readiness to quit were rarely seen in our study. Although it is possible that physicians had established readiness to quit during previous encounters with these patients, current guidelines recommend that this readiness be re-established at each visit. Because we do not know what proportion of smokers were ready to quit, we do not know what proportion of patients should have received assistance in smoking cessation, such as discussing pharmacotherapy, setting a quit date, or arranging follow-up.

In the 1995 National Ambulatory Medical Care Survey, nicotine-replacement therapy was prescribed during 1.3% of office visits with smokers.²⁹ (At that time, nicotine-replacement therapy was only available by prescription, and bupropion was not yet a standard treatment for nicotine addiction.) In contrast, our more recent data suggest that discussions of pharmacotherapy are a very common feature of physicians’ smoking cessation activities and that bupropion is being discussed more often than nicotine replacement.

Limitations

Direct observation of clinical practices has the advantage of reducing recall bias and increasing objectivity, yet there are limitations to this method as well. First, the use of separate observers precluded us from measuring the reliability of data collection. Second, we did not collect information regarding the reason for patient visits, which may include situations where the discussion of tobacco was not feasible or appropriate. Third, our study did not allow us to identify all smokers seen in the clinic and used volunteer physicians. Both of these factors could lead to an overestimate of the frequency in which assistance is provided. Although we found that assistance with smoking cessation was offered during 33% of visits with smokers, physicians participating in the National Ambulatory Medical Care Survey only reported offering assistance during 21% of visits. Finally, our study was limited to practices in Kansas and may not reflect those in other areas of the country.

Conclusions

Although smoking cessation is be a common topic in some physician-patient encounters, there are widespread variations in how it is addressed in primary care practices. More comprehensive and efficient management of nicotine dependence may be possible if physicians addressed the infrastructure in their offices that can support smoking cessation activities. Many physicians may not yet be considering changing their office systems to enhance smoking cessation activities. Further efforts will be needed to identify the barriers to system changes and to help physicians integrate effective and efficient smoking cessation systems into their practices.

Acknowledgments

Partial funding for this project was provided through the following grants: The Robert Wood Johnson Foundation Generalist Physicians Faculty Scholars award (#032686, J.S. Ahluwalia); Kansas Academy of Family Physicians (J. Gladden); J.H. Baker Trust of La Crosse, Kansas (J Gladden); Kansas Association for Medically Underserved (J. Gladden); and a Primary Care Physician Education grant from the Kansas Health Foundation. We would like to thank the family physicians who not only provided a valuable learning experience for the students but also allowed the data collection necessary for our paper. We appreciate the commitment of the students who collected the data. We would like to thank Kristin Hedberg, MA, for preliminary data analysis and Timothy P. Daaleman, DO, and Delwyn Catley, PhD, for their careful review of early versions of this manuscript.

Related Resources

For Patients:

• QuitNet http://www.quitnet.com/qn_main.jtml Developed by Boston University, this site provides information and tools to people trying to quit smoking. It contains peer support programs, information on pharmaceuticals, a directory of local smoking cessation programs, and the latest news from the tobacco front.
• Quitsmokingsupport.com http://www.quitsmokingsupport.com/intro.htm This advertiser-supported site has dozens of pages on weight control, methods, interactive chat/support rooms, quit smoking articles, smokers’ lungs, etc.

For physicians:

• Smoking cessation guidelines. http://www.surgeongeneral.gov/tobacco/default.htm Clinical practice guidelines from the Surgeon General’s Web site with full text and references for online retrieval. Patient education materials and posters for the doctor’s office are also available.
• PSNonline http://psnonline.org/Medical%20Management%20Program/Smoking% 20Cessation/smoking_cessation_front_page.htm This site provides a copy of the sticker or stamp that includes smoking cessation in the vital signs. Several studies have shown that this can significantly improve smoking cessation counseling in the doctor’s office.

Recent smoking cessation guidelines¹ identify critical ways primary care physicians can intervene with their patients to improve cessation rates. These guidelines recommend that all patients be asked about their smoking status at every visit. Also at every visit all smokers should be advised to quit and should be assessed for their readiness to do so. The guidelines include recommendations for physicians to incorporate elements into their practices that will help them maximize smoking cessation rates. These elements include systems to routinely identify all smokers, reminder systems to encourage physicians to discuss smoking, tools for assisting their patients in quitting, and assignments for nonphysician personnel to assist in smoking cessation. Preliminary data suggest that few practices have adopted these types of systems^2,3 and that many smokers have not been advised by their physicians to quit.⁴

Examining counseling behaviors, such as smoking cessation, in a physician’s office can be challenging. Previous attempts to examine such efforts have relied primarily on physician self-report^5,6 and patient surveys.^7,8 A few studies have employed direct observation as a method of data collection.^3,9 Stange and colleagues¹⁰ have suggested that direct observation of clinical practices may be the gold standard for measuring counseling activities. Although patient reports appear to have a high degree of correlation with direct observation, the accuracy of these reports tend to deteriorate with time;^10,11 medical records are frequently incomplete; and physician reports typically overestimate counseling activities.¹⁰

For our study, medical students directly observed physician-patient encounters in primary care physicians’ offices in Kansas. Our objectives were to describe physician activities related to smoking cessation efforts and to identify physician and office characteristics that support these efforts.

Methods

Study Setting

We identified 38 family physicians in 38 separate practices in Kansas who agreed to precept students for 6 weeks during June and July of 1999; 89% of these practices were in non-metropolitan areas. These physicians had served as preceptors to medical students in previous years and were familiar with data collection efforts by students. Each family physician consented to have students observe preventive care practices in their offices during the rotation.

Medical Student Training and Support

Students collected data on a summer research elective between their first and second years of medical school. They received extensive training on the research study. During the next 8 weeks, the students worked with their assigned physicians. They submitted weekly reports of research activities to the study coordinator, who was in contact with the students through electronic mail and telephone calls throughout the course of our study.

Sample Selection

Physician-patient encounters were included in our study if the patient was aged at least 18 years, the physician saw the patient during normal office hours, and the student was present for the entire visit. Encounters were excluded from data collection if the office visit was for a critical acute complaint or a procedure, if the patient appeared to be in immediate emotional distress, if the patient suffered from dementia, if there were language difficulties that precluded observation of counseling behaviors, or if the student had previously observed an encounter with that patient. The Human Subjects Committee of the University of Kansas Medical Center approved the protocol.

Data Collection

After using the first week to identify any local problems in the data collection process, the students observed up to 40 consecutive eligible physician-patient encounters per week. They discontinued data collection after observing a total of 80 such encounters. Students recorded their observations on preprinted standardized observational assessment cards that were designed to facilitate recording data in as unobtrusive a manner as possible.

If tobacco use was discussed and the patient was a smoker, the student recorded additional information about the discussion. This included whether the physician asked if the patient wanted to quit smoker, advised the patient to quit smoking, offered assistance with smoking cessation, asked the patient to set a quit date, arranged follow-up for smoking cessation, or discussed either nicotine replacement or bupropion.

During the final week in the practice, students conducted a formal examination of the office to identify smoking policies, the designation of office personnel to handle smoking cessation efforts, the presence of smoking cessation materials and pharmaceutical samples in the office, patient follow-up procedures, and charting tools used to record or prompt discussion of tobacco use.

During the last few days of the rotation the students administered a survey to the physicians to obtain demographic data about the physician, recent training or education on smoking cessation, and perceived confidence in providing assistance with smoking cessation (used with permission of DePue and colleagues, unpublished).

Data Analysis

We examined the relationship between characteristics of the patient, the physician, and the physician’s office with the presence or absence of tobacco discussions during a physician-patient encounter. Simple chi-square tests were not appropriate for many of the analyses in our study, because of the clustering of multiple patients within individual office practices. For this reason, we used logistic regression with generalized estimating equations to determine the association of patient, physician, and office characteristics with the outcomes, while simultaneously controlling for the clustering of patients within practices.¹²

Results

We completed observations of 2963 physician-patient encounters. The mean age of the patients was 56 years (range = 18 to 99 years); 66% were women. New patient visits accounted for 130 (4.4%) of the observations.

Tobacco was mentioned or discussed in 633 (21%) visits, with 560 (88%) of these discussions initiated by the physician. The rate at which tobacco was discussed varied substantially among the practices Figure 1. In one practice, tobacco was not discussed during any of the patient encounters observed. Another practice, which designated a nurse to provide assistance with smoking cessation and follow-up of patients, addressed tobacco use during 90% of patient encounters.

Of the 633 patients with whom tobacco was discussed, 244 (39%) were identified as current smokers. The content of these tobacco-related discussions is shown in Table 1. The most common type of assistance given to smokers was pharmacotherapy. Physicians discussed bupropion and nicotine replacement therapy during 31% and 17% of encounters with smokers, respectively, with both agents discussed during 15% of encounters. Of the 24 practices in which tobacco was discussed with at least 5 smokers, the rate at which assistance was provided ranged from 0% to 100%.

The majority of physicians (68%) reported spending 1 to 6 hours during the past year developing knowledge or skills specific to smoking cessation. Using a Likert scale of 1 to 10 (where 10 = definitely confident and 1 = definitely not confident), an 8 or higher was reported by 58% of the physicians for their ability to incorporate smoking cessation strategies into regular office visits and by 34% for their ability to set up an office environment to support smoking cessation strategies.

Although all of the physicians maintained smoke-free offices, resources to support smoking cessation varied among the practices. Of the 38 offices, 26 (68%) had patient education materials; 22 (58%) maintained a standard location in the medical record to document the patient’s smoking status; 2 (5%) recorded the patient’s smoking status at every visit; and 6 (16%) had a staff person assigned to smoking cessation activities. Although pharmaceutical samples of bupropion were available in 35 (92%) offices, only 12 (32%) had samples of nicotine-replacement therapy.

Women physicians, physicians with 10 years or fewer in practice, and those practicing in offices with a form for recording smoking status in a standard location in the medical record were significantly more likely to discuss tobacco with their patients Table 1. The 2 patient characteristics associated with discussion of tobacco were being younger than 65 years and being a new patient.

When these factors were included in a multivariable logistic regression model, patient age, new patient status, and the presence of a form for recording smoking status were found to be important independent predictors of tobacco discussion Table 3. One variable that was not retained in the model was being in practice for 10 years or fewer (this variable was highly correlated with having a form for recording smoking status). An additional finding in the model was an interaction between patient sex and age, with women 65 years and older being the group least likely to have tobacco discussed during the visit.

Discussion

Our study shows that tobacco is a common issue in primary care that is discussed in more than 1 in 5 office visits. There is substantial variation, however, in the extent to which primary care physicians incorporate smoking cessation activities into their practices. We saw some practices in which tobacco was rarely, if ever, mentioned and 1 practice in which smoking was addressed during 90% of visits. This widespread variation illustrates an opportunity for improvement.¹³

It can be difficult to address behavioral problems such as nicotine addiction in a busy primary care practice. Other barriers include perceived patient attitudes about quitting,⁶ a lack of office support systems, time constraints, and the need to respond to other urgent health needs.¹⁴ When patients are presenting with a variety of acute and chronic ailments, it is easy to forget preventive care issues, such as nicotine dependence.

There is a large and growing body of evidence showing that changes in office systems can improve smoking cessation practices.^1,4,15,16 We showed that some of the variation between offices in smoking cessation practices can be explained by the presence of charting systems that routinely identify smokers. Although only a handful of offices in our study documented smoking status at each visit, those that did discussed tobacco 3 times as often as those that did not do so routinely. Although the physicians in our study reported that they were confident in their abilities to develop systems to support smoking cessation, most offices had not implemented the types of office systems described in published guidelines available at the time of the study.¹⁷

Further improvements and greater efficiencies can be obtained by delegating specific activities to nonphysician personnel in the office.^18-20 Our data showed a greater than 50% increased frequency of tobacco discussions in offices that assigned specific staff persons to address smoking cessation. This difference, however, was not statistically significant. This may have been because of the small number of offices that had such a dedicated staff person, resulting in a small percentage of the total patient encounters with this factor present and therefore a loss of power to detect differences.

Tobacco was more than twice as likely to be addressed during office visits with new patients, perhaps as part of a comprehensive health assessment. This is consistent with a recent report showing that when smoking status was recorded, it was usually on a health history form at the back of the chart.²¹

Consistent with previous studies,^22-24 we found that women physicians and physicians more recently trained were more likely to ask about smoking. These same physicians were more likely to have a standard form to record smoking status. It may be that newer physicians were more likely to be exposed to protocols or similar charting materials during their training; this is only speculation, however, since it appears that few medical schools routinely include smoking cessation training in their curricula.²⁵

One of the strategies recommended by Prochaska and Goldstein²⁶ and others^27,28 is to tailor smoking cessation strategies to a patient’s readiness to quit, yet assessments of readiness to quit were rarely seen in our study. Although it is possible that physicians had established readiness to quit during previous encounters with these patients, current guidelines recommend that this readiness be re-established at each visit. Because we do not know what proportion of smokers were ready to quit, we do not know what proportion of patients should have received assistance in smoking cessation, such as discussing pharmacotherapy, setting a quit date, or arranging follow-up.

In the 1995 National Ambulatory Medical Care Survey, nicotine-replacement therapy was prescribed during 1.3% of office visits with smokers.²⁹ (At that time, nicotine-replacement therapy was only available by prescription, and bupropion was not yet a standard treatment for nicotine addiction.) In contrast, our more recent data suggest that discussions of pharmacotherapy are a very common feature of physicians’ smoking cessation activities and that bupropion is being discussed more often than nicotine replacement.

Limitations

Direct observation of clinical practices has the advantage of reducing recall bias and increasing objectivity, yet there are limitations to this method as well. First, the use of separate observers precluded us from measuring the reliability of data collection. Second, we did not collect information regarding the reason for patient visits, which may include situations where the discussion of tobacco was not feasible or appropriate. Third, our study did not allow us to identify all smokers seen in the clinic and used volunteer physicians. Both of these factors could lead to an overestimate of the frequency in which assistance is provided. Although we found that assistance with smoking cessation was offered during 33% of visits with smokers, physicians participating in the National Ambulatory Medical Care Survey only reported offering assistance during 21% of visits. Finally, our study was limited to practices in Kansas and may not reflect those in other areas of the country.

Conclusions

Although smoking cessation is be a common topic in some physician-patient encounters, there are widespread variations in how it is addressed in primary care practices. More comprehensive and efficient management of nicotine dependence may be possible if physicians addressed the infrastructure in their offices that can support smoking cessation activities. Many physicians may not yet be considering changing their office systems to enhance smoking cessation activities. Further efforts will be needed to identify the barriers to system changes and to help physicians integrate effective and efficient smoking cessation systems into their practices.

Acknowledgments

Partial funding for this project was provided through the following grants: The Robert Wood Johnson Foundation Generalist Physicians Faculty Scholars award (#032686, J.S. Ahluwalia); Kansas Academy of Family Physicians (J. Gladden); J.H. Baker Trust of La Crosse, Kansas (J Gladden); Kansas Association for Medically Underserved (J. Gladden); and a Primary Care Physician Education grant from the Kansas Health Foundation. We would like to thank the family physicians who not only provided a valuable learning experience for the students but also allowed the data collection necessary for our paper. We appreciate the commitment of the students who collected the data. We would like to thank Kristin Hedberg, MA, for preliminary data analysis and Timothy P. Daaleman, DO, and Delwyn Catley, PhD, for their careful review of early versions of this manuscript.

Related Resources

For Patients:

• QuitNet http://www.quitnet.com/qn_main.jtml Developed by Boston University, this site provides information and tools to people trying to quit smoking. It contains peer support programs, information on pharmaceuticals, a directory of local smoking cessation programs, and the latest news from the tobacco front.
• Quitsmokingsupport.com http://www.quitsmokingsupport.com/intro.htm This advertiser-supported site has dozens of pages on weight control, methods, interactive chat/support rooms, quit smoking articles, smokers’ lungs, etc.

For physicians:

• Smoking cessation guidelines. http://www.surgeongeneral.gov/tobacco/default.htm Clinical practice guidelines from the Surgeon General’s Web site with full text and references for online retrieval. Patient education materials and posters for the doctor’s office are also available.
• PSNonline http://psnonline.org/Medical%20Management%20Program/Smoking% 20Cessation/smoking_cessation_front_page.htm This site provides a copy of the sticker or stamp that includes smoking cessation in the vital signs. Several studies have shown that this can significantly improve smoking cessation counseling in the doctor’s office.

Every year physicians and patients receive hundreds of messages about guidelines for cancer screening. Ideally, physicians will adopt and adhere to the evidence-based clinical practice guidelines. By doing so, there is maximum application of a proven technology to those who can most benefit, and valuable resources are not wasted in examinations that are not based on good or fair evidence. However, many physicians are not adhering to cancer screening guidelines backed by good evidence.^1,2 Also, many are performing cancer screening procedures that are not recommended (either because of a lack of evidence or because they have been shown to be ineffective).³

Most of the literature on physician cancer screening has dealt with facilitators or barriers to the adoption of commonly recommended guidelines. These studies did not address the factors that affect physician practice when the guidelines are unclear or conflicting, or when they clearly recommend against the procedure. We defined an “unclear” guideline as a “C” recommendation (insufficient evidence to recommend the maneuver) from the Canadian Task Force on the Periodic Health Examination (CTFPHE).⁴ Guidelines were “conflicting” when at least 2 organizations gave different recommendations for the same cancer screening examination.

Despite the CTFPHE guidelines,⁴ inconsistencies in practice remain. Although the CTFPHE recommends that breast cancer screening begin at age 50 years, 59% of women aged 40 to 49 years reported having mammograms in 1994,⁵ a rate nearly equivalent to those aged 50 to 59 in Ontario.⁶ It is clear that family physicians—the major cancer screeners in many countries—are frequently not following guidelines. The use of ineffective procedures or those for which the evidence is unclear can waste scarce health resources and lead to harm for those whose test results are false positive. The objective of our study⁷ was to determine the factors involved in the cancer screening decisions of family physicians in situations where the clinical practice guidelines were unclear or conflicting (prostate-specific antigen testing, mammography for ages 40 to 49 years, colorectal tests) as opposed to when they were clear and uncontroversial.

Methods

Ten focus groups⁷ were conducted with 1 urban group and 1 rural group in each of 5 Canadian provinces: British Columbia (BC), Alberta (AB), Ontario (ON), Quebec (QC), and Prince Edward Island (PEI). Ethical approval was obtained from all participating institutions. We focused on family physicians because they are the main preventive health care providers in Canada, and physician recommendation is the most important predictor of whether an individual obtains a particular screening test.⁸ Eight focus groups were conducted face to face, and 2 were done by teleconference because of the geographic remoteness of 2 rural areas. Each focus group was co-facilitated by a local research assistant and 1 of the investigators. The focus group moderators participated in a 2-hour training session to ensure standardization across sites. The group sessions lasted approximately 60 to 90 minutes; all were audiotaped and transcribed verbatim.*Table w1

Recruitment and Sampling

We used maximum variation sampling to ensure heterogeneity within the groups and to recruit physicians who would serve as information-rich participants with a wide range for age, practice type, location, and education.^9,10 Recruitment involved a 2-step process:¹¹ First, urban and rural family physicians were randomly selected from lists provided by each local area’s licensing body; and second, physician recruiters (“leader figures”) from each local area identified physicians who they believed would provide an adequate variance of opinions.

Data Collection and Analysis

Data collection and analysis occurred iteratively.^12-14 After every focus group 3 investigators reviewed transcripts independently to identify the central issues that emerged. Over several meetings they compared and combined their independent analyses. Emerging themes were explored and expanded in subsequent focus groups. Although saturation¹⁵ had been achieved by the 8th focus group, we completed the final 2 groups to ensure regional representation. The second step in the analysis involved determining the similarities, differences, and potential connections among key words, phrases, and concepts within and among each focus group transcript. Finally, the themes and subcategories of all focus groups were compared and contrasted, and the quotes that most accurately illustrated the themes were identified.

Trustworthiness and Validation

All groups were audiotaped and transcribed verbatim, and extensive field notes were made during the focus groups and throughout the analysis. Validation of the data was achieved by conducting member-checking interviews¹⁶ with 15 information-rich participants from the focus groups after completion of the initial analysis. We then refined the themes.

Results

The physicians’ demographics Table 1 reflect the Canadian family physician population.¹⁷ Three major themes emerged from the analysis as determinants of cancer screening with unclear or controversial guidelines: patient factors, physician factors, and physician-patient relationship factors Table 2.

Patient Factors

Patient factors included expectations, anxieties, family history, peers, and media influences. Many of the physician participants commented that patient expectations and demands for screening were major determinants of their decision to screen when guidelines were unclear. Although they expressed discomfort with this behavior, physicians acknowledged being frequently swayed by patient demands. One said, “I think that if the patient comes into my office and he wants something, that influences me a hell of a lot.” (QC rural)

The physicians also suggested that patients’ anxieties about cancer were important. The higher the perceived anxiety, the more likely they were to order the relevant cancer screening test, even if the recommendations were unclear. A participant said, “If a patient came in with a particular anxiety and would be allayed by [screening]…I would go ahead and recommend it.” (BC rural)

The presence of any positive family history appeared to influence the physicians’ screening decisions, even if it was not a recognized risk factor in the cancer screening guideline. Physicians also felt that the media is an important influence on patients’ requests for screening. One of the physicians said, “I think the media really influences a lot of patients, and unfortunately it doesn’t always give them the correct information.” (ON urban)

Physician Factors

Physician factors included the perception of guidelines, clinical practice experience, the influence of colleagues, the distinction between the screening styles of specialists and family physicians, and the time and financial costs involved in performing the screening maneuver. The 2 most important determinants appeared to be the physicians’ perceptions of guidelines and their clinical experience.

The physicians’ perception of guidelines had 5 components Table 2. First, many physicians saw guidelines as just guidelines and not as directives. This was most evident when the guideline was viewed as unclear or conflicting. Second, many indicated that unclear guidelines are not guidelines at all and that their task was to individualize the screening decisions to patients and their situations. A participant said, “If they’re unclear, then you have to use your judgment in terms of the patient, your patient population, their follow-up ability, what their risk factors, age, etcetera, are.” (AB rural)

The third perception of guidelines was confusion due to the multiplicity and changing nature of guidelines. One physician said, “As far as breast cancer goes, it appears…things are still…in flux…changing all the time.” (ON urban)

The physicians’ degree of trust in the source of the guideline was the fourth component. A participant said, “If you get a guideline from a consensus group where…a group of specialists get together…including some family docs…certainly I would take that with more…clout.” (AB rural)

The fifth component was the perceived effectiveness of a particular screening maneuver. One physician said, “In the…years that we’ve been [screening] we have found cancers at the stage A and B…that have been easily looked after…. We have not had 1 patient pass away.” (AB rural)

Physicians viewed their clinical experience as influencing their cancer screening decisions, and many felt that they were much more likely to order screening tests early in their careers. A participant said, “In terms of screening there’s a tendency, especially when you’re young and keen and scared, that you’re gonna miss something.” (AB urban)

Physicians were concerned about missing a diagnosis of cancer. If they actually had such an experience, it subsequently lowered their threshold for cancer screening for some time afterward. One physician said, “Suppose you missed a case of colorectal cancer, and someone else finds it; then you tend to run gun shy for a long time and perhaps overinvestigate and over-refer for a time.” (BC rural)

Colleagues could positively or negatively influence screening decisions. A participant said, “Some guidelines come out, and somebody will say, ‘Oh that’s trash. I’m not going to do that.’ And then it’s a little hard for the rest of us to easily incorporate that.” (BC rural)

Family physicians also felt that they had a unique screening style compared with specialists, stemming from their continuing long-term relationships with their patients. One physician said, “The specialists will tend to jump on the blood test wagon a lot faster than I think we will, because again they don’t know the patients.” (AB urban)

Time and financial costs were also identified as important practice factors in the decision-making process. A participant said, “Economics also plays a part…because it can take…half an hour to explain to a patient why you don’t want to do something. It can take 2 minutes to do it.” (ON rural)

The Physician-Patient Relationship

Decisions about cancer screening took place within an interactive relationship between the patient and physician. Physicians characterized the relationship as one of varying intensity and depth, and there appeared to be 3 key points about the relationship in terms of cancer screening. First, the stronger and more positive the relationship, the more likely that the physician would feel free to engage the patient in a discussion about not performing a test that is based on an unclear or negative guideline. One physician said, “If you’ve known somebody for a long time and they come to you with something that you don’t think is right, it’s a little bit easier to talk to them.” (PEI rural)

Conversely, if the relationship was new or tenuous, physicians felt “The lack of a good relationship has an impact…they tend not to go along with your recommendations.” (AB rural)

The second point regarding the physician-patient relationship was that when a guideline was unclear, it often called for a different interaction than when the guideline was clear. It involved more information giving, presented in a manner that assisted the patient. One physician said, “I try to give the patient as much information as I have, in words that they will understand, so that they can come to an informed decision. That’s what I do when the guidelines are unclear.” (ON urban)

The process of information-giving promoted finding common ground, particularly when patients were requesting a screening maneuver not backed by clear evidence. One participant said:[For] patients who want tests that we don’t necessarily think are indicated, I follow the evidence, and that’s a negotiation. …an explanation of the evidence and then almost throw it back at the patient...it’s not medical-legal. It’s not economic. It’s between me and my patient. (QC urban)

Finally, many physician participants observed that even when the guidelines are clear, many cancer screening decisions are not. As a result, they noted that this often necessitated a process of finding common ground by engaging patients in mutual decision making.

Discussion

Many of the factors we identified have been described previously.^18-36 However, to the best of our knowledge they have not been combined into a comprehensive typology for cancer screening decision making that includes the physician-patient relationship and that deals with unclear and conflicting guidelines. One conceptual framework for the determinants of screening behavior²² is based on pediatric vaccinations and does not include unclear or controversial guidelines. Another more recent model is based on cancer care, not screening, but it does include some elements of communication between provider and patient.²³ Our proposed model of decision making regarding cancer screening Figure 1 is a modification of these frameworks based on our findings and is specific to decisions about cancer screening.

One unique feature of our model is that it is embedded in the physician-patient relationship. In particular, the quality of this relationship and the clarity of the recommendation appears to be most important. It involves an interactive process and mutual discussion with the patient. This ultimately includes finding agreement and culminates in a mutual agreement between the patient and the physician about the cancer screening maneuver.^37,38 Our findings are also in concordance with other literature on physician test-ordering. The concern about missing a diagnosis of cancer is similar to “chagrin bias” —when physicians are more likely to order inappropriate chest radiographs if they anticipated feeling regret if they missed a diagnosis of pneumonia.³⁹

Limitations

Although attempts were made to have regional representation from the entire country (Canada), the findings may not be transferable to other family medicine settings. Two of the 10 focus groups were conducted by teleconference, which may bias results, because it is a different data collection method. However, previous experience with telephone focus groups had been successful. (C.H., personal communication) The 2 teleconference groups did not provide markedly different data from those conducted in-person. Also, because of budget restraints, 5 different moderators were used. The investigators organized training sessions to standardize focus group moderation across sites; however, it is difficult to estimate the potential bias, given that moderators have their own styles. Finally, the data were based on the perspective of physicians and not patients.

Future Research

In the next phase of our study we will test the model’s factors quantitatively on a random sample of physicians and go through the same steps with a patient/consumer sample. Ultimately, we will use a modified model to design interventions to assist with the implementation of preventive services guidelines.

Conclusions

Our findings are of importance for those implementing preventive care guidelines. The focus group participants were clearly less happy with guidelines that were equivocal, and were less likely to follow them. Patient factors and the physician-patient relationship appear to be important in such cases. Although patient-oriented decision aids could help physicians in these situations, it is clearly more difficult to develop aids to guide patients in settings when the evidence is unclear, because the information required is more complex. The family physicians’ perceptions of the effectiveness of a particular screening test was very important, perhaps more important to the participants than the scientific evidence behind a guideline. Although personal experience is a weak and unscientific level of evidence subject to many biases, it is likely an important influence on cancer screening decision making in primary care, particularly when the evidence is uncertain. Future education efforts directed at primary care providers should address the influence of personal experience as well as the failure to attend to the level of evidence behind recommendations.

Acknowledgments

Our project was funded by a peer-reviewed grant from the Medical Research Council of Canada (grant number 14673) and by the Prince Edward Island Cancer Research Council. We wish to thank the staff of the Department of Family and Community Medicine, University of Toronto, for their tireless support of this project.

Every year physicians and patients receive hundreds of messages about guidelines for cancer screening. Ideally, physicians will adopt and adhere to the evidence-based clinical practice guidelines. By doing so, there is maximum application of a proven technology to those who can most benefit, and valuable resources are not wasted in examinations that are not based on good or fair evidence. However, many physicians are not adhering to cancer screening guidelines backed by good evidence.^1,2 Also, many are performing cancer screening procedures that are not recommended (either because of a lack of evidence or because they have been shown to be ineffective).³

Most of the literature on physician cancer screening has dealt with facilitators or barriers to the adoption of commonly recommended guidelines. These studies did not address the factors that affect physician practice when the guidelines are unclear or conflicting, or when they clearly recommend against the procedure. We defined an “unclear” guideline as a “C” recommendation (insufficient evidence to recommend the maneuver) from the Canadian Task Force on the Periodic Health Examination (CTFPHE).⁴ Guidelines were “conflicting” when at least 2 organizations gave different recommendations for the same cancer screening examination.

Despite the CTFPHE guidelines,⁴ inconsistencies in practice remain. Although the CTFPHE recommends that breast cancer screening begin at age 50 years, 59% of women aged 40 to 49 years reported having mammograms in 1994,⁵ a rate nearly equivalent to those aged 50 to 59 in Ontario.⁶ It is clear that family physicians—the major cancer screeners in many countries—are frequently not following guidelines. The use of ineffective procedures or those for which the evidence is unclear can waste scarce health resources and lead to harm for those whose test results are false positive. The objective of our study⁷ was to determine the factors involved in the cancer screening decisions of family physicians in situations where the clinical practice guidelines were unclear or conflicting (prostate-specific antigen testing, mammography for ages 40 to 49 years, colorectal tests) as opposed to when they were clear and uncontroversial.

Methods

Ten focus groups⁷ were conducted with 1 urban group and 1 rural group in each of 5 Canadian provinces: British Columbia (BC), Alberta (AB), Ontario (ON), Quebec (QC), and Prince Edward Island (PEI). Ethical approval was obtained from all participating institutions. We focused on family physicians because they are the main preventive health care providers in Canada, and physician recommendation is the most important predictor of whether an individual obtains a particular screening test.⁸ Eight focus groups were conducted face to face, and 2 were done by teleconference because of the geographic remoteness of 2 rural areas. Each focus group was co-facilitated by a local research assistant and 1 of the investigators. The focus group moderators participated in a 2-hour training session to ensure standardization across sites. The group sessions lasted approximately 60 to 90 minutes; all were audiotaped and transcribed verbatim.*Table w1

Recruitment and Sampling

We used maximum variation sampling to ensure heterogeneity within the groups and to recruit physicians who would serve as information-rich participants with a wide range for age, practice type, location, and education.^9,10 Recruitment involved a 2-step process:¹¹ First, urban and rural family physicians were randomly selected from lists provided by each local area’s licensing body; and second, physician recruiters (“leader figures”) from each local area identified physicians who they believed would provide an adequate variance of opinions.

Data Collection and Analysis

Data collection and analysis occurred iteratively.^12-14 After every focus group 3 investigators reviewed transcripts independently to identify the central issues that emerged. Over several meetings they compared and combined their independent analyses. Emerging themes were explored and expanded in subsequent focus groups. Although saturation¹⁵ had been achieved by the 8th focus group, we completed the final 2 groups to ensure regional representation. The second step in the analysis involved determining the similarities, differences, and potential connections among key words, phrases, and concepts within and among each focus group transcript. Finally, the themes and subcategories of all focus groups were compared and contrasted, and the quotes that most accurately illustrated the themes were identified.

Trustworthiness and Validation

All groups were audiotaped and transcribed verbatim, and extensive field notes were made during the focus groups and throughout the analysis. Validation of the data was achieved by conducting member-checking interviews¹⁶ with 15 information-rich participants from the focus groups after completion of the initial analysis. We then refined the themes.

Results

The physicians’ demographics Table 1 reflect the Canadian family physician population.¹⁷ Three major themes emerged from the analysis as determinants of cancer screening with unclear or controversial guidelines: patient factors, physician factors, and physician-patient relationship factors Table 2.

Patient Factors

Patient factors included expectations, anxieties, family history, peers, and media influences. Many of the physician participants commented that patient expectations and demands for screening were major determinants of their decision to screen when guidelines were unclear. Although they expressed discomfort with this behavior, physicians acknowledged being frequently swayed by patient demands. One said, “I think that if the patient comes into my office and he wants something, that influences me a hell of a lot.” (QC rural)

The physicians also suggested that patients’ anxieties about cancer were important. The higher the perceived anxiety, the more likely they were to order the relevant cancer screening test, even if the recommendations were unclear. A participant said, “If a patient came in with a particular anxiety and would be allayed by [screening]…I would go ahead and recommend it.” (BC rural)

The presence of any positive family history appeared to influence the physicians’ screening decisions, even if it was not a recognized risk factor in the cancer screening guideline. Physicians also felt that the media is an important influence on patients’ requests for screening. One of the physicians said, “I think the media really influences a lot of patients, and unfortunately it doesn’t always give them the correct information.” (ON urban)

Physician Factors

Physician factors included the perception of guidelines, clinical practice experience, the influence of colleagues, the distinction between the screening styles of specialists and family physicians, and the time and financial costs involved in performing the screening maneuver. The 2 most important determinants appeared to be the physicians’ perceptions of guidelines and their clinical experience.

The physicians’ perception of guidelines had 5 components Table 2. First, many physicians saw guidelines as just guidelines and not as directives. This was most evident when the guideline was viewed as unclear or conflicting. Second, many indicated that unclear guidelines are not guidelines at all and that their task was to individualize the screening decisions to patients and their situations. A participant said, “If they’re unclear, then you have to use your judgment in terms of the patient, your patient population, their follow-up ability, what their risk factors, age, etcetera, are.” (AB rural)

The third perception of guidelines was confusion due to the multiplicity and changing nature of guidelines. One physician said, “As far as breast cancer goes, it appears…things are still…in flux…changing all the time.” (ON urban)

The physicians’ degree of trust in the source of the guideline was the fourth component. A participant said, “If you get a guideline from a consensus group where…a group of specialists get together…including some family docs…certainly I would take that with more…clout.” (AB rural)

The fifth component was the perceived effectiveness of a particular screening maneuver. One physician said, “In the…years that we’ve been [screening] we have found cancers at the stage A and B…that have been easily looked after…. We have not had 1 patient pass away.” (AB rural)

Physicians viewed their clinical experience as influencing their cancer screening decisions, and many felt that they were much more likely to order screening tests early in their careers. A participant said, “In terms of screening there’s a tendency, especially when you’re young and keen and scared, that you’re gonna miss something.” (AB urban)

Physicians were concerned about missing a diagnosis of cancer. If they actually had such an experience, it subsequently lowered their threshold for cancer screening for some time afterward. One physician said, “Suppose you missed a case of colorectal cancer, and someone else finds it; then you tend to run gun shy for a long time and perhaps overinvestigate and over-refer for a time.” (BC rural)

Colleagues could positively or negatively influence screening decisions. A participant said, “Some guidelines come out, and somebody will say, ‘Oh that’s trash. I’m not going to do that.’ And then it’s a little hard for the rest of us to easily incorporate that.” (BC rural)

Family physicians also felt that they had a unique screening style compared with specialists, stemming from their continuing long-term relationships with their patients. One physician said, “The specialists will tend to jump on the blood test wagon a lot faster than I think we will, because again they don’t know the patients.” (AB urban)

Time and financial costs were also identified as important practice factors in the decision-making process. A participant said, “Economics also plays a part…because it can take…half an hour to explain to a patient why you don’t want to do something. It can take 2 minutes to do it.” (ON rural)

The Physician-Patient Relationship

Decisions about cancer screening took place within an interactive relationship between the patient and physician. Physicians characterized the relationship as one of varying intensity and depth, and there appeared to be 3 key points about the relationship in terms of cancer screening. First, the stronger and more positive the relationship, the more likely that the physician would feel free to engage the patient in a discussion about not performing a test that is based on an unclear or negative guideline. One physician said, “If you’ve known somebody for a long time and they come to you with something that you don’t think is right, it’s a little bit easier to talk to them.” (PEI rural)

Conversely, if the relationship was new or tenuous, physicians felt “The lack of a good relationship has an impact…they tend not to go along with your recommendations.” (AB rural)

The second point regarding the physician-patient relationship was that when a guideline was unclear, it often called for a different interaction than when the guideline was clear. It involved more information giving, presented in a manner that assisted the patient. One physician said, “I try to give the patient as much information as I have, in words that they will understand, so that they can come to an informed decision. That’s what I do when the guidelines are unclear.” (ON urban)

The process of information-giving promoted finding common ground, particularly when patients were requesting a screening maneuver not backed by clear evidence. One participant said:[For] patients who want tests that we don’t necessarily think are indicated, I follow the evidence, and that’s a negotiation. …an explanation of the evidence and then almost throw it back at the patient...it’s not medical-legal. It’s not economic. It’s between me and my patient. (QC urban)

Finally, many physician participants observed that even when the guidelines are clear, many cancer screening decisions are not. As a result, they noted that this often necessitated a process of finding common ground by engaging patients in mutual decision making.

Discussion

Many of the factors we identified have been described previously.^18-36 However, to the best of our knowledge they have not been combined into a comprehensive typology for cancer screening decision making that includes the physician-patient relationship and that deals with unclear and conflicting guidelines. One conceptual framework for the determinants of screening behavior²² is based on pediatric vaccinations and does not include unclear or controversial guidelines. Another more recent model is based on cancer care, not screening, but it does include some elements of communication between provider and patient.²³ Our proposed model of decision making regarding cancer screening Figure 1 is a modification of these frameworks based on our findings and is specific to decisions about cancer screening.

One unique feature of our model is that it is embedded in the physician-patient relationship. In particular, the quality of this relationship and the clarity of the recommendation appears to be most important. It involves an interactive process and mutual discussion with the patient. This ultimately includes finding agreement and culminates in a mutual agreement between the patient and the physician about the cancer screening maneuver.^37,38 Our findings are also in concordance with other literature on physician test-ordering. The concern about missing a diagnosis of cancer is similar to “chagrin bias” —when physicians are more likely to order inappropriate chest radiographs if they anticipated feeling regret if they missed a diagnosis of pneumonia.³⁹

Limitations

Although attempts were made to have regional representation from the entire country (Canada), the findings may not be transferable to other family medicine settings. Two of the 10 focus groups were conducted by teleconference, which may bias results, because it is a different data collection method. However, previous experience with telephone focus groups had been successful. (C.H., personal communication) The 2 teleconference groups did not provide markedly different data from those conducted in-person. Also, because of budget restraints, 5 different moderators were used. The investigators organized training sessions to standardize focus group moderation across sites; however, it is difficult to estimate the potential bias, given that moderators have their own styles. Finally, the data were based on the perspective of physicians and not patients.

Future Research

In the next phase of our study we will test the model’s factors quantitatively on a random sample of physicians and go through the same steps with a patient/consumer sample. Ultimately, we will use a modified model to design interventions to assist with the implementation of preventive services guidelines.

Conclusions

Our findings are of importance for those implementing preventive care guidelines. The focus group participants were clearly less happy with guidelines that were equivocal, and were less likely to follow them. Patient factors and the physician-patient relationship appear to be important in such cases. Although patient-oriented decision aids could help physicians in these situations, it is clearly more difficult to develop aids to guide patients in settings when the evidence is unclear, because the information required is more complex. The family physicians’ perceptions of the effectiveness of a particular screening test was very important, perhaps more important to the participants than the scientific evidence behind a guideline. Although personal experience is a weak and unscientific level of evidence subject to many biases, it is likely an important influence on cancer screening decision making in primary care, particularly when the evidence is uncertain. Future education efforts directed at primary care providers should address the influence of personal experience as well as the failure to attend to the level of evidence behind recommendations.

Acknowledgments

Our project was funded by a peer-reviewed grant from the Medical Research Council of Canada (grant number 14673) and by the Prince Edward Island Cancer Research Council. We wish to thank the staff of the Department of Family and Community Medicine, University of Toronto, for their tireless support of this project.

Every year physicians and patients receive hundreds of messages about guidelines for cancer screening. Ideally, physicians will adopt and adhere to the evidence-based clinical practice guidelines. By doing so, there is maximum application of a proven technology to those who can most benefit, and valuable resources are not wasted in examinations that are not based on good or fair evidence. However, many physicians are not adhering to cancer screening guidelines backed by good evidence.^1,2 Also, many are performing cancer screening procedures that are not recommended (either because of a lack of evidence or because they have been shown to be ineffective).³

Most of the literature on physician cancer screening has dealt with facilitators or barriers to the adoption of commonly recommended guidelines. These studies did not address the factors that affect physician practice when the guidelines are unclear or conflicting, or when they clearly recommend against the procedure. We defined an “unclear” guideline as a “C” recommendation (insufficient evidence to recommend the maneuver) from the Canadian Task Force on the Periodic Health Examination (CTFPHE).⁴ Guidelines were “conflicting” when at least 2 organizations gave different recommendations for the same cancer screening examination.

Despite the CTFPHE guidelines,⁴ inconsistencies in practice remain. Although the CTFPHE recommends that breast cancer screening begin at age 50 years, 59% of women aged 40 to 49 years reported having mammograms in 1994,⁵ a rate nearly equivalent to those aged 50 to 59 in Ontario.⁶ It is clear that family physicians—the major cancer screeners in many countries—are frequently not following guidelines. The use of ineffective procedures or those for which the evidence is unclear can waste scarce health resources and lead to harm for those whose test results are false positive. The objective of our study⁷ was to determine the factors involved in the cancer screening decisions of family physicians in situations where the clinical practice guidelines were unclear or conflicting (prostate-specific antigen testing, mammography for ages 40 to 49 years, colorectal tests) as opposed to when they were clear and uncontroversial.

Methods

Ten focus groups⁷ were conducted with 1 urban group and 1 rural group in each of 5 Canadian provinces: British Columbia (BC), Alberta (AB), Ontario (ON), Quebec (QC), and Prince Edward Island (PEI). Ethical approval was obtained from all participating institutions. We focused on family physicians because they are the main preventive health care providers in Canada, and physician recommendation is the most important predictor of whether an individual obtains a particular screening test.⁸ Eight focus groups were conducted face to face, and 2 were done by teleconference because of the geographic remoteness of 2 rural areas. Each focus group was co-facilitated by a local research assistant and 1 of the investigators. The focus group moderators participated in a 2-hour training session to ensure standardization across sites. The group sessions lasted approximately 60 to 90 minutes; all were audiotaped and transcribed verbatim.*Table w1

Recruitment and Sampling

We used maximum variation sampling to ensure heterogeneity within the groups and to recruit physicians who would serve as information-rich participants with a wide range for age, practice type, location, and education.^9,10 Recruitment involved a 2-step process:¹¹ First, urban and rural family physicians were randomly selected from lists provided by each local area’s licensing body; and second, physician recruiters (“leader figures”) from each local area identified physicians who they believed would provide an adequate variance of opinions.

Data Collection and Analysis

Data collection and analysis occurred iteratively.^12-14 After every focus group 3 investigators reviewed transcripts independently to identify the central issues that emerged. Over several meetings they compared and combined their independent analyses. Emerging themes were explored and expanded in subsequent focus groups. Although saturation¹⁵ had been achieved by the 8th focus group, we completed the final 2 groups to ensure regional representation. The second step in the analysis involved determining the similarities, differences, and potential connections among key words, phrases, and concepts within and among each focus group transcript. Finally, the themes and subcategories of all focus groups were compared and contrasted, and the quotes that most accurately illustrated the themes were identified.

Trustworthiness and Validation

All groups were audiotaped and transcribed verbatim, and extensive field notes were made during the focus groups and throughout the analysis. Validation of the data was achieved by conducting member-checking interviews¹⁶ with 15 information-rich participants from the focus groups after completion of the initial analysis. We then refined the themes.

Results

The physicians’ demographics Table 1 reflect the Canadian family physician population.¹⁷ Three major themes emerged from the analysis as determinants of cancer screening with unclear or controversial guidelines: patient factors, physician factors, and physician-patient relationship factors Table 2.

Patient Factors

Patient factors included expectations, anxieties, family history, peers, and media influences. Many of the physician participants commented that patient expectations and demands for screening were major determinants of their decision to screen when guidelines were unclear. Although they expressed discomfort with this behavior, physicians acknowledged being frequently swayed by patient demands. One said, “I think that if the patient comes into my office and he wants something, that influences me a hell of a lot.” (QC rural)

The physicians also suggested that patients’ anxieties about cancer were important. The higher the perceived anxiety, the more likely they were to order the relevant cancer screening test, even if the recommendations were unclear. A participant said, “If a patient came in with a particular anxiety and would be allayed by [screening]…I would go ahead and recommend it.” (BC rural)

The presence of any positive family history appeared to influence the physicians’ screening decisions, even if it was not a recognized risk factor in the cancer screening guideline. Physicians also felt that the media is an important influence on patients’ requests for screening. One of the physicians said, “I think the media really influences a lot of patients, and unfortunately it doesn’t always give them the correct information.” (ON urban)

Physician Factors

Physician factors included the perception of guidelines, clinical practice experience, the influence of colleagues, the distinction between the screening styles of specialists and family physicians, and the time and financial costs involved in performing the screening maneuver. The 2 most important determinants appeared to be the physicians’ perceptions of guidelines and their clinical experience.

The physicians’ perception of guidelines had 5 components Table 2. First, many physicians saw guidelines as just guidelines and not as directives. This was most evident when the guideline was viewed as unclear or conflicting. Second, many indicated that unclear guidelines are not guidelines at all and that their task was to individualize the screening decisions to patients and their situations. A participant said, “If they’re unclear, then you have to use your judgment in terms of the patient, your patient population, their follow-up ability, what their risk factors, age, etcetera, are.” (AB rural)

The third perception of guidelines was confusion due to the multiplicity and changing nature of guidelines. One physician said, “As far as breast cancer goes, it appears…things are still…in flux…changing all the time.” (ON urban)

The physicians’ degree of trust in the source of the guideline was the fourth component. A participant said, “If you get a guideline from a consensus group where…a group of specialists get together…including some family docs…certainly I would take that with more…clout.” (AB rural)

The fifth component was the perceived effectiveness of a particular screening maneuver. One physician said, “In the…years that we’ve been [screening] we have found cancers at the stage A and B…that have been easily looked after…. We have not had 1 patient pass away.” (AB rural)

Physicians viewed their clinical experience as influencing their cancer screening decisions, and many felt that they were much more likely to order screening tests early in their careers. A participant said, “In terms of screening there’s a tendency, especially when you’re young and keen and scared, that you’re gonna miss something.” (AB urban)

Physicians were concerned about missing a diagnosis of cancer. If they actually had such an experience, it subsequently lowered their threshold for cancer screening for some time afterward. One physician said, “Suppose you missed a case of colorectal cancer, and someone else finds it; then you tend to run gun shy for a long time and perhaps overinvestigate and over-refer for a time.” (BC rural)

Colleagues could positively or negatively influence screening decisions. A participant said, “Some guidelines come out, and somebody will say, ‘Oh that’s trash. I’m not going to do that.’ And then it’s a little hard for the rest of us to easily incorporate that.” (BC rural)

Family physicians also felt that they had a unique screening style compared with specialists, stemming from their continuing long-term relationships with their patients. One physician said, “The specialists will tend to jump on the blood test wagon a lot faster than I think we will, because again they don’t know the patients.” (AB urban)

Time and financial costs were also identified as important practice factors in the decision-making process. A participant said, “Economics also plays a part…because it can take…half an hour to explain to a patient why you don’t want to do something. It can take 2 minutes to do it.” (ON rural)

The Physician-Patient Relationship

Decisions about cancer screening took place within an interactive relationship between the patient and physician. Physicians characterized the relationship as one of varying intensity and depth, and there appeared to be 3 key points about the relationship in terms of cancer screening. First, the stronger and more positive the relationship, the more likely that the physician would feel free to engage the patient in a discussion about not performing a test that is based on an unclear or negative guideline. One physician said, “If you’ve known somebody for a long time and they come to you with something that you don’t think is right, it’s a little bit easier to talk to them.” (PEI rural)

Conversely, if the relationship was new or tenuous, physicians felt “The lack of a good relationship has an impact…they tend not to go along with your recommendations.” (AB rural)

The second point regarding the physician-patient relationship was that when a guideline was unclear, it often called for a different interaction than when the guideline was clear. It involved more information giving, presented in a manner that assisted the patient. One physician said, “I try to give the patient as much information as I have, in words that they will understand, so that they can come to an informed decision. That’s what I do when the guidelines are unclear.” (ON urban)

The process of information-giving promoted finding common ground, particularly when patients were requesting a screening maneuver not backed by clear evidence. One participant said:[For] patients who want tests that we don’t necessarily think are indicated, I follow the evidence, and that’s a negotiation. …an explanation of the evidence and then almost throw it back at the patient...it’s not medical-legal. It’s not economic. It’s between me and my patient. (QC urban)

Finally, many physician participants observed that even when the guidelines are clear, many cancer screening decisions are not. As a result, they noted that this often necessitated a process of finding common ground by engaging patients in mutual decision making.

Discussion

Many of the factors we identified have been described previously.^18-36 However, to the best of our knowledge they have not been combined into a comprehensive typology for cancer screening decision making that includes the physician-patient relationship and that deals with unclear and conflicting guidelines. One conceptual framework for the determinants of screening behavior²² is based on pediatric vaccinations and does not include unclear or controversial guidelines. Another more recent model is based on cancer care, not screening, but it does include some elements of communication between provider and patient.²³ Our proposed model of decision making regarding cancer screening Figure 1 is a modification of these frameworks based on our findings and is specific to decisions about cancer screening.

One unique feature of our model is that it is embedded in the physician-patient relationship. In particular, the quality of this relationship and the clarity of the recommendation appears to be most important. It involves an interactive process and mutual discussion with the patient. This ultimately includes finding agreement and culminates in a mutual agreement between the patient and the physician about the cancer screening maneuver.^37,38 Our findings are also in concordance with other literature on physician test-ordering. The concern about missing a diagnosis of cancer is similar to “chagrin bias” —when physicians are more likely to order inappropriate chest radiographs if they anticipated feeling regret if they missed a diagnosis of pneumonia.³⁹

Limitations

Although attempts were made to have regional representation from the entire country (Canada), the findings may not be transferable to other family medicine settings. Two of the 10 focus groups were conducted by teleconference, which may bias results, because it is a different data collection method. However, previous experience with telephone focus groups had been successful. (C.H., personal communication) The 2 teleconference groups did not provide markedly different data from those conducted in-person. Also, because of budget restraints, 5 different moderators were used. The investigators organized training sessions to standardize focus group moderation across sites; however, it is difficult to estimate the potential bias, given that moderators have their own styles. Finally, the data were based on the perspective of physicians and not patients.

Future Research

In the next phase of our study we will test the model’s factors quantitatively on a random sample of physicians and go through the same steps with a patient/consumer sample. Ultimately, we will use a modified model to design interventions to assist with the implementation of preventive services guidelines.

Conclusions

Our findings are of importance for those implementing preventive care guidelines. The focus group participants were clearly less happy with guidelines that were equivocal, and were less likely to follow them. Patient factors and the physician-patient relationship appear to be important in such cases. Although patient-oriented decision aids could help physicians in these situations, it is clearly more difficult to develop aids to guide patients in settings when the evidence is unclear, because the information required is more complex. The family physicians’ perceptions of the effectiveness of a particular screening test was very important, perhaps more important to the participants than the scientific evidence behind a guideline. Although personal experience is a weak and unscientific level of evidence subject to many biases, it is likely an important influence on cancer screening decision making in primary care, particularly when the evidence is uncertain. Future education efforts directed at primary care providers should address the influence of personal experience as well as the failure to attend to the level of evidence behind recommendations.

Acknowledgments

Our project was funded by a peer-reviewed grant from the Medical Research Council of Canada (grant number 14673) and by the Prince Edward Island Cancer Research Council. We wish to thank the staff of the Department of Family and Community Medicine, University of Toronto, for their tireless support of this project.

It is difficult to provide high-quality care to elderly patients with diabetes, and this task may be even more problematic in rural areas.¹ There are fewer physicians in such areas, and chronic conditions may get short shrift from both physicians and patients.² The relative shortage of specialists in rural areas may make it more difficult for physicians and their patients to get some of the specialized services they may need.³ Knowledge about advances in diabetic care may diffuse more slowly to these areas, making it less likely that physicians and patients will be aware of or adhere to published guidelines.

Previous studies have shown that the rural elderly-particularly those living in the smallest and most remote areas-make fewer office visits to physicians.² These same patients are more likely to see family physicians-and less likely to visit specialists-than their urban counterparts.⁴ It is not known whether this is true specifically of patients with diabetes, and the impact of these patterns on adherence to generally accepted guidelines is unknown.

We examined rural-urban differences in the care of persons with diabetes to determine what kinds of locations promote high-quality care. It may be possible to improve diabetes care either through further training of generalists or by providing opportunities for formal consultation with relevant specialists within the communities where these patients live.

Methods

Our study was based on data from the Medicare program for Washington state in 1994. During that year 362,145 Medicare recipients 65 years and older used medical care, did not belong to a capitated plan, had continuous Medicare coverage, received all their medical care in Washington State, and were alive at the end of the year.

For the purposes of our study, a diabetic visit is defined as any visit to a physician in an ambulatory setting where that physician entered any of the following International Classification of Diseases–ninth revision codes as a diagnosis: 250.XX (diabetes), 362.01 and 262.02 (diabetic retinopathy), 357.2 (diabetic polyneuropathy), or 366.41 (diabetic cataract). Patients are considered to have diabetes if they have at least 2 physician encounters for 1 of these codes in an ambulatory setting on separate days.

Patient residence was determined by the residential ZIP Code, and all patients were assigned as being rural or urban based on their residence.⁵ Rural communities were considered to be large if they had hospitals with more than 100 beds. The identity of the physician was determined from the unique physician identification number (UPIN) assigned by Medicare. UPINs were present 99.1% of the time, and specialty could be determined for 99.0% of these physicians.

Quality of Care Measurements

We created a core quality index of items that most authoritative sources agree should be performed regularly in patients with diabetes^6-9 and that can be identified using the Medicare Part B claims file.^10-12 The core quality index included a glycated hemoglobin determination, cholesterol measurement, and an eye examination by either an ophthalmologist or an optometrist. A service was considered to have been performed if a claim for any of the above items-or for a multi-test procedure of which that item is a part-was submitted by any provider during the 1994 study year.

Analytic Approach

We used the ambulatory care group (ACG) case-mix classification system to control for patient comorbidities.^13,14 Confidence intervals were calculated for independent and control variables in the logistic regression. Chi-square tests were used to compare results across different geographic areas. Because of multiple comparisons, we only report differences significant at the .01 level.

Results

According to our criteria, a total of 30,589 patients (56.4% women) representing 8.4% of all Medicare patients had diabetes. These patients made 392,831 outpatient visits to physicians during 1994, for an average of 12.8 visits per person. A diagnosis of diabetes was recorded for 42.7% of all outpatient visits by patients with diabetes.

Urban patients made more ambulatory visits overall than their rural counterparts, although there was no significant difference in the number of visits for diabetes. Patients living in small remote rural communities made significantly fewer ambulatory visits than patients living in any other place. The overall illness severity mirrored the number of ambulatory visits: 55.1% of urban patients and those living in large remote areas had 4 or more major chronic conditions; 51.3% of the group living in the small remote rural areas had the same burden of disease (P <.01).

Geographic location had a profound effect on where patients received their care. Urban patients received virtually all their outpatient care in their local urban areas (97.9%). Patients living in large rural communities also received most of their outpatient care in their own community. When patients in these communities did travel for care, they usually went to an urban community.

The small rural communities were much less self-sufficient, with almost half of all outpatient visits occurring outside the local community. Patients from small towns adjacent to cities went to urban areas. Patients from the remote small communities were more likely to get care in large rural communities; a substantial number, however, went to urban areas.

Generalists provided most of the care for patients with diabetes Table 1. Family physicians and general internists accounted for 62.4% of all visits coded for diabetes. The smaller and more remote the area, the higher the proportion of visits to family physicians. Endocrinologists, who handle more than 11% of the outpatient diabetic visits of the urban elderly, were seen for only 3% of the diabetic visits of those living in small remote communities . Urban patients were much more likely to consult an endocrinologist than their rural counterparts; 16.3% of urban patients visited an endocrinologist at least once during the year, compared with 6.9% of rural patients.

Adherence to Guidelines

The majority of patients had their cholesterol and glycated hemoglobin measured and their eyes examined at least once during the study year Table 2, although only 27.5% of patients had all 3 determinations performed. Urban patients were significantly more likely to have their glycated hemoglobin and cholesterol levels measured than rural patients, although the differences were small. Most patients who had glycated hemoglobin measured had either 1 or 2 such tests during the study year, with 31.3% of patients receiving 2 glycated hemoglobin determinations during the year.

Patients living in large remote rural communities were significantly more likely to have received all 3 of the core diabetes quality measures than patients in any of the other areas. By contrast, patients living in large rural communities adjacent to metropolitan areas were much less likely to have a glycated hemoglobin determination or an eye examination. Small rural towns had essentially identical screening rates, independent of their proximity to an urban area.

The specialty of the physicians was not associated with differences in adherence to screening guidelines, with one exception. Patients who saw an endocrinologist at least once during the year were much more likely to have received a glycated hemoglobin determination. Of patients who saw an endocrinologist, 77.9% received this test versus 51.0% of the patients with diabetes who had not seen an endocrinologist. The proportion of eye examinations and cholesterol measurements were also higher for patients who consulted an endocrinologist, although the differences are not as large as for glycated hemoglobin tests.

We used logistic regression to test the independent effect of patient residence on the likelihood of receiving the recommended tests.*Table w1 Patient residence is associated with significant differences in the likelihood that a patient received a glycated hemoglobin test. Patients living in large rural communities adjacent to metropolitan areas were significantly less likely to have a glycated hemoglobin determination than patients living in all other locations, even after controlling for sociodemographic factors, illness severity, and physician specialty. By contrast, patients living in large remote areas were much more likely to have received the test. Patients living in small remote rural areas received the test at a rate similar to that of patients living in urban areas, all other factors being equal. The single variable with the greatest independent effect was whether the patient saw an endocrinologist during the year.

A similar pattern prevails when using the core diabetes quality index in a multiple linear regression (not tabled). Study variables explain 18.18% of the variance in the index value. All 4 of the rural residential variables were statistically significant; patients living in remote large rural areas had a greater likelihood of receiving the recommended tests after controlling for potential confounders, while patients living in other types of rural areas were less likely to receive the tests.

Discussion

The quality of outpatient care for elderly persons with diabetes leaves much to be desired.^10-12,15 On a national level, only 21% of patients received a glycated hemoglobin determination in 1994, perhaps the best single summary of diabetic control available to physicians.^10,11 In our study of Washington for the same year, a much higher proportion of patients received this test, suggesting the existence of major regional differences. Yet even in our study, almost half of patients with a diagnosis of diabetes did not receive a glycated hemoglobin determination even though Medicare reimburses separately for this test. Only 27.5% received all 3 of the tests recommended by authoritative national organizations during the study year.

The location of the patients’ community affects their likelihood of receiving the recommended screening tests. Patients living in large rural communities remote from cities were significantly more likely to receive the recommended services than their urban counterparts; patients living in other rural locations were less likely to receive these services.

What might explain these findings? One contributing factor is the relative unavailability of endocrinologists in many rural communities. Rural patients who saw an endocrinologist at least once during the year were almost twice as likely to have had a glycated hemoglobin determination, probably because ordering such a test is part of the routine when endocrinologists see a new patient with diabetes.¹⁶ Only 24.6% of the visits to an endocrinologist occurred within the rural area where the patient lived, since most endocrinologists practice in urban areas. It is likely that this access barrier explains the much lower rate at which rural patients see endocrinologists and contributes to the lower rate of appropriate testing.

But this is not the only factor. There are very few endocrinologists in the state of Washington (69 in our study), and most diabetic care is provided by primary care physicians.¹⁷ The highest rate of guideline adherence occurs in large remote rural communities—communities that have endocrinologists but where the rate at which patients visit these specialists is still less than half of that in urban communities. It may be that large rural towns represent the best of both worlds: vibrant, rapidly growing communities with an adequate supply of both generalist and specialist physicians that serve as regional referral centers for surrounding rural towns.

Limitations

These data are based on the elderly Medicare population in Washington who are not members of managed care organizations. Managed care penetration in 1994 was relatively low (12% of the entire population), but was higher in urban than in rural areas. With the increased attention that managed care pays to adherence to guidelines, it is possible that the true rate of urban compliance is higher than we reported. The rates in rural areas would be little affected by this limitation. Patterns of care may also be different for younger people, irrespective of insurance coverage. Care may also have improved since 1994.

Also, Medicare’s data systems are primarily mechanisms to ensure accurate billing and payment; they were not designed as research tools. However, previous work by Weiner and colleagues¹² shows that the Medicare data were of generally good quality. Finally, our study relied entirely on process of care as a surrogate for medical care quality. Although there is general consensus that the process measures studied here are desirable in the care of patients with diabetes, we do not know whether patients who received these tests had better outcomes.

Conclusions

The results of our study demonstrate that the quality of care received by Medicare patients in Washington in 1994 was better in some important respects than that received in other parts of the country. Although there is still significant room for improvement, the fact that there is marked regional variation suggests that physicians can make meaningful improvements in the quality of care.^18,19 It would be useful to identify specific communities where quality of care indicators were suboptimal and design educational efforts for patients and care providers. Perhaps using Medicare data to provide physician scorecards would improve adherence.

Adherence to quality standards was not uniform across rural communities. Rural communities in counties adjacent to metropolitan areas had significantly lower quality-of-care measures than people living in nearby urban areas. Perhaps there are unmeasured socioeconomic or medical practice factors among these populations that explain this lower level of adherence to established standards, even after correcting for the confounding variables that we were able to measure. It would be worth embarking on a systematic exploration of the clinical, social, and organizational factors that led to this relatively substandard experience that has been noted for other defined populations.²⁰

The fact that the highest-quality care occurs in large remote rural communities may contain some lessons for the optimal organization of health services. These are communities that have moderate-sized hospitals, a balanced mix of generalists and specialists, and population sizes between 10,000 and 50,000 people. There may be advantages to living in areas such as these where patients are not exposed to the potentially deleterious effect of too few physicians or fragmentation of services amidst a surplus of specialists. Future studies are needed to determine if these findings can be generalized to the care of other patients and other conditions.

Acknowledgments

Our work was funded by a grant from the Federal Office of Rural Health Policy, the Robert Wood Johnson Foundation, and the Agency for Health Care Policy and Research.

Related Resources

• American Diabetes Association www.diabetes.org Definitive source of patient-centered information about diabetes and its treatment.
• Canadian Diabetes Association http://www.diabetes.ca/ Information on insulin, nutrition, research, complications of diabetes, juvenile diabetes and other disease-related issues. Resources for patients and physicians.
• National Institute of Diabetes and Digestive Diseases and Kidney Diseases http://www.niddk.nih.gov/ Health education programs and publications on diabetes for patients, Information on clinical trials and research funding opportunities for physicians, faculty and researchers.

It is difficult to provide high-quality care to elderly patients with diabetes, and this task may be even more problematic in rural areas.¹ There are fewer physicians in such areas, and chronic conditions may get short shrift from both physicians and patients.² The relative shortage of specialists in rural areas may make it more difficult for physicians and their patients to get some of the specialized services they may need.³ Knowledge about advances in diabetic care may diffuse more slowly to these areas, making it less likely that physicians and patients will be aware of or adhere to published guidelines.

Previous studies have shown that the rural elderly-particularly those living in the smallest and most remote areas-make fewer office visits to physicians.² These same patients are more likely to see family physicians-and less likely to visit specialists-than their urban counterparts.⁴ It is not known whether this is true specifically of patients with diabetes, and the impact of these patterns on adherence to generally accepted guidelines is unknown.

We examined rural-urban differences in the care of persons with diabetes to determine what kinds of locations promote high-quality care. It may be possible to improve diabetes care either through further training of generalists or by providing opportunities for formal consultation with relevant specialists within the communities where these patients live.

Methods

Our study was based on data from the Medicare program for Washington state in 1994. During that year 362,145 Medicare recipients 65 years and older used medical care, did not belong to a capitated plan, had continuous Medicare coverage, received all their medical care in Washington State, and were alive at the end of the year.

For the purposes of our study, a diabetic visit is defined as any visit to a physician in an ambulatory setting where that physician entered any of the following International Classification of Diseases–ninth revision codes as a diagnosis: 250.XX (diabetes), 362.01 and 262.02 (diabetic retinopathy), 357.2 (diabetic polyneuropathy), or 366.41 (diabetic cataract). Patients are considered to have diabetes if they have at least 2 physician encounters for 1 of these codes in an ambulatory setting on separate days.

Patient residence was determined by the residential ZIP Code, and all patients were assigned as being rural or urban based on their residence.⁵ Rural communities were considered to be large if they had hospitals with more than 100 beds. The identity of the physician was determined from the unique physician identification number (UPIN) assigned by Medicare. UPINs were present 99.1% of the time, and specialty could be determined for 99.0% of these physicians.

Quality of Care Measurements

We created a core quality index of items that most authoritative sources agree should be performed regularly in patients with diabetes^6-9 and that can be identified using the Medicare Part B claims file.^10-12 The core quality index included a glycated hemoglobin determination, cholesterol measurement, and an eye examination by either an ophthalmologist or an optometrist. A service was considered to have been performed if a claim for any of the above items-or for a multi-test procedure of which that item is a part-was submitted by any provider during the 1994 study year.

Analytic Approach

We used the ambulatory care group (ACG) case-mix classification system to control for patient comorbidities.^13,14 Confidence intervals were calculated for independent and control variables in the logistic regression. Chi-square tests were used to compare results across different geographic areas. Because of multiple comparisons, we only report differences significant at the .01 level.

Results

According to our criteria, a total of 30,589 patients (56.4% women) representing 8.4% of all Medicare patients had diabetes. These patients made 392,831 outpatient visits to physicians during 1994, for an average of 12.8 visits per person. A diagnosis of diabetes was recorded for 42.7% of all outpatient visits by patients with diabetes.

Urban patients made more ambulatory visits overall than their rural counterparts, although there was no significant difference in the number of visits for diabetes. Patients living in small remote rural communities made significantly fewer ambulatory visits than patients living in any other place. The overall illness severity mirrored the number of ambulatory visits: 55.1% of urban patients and those living in large remote areas had 4 or more major chronic conditions; 51.3% of the group living in the small remote rural areas had the same burden of disease (P <.01).

Geographic location had a profound effect on where patients received their care. Urban patients received virtually all their outpatient care in their local urban areas (97.9%). Patients living in large rural communities also received most of their outpatient care in their own community. When patients in these communities did travel for care, they usually went to an urban community.

The small rural communities were much less self-sufficient, with almost half of all outpatient visits occurring outside the local community. Patients from small towns adjacent to cities went to urban areas. Patients from the remote small communities were more likely to get care in large rural communities; a substantial number, however, went to urban areas.

Generalists provided most of the care for patients with diabetes Table 1. Family physicians and general internists accounted for 62.4% of all visits coded for diabetes. The smaller and more remote the area, the higher the proportion of visits to family physicians. Endocrinologists, who handle more than 11% of the outpatient diabetic visits of the urban elderly, were seen for only 3% of the diabetic visits of those living in small remote communities . Urban patients were much more likely to consult an endocrinologist than their rural counterparts; 16.3% of urban patients visited an endocrinologist at least once during the year, compared with 6.9% of rural patients.

Adherence to Guidelines

The majority of patients had their cholesterol and glycated hemoglobin measured and their eyes examined at least once during the study year Table 2, although only 27.5% of patients had all 3 determinations performed. Urban patients were significantly more likely to have their glycated hemoglobin and cholesterol levels measured than rural patients, although the differences were small. Most patients who had glycated hemoglobin measured had either 1 or 2 such tests during the study year, with 31.3% of patients receiving 2 glycated hemoglobin determinations during the year.

Patients living in large remote rural communities were significantly more likely to have received all 3 of the core diabetes quality measures than patients in any of the other areas. By contrast, patients living in large rural communities adjacent to metropolitan areas were much less likely to have a glycated hemoglobin determination or an eye examination. Small rural towns had essentially identical screening rates, independent of their proximity to an urban area.

The specialty of the physicians was not associated with differences in adherence to screening guidelines, with one exception. Patients who saw an endocrinologist at least once during the year were much more likely to have received a glycated hemoglobin determination. Of patients who saw an endocrinologist, 77.9% received this test versus 51.0% of the patients with diabetes who had not seen an endocrinologist. The proportion of eye examinations and cholesterol measurements were also higher for patients who consulted an endocrinologist, although the differences are not as large as for glycated hemoglobin tests.

We used logistic regression to test the independent effect of patient residence on the likelihood of receiving the recommended tests.*Table w1 Patient residence is associated with significant differences in the likelihood that a patient received a glycated hemoglobin test. Patients living in large rural communities adjacent to metropolitan areas were significantly less likely to have a glycated hemoglobin determination than patients living in all other locations, even after controlling for sociodemographic factors, illness severity, and physician specialty. By contrast, patients living in large remote areas were much more likely to have received the test. Patients living in small remote rural areas received the test at a rate similar to that of patients living in urban areas, all other factors being equal. The single variable with the greatest independent effect was whether the patient saw an endocrinologist during the year.

A similar pattern prevails when using the core diabetes quality index in a multiple linear regression (not tabled). Study variables explain 18.18% of the variance in the index value. All 4 of the rural residential variables were statistically significant; patients living in remote large rural areas had a greater likelihood of receiving the recommended tests after controlling for potential confounders, while patients living in other types of rural areas were less likely to receive the tests.

Discussion

The quality of outpatient care for elderly persons with diabetes leaves much to be desired.^10-12,15 On a national level, only 21% of patients received a glycated hemoglobin determination in 1994, perhaps the best single summary of diabetic control available to physicians.^10,11 In our study of Washington for the same year, a much higher proportion of patients received this test, suggesting the existence of major regional differences. Yet even in our study, almost half of patients with a diagnosis of diabetes did not receive a glycated hemoglobin determination even though Medicare reimburses separately for this test. Only 27.5% received all 3 of the tests recommended by authoritative national organizations during the study year.

The location of the patients’ community affects their likelihood of receiving the recommended screening tests. Patients living in large rural communities remote from cities were significantly more likely to receive the recommended services than their urban counterparts; patients living in other rural locations were less likely to receive these services.

What might explain these findings? One contributing factor is the relative unavailability of endocrinologists in many rural communities. Rural patients who saw an endocrinologist at least once during the year were almost twice as likely to have had a glycated hemoglobin determination, probably because ordering such a test is part of the routine when endocrinologists see a new patient with diabetes.¹⁶ Only 24.6% of the visits to an endocrinologist occurred within the rural area where the patient lived, since most endocrinologists practice in urban areas. It is likely that this access barrier explains the much lower rate at which rural patients see endocrinologists and contributes to the lower rate of appropriate testing.

But this is not the only factor. There are very few endocrinologists in the state of Washington (69 in our study), and most diabetic care is provided by primary care physicians.¹⁷ The highest rate of guideline adherence occurs in large remote rural communities—communities that have endocrinologists but where the rate at which patients visit these specialists is still less than half of that in urban communities. It may be that large rural towns represent the best of both worlds: vibrant, rapidly growing communities with an adequate supply of both generalist and specialist physicians that serve as regional referral centers for surrounding rural towns.

Limitations

These data are based on the elderly Medicare population in Washington who are not members of managed care organizations. Managed care penetration in 1994 was relatively low (12% of the entire population), but was higher in urban than in rural areas. With the increased attention that managed care pays to adherence to guidelines, it is possible that the true rate of urban compliance is higher than we reported. The rates in rural areas would be little affected by this limitation. Patterns of care may also be different for younger people, irrespective of insurance coverage. Care may also have improved since 1994.

Also, Medicare’s data systems are primarily mechanisms to ensure accurate billing and payment; they were not designed as research tools. However, previous work by Weiner and colleagues¹² shows that the Medicare data were of generally good quality. Finally, our study relied entirely on process of care as a surrogate for medical care quality. Although there is general consensus that the process measures studied here are desirable in the care of patients with diabetes, we do not know whether patients who received these tests had better outcomes.

Conclusions

The results of our study demonstrate that the quality of care received by Medicare patients in Washington in 1994 was better in some important respects than that received in other parts of the country. Although there is still significant room for improvement, the fact that there is marked regional variation suggests that physicians can make meaningful improvements in the quality of care.^18,19 It would be useful to identify specific communities where quality of care indicators were suboptimal and design educational efforts for patients and care providers. Perhaps using Medicare data to provide physician scorecards would improve adherence.

Adherence to quality standards was not uniform across rural communities. Rural communities in counties adjacent to metropolitan areas had significantly lower quality-of-care measures than people living in nearby urban areas. Perhaps there are unmeasured socioeconomic or medical practice factors among these populations that explain this lower level of adherence to established standards, even after correcting for the confounding variables that we were able to measure. It would be worth embarking on a systematic exploration of the clinical, social, and organizational factors that led to this relatively substandard experience that has been noted for other defined populations.²⁰

The fact that the highest-quality care occurs in large remote rural communities may contain some lessons for the optimal organization of health services. These are communities that have moderate-sized hospitals, a balanced mix of generalists and specialists, and population sizes between 10,000 and 50,000 people. There may be advantages to living in areas such as these where patients are not exposed to the potentially deleterious effect of too few physicians or fragmentation of services amidst a surplus of specialists. Future studies are needed to determine if these findings can be generalized to the care of other patients and other conditions.

Acknowledgments

Our work was funded by a grant from the Federal Office of Rural Health Policy, the Robert Wood Johnson Foundation, and the Agency for Health Care Policy and Research.

Related Resources

• American Diabetes Association www.diabetes.org Definitive source of patient-centered information about diabetes and its treatment.
• Canadian Diabetes Association http://www.diabetes.ca/ Information on insulin, nutrition, research, complications of diabetes, juvenile diabetes and other disease-related issues. Resources for patients and physicians.
• National Institute of Diabetes and Digestive Diseases and Kidney Diseases http://www.niddk.nih.gov/ Health education programs and publications on diabetes for patients, Information on clinical trials and research funding opportunities for physicians, faculty and researchers.

It is difficult to provide high-quality care to elderly patients with diabetes, and this task may be even more problematic in rural areas.¹ There are fewer physicians in such areas, and chronic conditions may get short shrift from both physicians and patients.² The relative shortage of specialists in rural areas may make it more difficult for physicians and their patients to get some of the specialized services they may need.³ Knowledge about advances in diabetic care may diffuse more slowly to these areas, making it less likely that physicians and patients will be aware of or adhere to published guidelines.

Previous studies have shown that the rural elderly-particularly those living in the smallest and most remote areas-make fewer office visits to physicians.² These same patients are more likely to see family physicians-and less likely to visit specialists-than their urban counterparts.⁴ It is not known whether this is true specifically of patients with diabetes, and the impact of these patterns on adherence to generally accepted guidelines is unknown.

We examined rural-urban differences in the care of persons with diabetes to determine what kinds of locations promote high-quality care. It may be possible to improve diabetes care either through further training of generalists or by providing opportunities for formal consultation with relevant specialists within the communities where these patients live.

Methods

Our study was based on data from the Medicare program for Washington state in 1994. During that year 362,145 Medicare recipients 65 years and older used medical care, did not belong to a capitated plan, had continuous Medicare coverage, received all their medical care in Washington State, and were alive at the end of the year.

For the purposes of our study, a diabetic visit is defined as any visit to a physician in an ambulatory setting where that physician entered any of the following International Classification of Diseases–ninth revision codes as a diagnosis: 250.XX (diabetes), 362.01 and 262.02 (diabetic retinopathy), 357.2 (diabetic polyneuropathy), or 366.41 (diabetic cataract). Patients are considered to have diabetes if they have at least 2 physician encounters for 1 of these codes in an ambulatory setting on separate days.

Patient residence was determined by the residential ZIP Code, and all patients were assigned as being rural or urban based on their residence.⁵ Rural communities were considered to be large if they had hospitals with more than 100 beds. The identity of the physician was determined from the unique physician identification number (UPIN) assigned by Medicare. UPINs were present 99.1% of the time, and specialty could be determined for 99.0% of these physicians.

Quality of Care Measurements

We created a core quality index of items that most authoritative sources agree should be performed regularly in patients with diabetes^6-9 and that can be identified using the Medicare Part B claims file.^10-12 The core quality index included a glycated hemoglobin determination, cholesterol measurement, and an eye examination by either an ophthalmologist or an optometrist. A service was considered to have been performed if a claim for any of the above items-or for a multi-test procedure of which that item is a part-was submitted by any provider during the 1994 study year.

Analytic Approach

We used the ambulatory care group (ACG) case-mix classification system to control for patient comorbidities.^13,14 Confidence intervals were calculated for independent and control variables in the logistic regression. Chi-square tests were used to compare results across different geographic areas. Because of multiple comparisons, we only report differences significant at the .01 level.

Results

According to our criteria, a total of 30,589 patients (56.4% women) representing 8.4% of all Medicare patients had diabetes. These patients made 392,831 outpatient visits to physicians during 1994, for an average of 12.8 visits per person. A diagnosis of diabetes was recorded for 42.7% of all outpatient visits by patients with diabetes.

Urban patients made more ambulatory visits overall than their rural counterparts, although there was no significant difference in the number of visits for diabetes. Patients living in small remote rural communities made significantly fewer ambulatory visits than patients living in any other place. The overall illness severity mirrored the number of ambulatory visits: 55.1% of urban patients and those living in large remote areas had 4 or more major chronic conditions; 51.3% of the group living in the small remote rural areas had the same burden of disease (P <.01).

Geographic location had a profound effect on where patients received their care. Urban patients received virtually all their outpatient care in their local urban areas (97.9%). Patients living in large rural communities also received most of their outpatient care in their own community. When patients in these communities did travel for care, they usually went to an urban community.

The small rural communities were much less self-sufficient, with almost half of all outpatient visits occurring outside the local community. Patients from small towns adjacent to cities went to urban areas. Patients from the remote small communities were more likely to get care in large rural communities; a substantial number, however, went to urban areas.

Generalists provided most of the care for patients with diabetes Table 1. Family physicians and general internists accounted for 62.4% of all visits coded for diabetes. The smaller and more remote the area, the higher the proportion of visits to family physicians. Endocrinologists, who handle more than 11% of the outpatient diabetic visits of the urban elderly, were seen for only 3% of the diabetic visits of those living in small remote communities . Urban patients were much more likely to consult an endocrinologist than their rural counterparts; 16.3% of urban patients visited an endocrinologist at least once during the year, compared with 6.9% of rural patients.

Adherence to Guidelines

The majority of patients had their cholesterol and glycated hemoglobin measured and their eyes examined at least once during the study year Table 2, although only 27.5% of patients had all 3 determinations performed. Urban patients were significantly more likely to have their glycated hemoglobin and cholesterol levels measured than rural patients, although the differences were small. Most patients who had glycated hemoglobin measured had either 1 or 2 such tests during the study year, with 31.3% of patients receiving 2 glycated hemoglobin determinations during the year.

Patients living in large remote rural communities were significantly more likely to have received all 3 of the core diabetes quality measures than patients in any of the other areas. By contrast, patients living in large rural communities adjacent to metropolitan areas were much less likely to have a glycated hemoglobin determination or an eye examination. Small rural towns had essentially identical screening rates, independent of their proximity to an urban area.

The specialty of the physicians was not associated with differences in adherence to screening guidelines, with one exception. Patients who saw an endocrinologist at least once during the year were much more likely to have received a glycated hemoglobin determination. Of patients who saw an endocrinologist, 77.9% received this test versus 51.0% of the patients with diabetes who had not seen an endocrinologist. The proportion of eye examinations and cholesterol measurements were also higher for patients who consulted an endocrinologist, although the differences are not as large as for glycated hemoglobin tests.

We used logistic regression to test the independent effect of patient residence on the likelihood of receiving the recommended tests.*Table w1 Patient residence is associated with significant differences in the likelihood that a patient received a glycated hemoglobin test. Patients living in large rural communities adjacent to metropolitan areas were significantly less likely to have a glycated hemoglobin determination than patients living in all other locations, even after controlling for sociodemographic factors, illness severity, and physician specialty. By contrast, patients living in large remote areas were much more likely to have received the test. Patients living in small remote rural areas received the test at a rate similar to that of patients living in urban areas, all other factors being equal. The single variable with the greatest independent effect was whether the patient saw an endocrinologist during the year.

A similar pattern prevails when using the core diabetes quality index in a multiple linear regression (not tabled). Study variables explain 18.18% of the variance in the index value. All 4 of the rural residential variables were statistically significant; patients living in remote large rural areas had a greater likelihood of receiving the recommended tests after controlling for potential confounders, while patients living in other types of rural areas were less likely to receive the tests.

Discussion

The quality of outpatient care for elderly persons with diabetes leaves much to be desired.^10-12,15 On a national level, only 21% of patients received a glycated hemoglobin determination in 1994, perhaps the best single summary of diabetic control available to physicians.^10,11 In our study of Washington for the same year, a much higher proportion of patients received this test, suggesting the existence of major regional differences. Yet even in our study, almost half of patients with a diagnosis of diabetes did not receive a glycated hemoglobin determination even though Medicare reimburses separately for this test. Only 27.5% received all 3 of the tests recommended by authoritative national organizations during the study year.

The location of the patients’ community affects their likelihood of receiving the recommended screening tests. Patients living in large rural communities remote from cities were significantly more likely to receive the recommended services than their urban counterparts; patients living in other rural locations were less likely to receive these services.

What might explain these findings? One contributing factor is the relative unavailability of endocrinologists in many rural communities. Rural patients who saw an endocrinologist at least once during the year were almost twice as likely to have had a glycated hemoglobin determination, probably because ordering such a test is part of the routine when endocrinologists see a new patient with diabetes.¹⁶ Only 24.6% of the visits to an endocrinologist occurred within the rural area where the patient lived, since most endocrinologists practice in urban areas. It is likely that this access barrier explains the much lower rate at which rural patients see endocrinologists and contributes to the lower rate of appropriate testing.

But this is not the only factor. There are very few endocrinologists in the state of Washington (69 in our study), and most diabetic care is provided by primary care physicians.¹⁷ The highest rate of guideline adherence occurs in large remote rural communities—communities that have endocrinologists but where the rate at which patients visit these specialists is still less than half of that in urban communities. It may be that large rural towns represent the best of both worlds: vibrant, rapidly growing communities with an adequate supply of both generalist and specialist physicians that serve as regional referral centers for surrounding rural towns.

Limitations

These data are based on the elderly Medicare population in Washington who are not members of managed care organizations. Managed care penetration in 1994 was relatively low (12% of the entire population), but was higher in urban than in rural areas. With the increased attention that managed care pays to adherence to guidelines, it is possible that the true rate of urban compliance is higher than we reported. The rates in rural areas would be little affected by this limitation. Patterns of care may also be different for younger people, irrespective of insurance coverage. Care may also have improved since 1994.

Also, Medicare’s data systems are primarily mechanisms to ensure accurate billing and payment; they were not designed as research tools. However, previous work by Weiner and colleagues¹² shows that the Medicare data were of generally good quality. Finally, our study relied entirely on process of care as a surrogate for medical care quality. Although there is general consensus that the process measures studied here are desirable in the care of patients with diabetes, we do not know whether patients who received these tests had better outcomes.

Conclusions

The results of our study demonstrate that the quality of care received by Medicare patients in Washington in 1994 was better in some important respects than that received in other parts of the country. Although there is still significant room for improvement, the fact that there is marked regional variation suggests that physicians can make meaningful improvements in the quality of care.^18,19 It would be useful to identify specific communities where quality of care indicators were suboptimal and design educational efforts for patients and care providers. Perhaps using Medicare data to provide physician scorecards would improve adherence.

Adherence to quality standards was not uniform across rural communities. Rural communities in counties adjacent to metropolitan areas had significantly lower quality-of-care measures than people living in nearby urban areas. Perhaps there are unmeasured socioeconomic or medical practice factors among these populations that explain this lower level of adherence to established standards, even after correcting for the confounding variables that we were able to measure. It would be worth embarking on a systematic exploration of the clinical, social, and organizational factors that led to this relatively substandard experience that has been noted for other defined populations.²⁰

The fact that the highest-quality care occurs in large remote rural communities may contain some lessons for the optimal organization of health services. These are communities that have moderate-sized hospitals, a balanced mix of generalists and specialists, and population sizes between 10,000 and 50,000 people. There may be advantages to living in areas such as these where patients are not exposed to the potentially deleterious effect of too few physicians or fragmentation of services amidst a surplus of specialists. Future studies are needed to determine if these findings can be generalized to the care of other patients and other conditions.

Acknowledgments

Our work was funded by a grant from the Federal Office of Rural Health Policy, the Robert Wood Johnson Foundation, and the Agency for Health Care Policy and Research.

Related Resources

• American Diabetes Association www.diabetes.org Definitive source of patient-centered information about diabetes and its treatment.
• Canadian Diabetes Association http://www.diabetes.ca/ Information on insulin, nutrition, research, complications of diabetes, juvenile diabetes and other disease-related issues. Resources for patients and physicians.
• National Institute of Diabetes and Digestive Diseases and Kidney Diseases http://www.niddk.nih.gov/ Health education programs and publications on diabetes for patients, Information on clinical trials and research funding opportunities for physicians, faculty and researchers.

Nephropathy is one of the most common long-term side effects of diabetes mellitus and accounts for the largest percentage of patients requiring chronic renal dialysis in the United States and Europe.^1,2 The high prevalence of type 2 diabetes among adults in the United States and the high rate of nephropathy in these individuals pose a great economic burden to the health care system.

Several studies have noted that angiotensin-converting enzyme inhibitors (ACEIs) can delay the progression of renal impairment in patients with type 2 diabetes.^3-7 Patients with diabetic nephropathy generally progress from a stage of normal renal function to microalbuminuria, gross proteinuria, and then renal dysfunction.¹ ACEIs appear to delay or prevent the progression from microalbuminuria to proteinuria. Although there are no controlled trials that show microalbuminuria screening as effective at reducing proteinuria, expert panels of the American Diabetes Association⁸ and National Kidney Foundation⁹ have recommended that patients with type 2 diabetes receive annual screening for microalbuminuria, and if it is detected on 2 of 3 occasions, these patients should be placed on an ACEI or an angiotensin receptor blocker (ARB) for renal protection.

Initial evaluation of data from primary care practices, however, reveals that screening for microalbuminuria is not optimal.^10,11 One reason microalbuminuria screening may happen less often than expected could be that many patients with diabetes mellitus are already being treated with an ACEI or ARB for hypertension, congestive heart failure, or other reasons. Some physicians also might employ an ACEI or ARB prophylactically, starting treatment before recognizing microalbuminuria. Given that these patients are already being treated with an ACEI or ARB, clinicians may not recognize any usefulness in performing a microalbuminuria test.

The purpose of our study was to examine what patient factors are associated with screening for microalbuminuria in patients with type 2 diabetes mellitus. Specifically, we examined how often patients who were not screened were already being treated with an ACEI or ARB. Also, we hoped to characterize the populations being screened more fully to determine if certain patient and disease characteristics were associated with the likelihood of a screening test being performed. A better awareness of these characteristics will help in targeting specific patient groups and changing physician behavior.

Methods

Sample

Our sample was drawn from the primary care practice in the department of family medicine at the Medical University of South Carolina (MUSC) in 1998 and 1999. The department provided care for approximately 18,000 patients who made 42,000 and 48,000 patient visits in 1998 and 1999, respectively, at 2 clinical sites. These 2 sites serve a diverse population of patients in downtown Charleston and a nearby suburban area, which in 1998 had a payer mix distribution that was 26% Medicaid, 27% Medicare, 37% commercially insured, and 10% self-pay.

We identified patients with diabetes at the 2 clinical sites from a search of the problem list in an electronic medical record database that has been used in the department of family medicine since 1992 Table 1. All patients aged between 18 years and 65 years in 2000 and who had an appointment scheduled in 1998 or 1999 were included. The charts that were initially selected for review had diabetes mellitus listed as a problem; after the chart review, we excluded 18 patients from the study because they were using insulin or had not been seen in the practices since 1995, even though they had scheduled an appointment in 1998 or 1999. This left a final sample size of 278.

Data Collection and Variables

Two medical students performed the chart reviews and recorded the following variables when available: age, weight, sex, serum creatinine level, hemoglobin A_1C (Hb A_1C) level, proteinuria on urinalysis testing, blood pressure, total serum cholesterol, and whether a microalbuminuria test was recommended and, if performed, the results. Race was not included because the patient charts do not consistently note the patient’s race. Also, because care is often shared between attending and resident physicians, we did not include the physician training level as a variable in our analysis.

To determine whether patients were on ACEI or ARB therapy, we searched the electronic medical record database for all medications in the previous 5 years. The medical record used during this period required all prescriptions to be entered before a printed version could be generated, so we could determine if a drug had been used in the past. Although this system overlooked prescriptions that might be called in to a pharmacy and not documented in the record, it captured every prescription written by a physician in the practice. When an ACEI or ARB was used, we examined whether the medication had been started before screening was indicated or after a microalbuminuria test was performed.

We searched the laboratory section of the electronic medical record and also the hospital patient database to determine if the hospital laboratory had performed the test. Searching the hospital database would indicate if the test was performed by any other clinician (eg, an endocrinologist) or in another setting (eg, inpatient) in the university medical center. Whether a microalbuminuria test was recommended was recorded, with the returned value (if available) and the date the test was recommended. We considered values greater than 20 mg per L positive for microalbuminuria. Protein-uria tests were considered positive if they returned a 1+ protein or greater result. We also recorded whether the subject was on an ACEI or ARB therapy, and if so at what date it had been prescribed.

To minimize inter-rater variability, the 2 medical students each reviewed a pilot sample of the same 20 charts. Data were compared and differences between the auditors were reviewed to standardize definitions of data elements. After standardization, sets of 10 different charts were selected, and the process was repeated until the data from 40 consecutive charts were recorded identically by both students.

Analysis

When comparing mean values, we performed a Student t test to determine statistical significance. A chi-square test was done to determine statistical significance when comparing proportions. A P value of <.05 was determined to be statistically significant.

Results

Of the 278 eligible patients, 44 (16%) had a urinalysis with 1+ or greater protein result at baseline; 18 (41%) of these were already taking an ACEI or ARB drug. In patients without previous evidence of proteinuria, 51 (18%) patients were using ACEI or ARB therapy. This left 183 patients (66%) who had no evidence of renal disease and who were not using ACEI or ARB therapy and therefore were the prime candidates for microalbuminuria screening Figure 1.

When we examined the demographics and clinical variables of these 3 groups, we found that patients with proteinuria or who were already using drug treatment were older and had higher systolic and diastolic blood pressures than those who were not. Unexpectedly, we also found that patients with existing proteinuria had lower Hb A_1C levels than patients in the other 2 categories.

Of these prime candidates for screening, only 31 (17%) received at least 1 microalbuminuria test between 1995 and 1999. The rate of screening in this group was no different from those who were taking an ACEI or ARB drug (16%, P=.83) or already had gross proteinuria (18%, P=.84).

When we examined the patients who were most likely to benefit from screening and looked at demographic or clinical factors that might influence whether a screening test was performed, we found that patients who received microalbuminuria testing were very similar to those who did not. The only difference we found was that patients who received screening had lower systolic blood pressures than those who were not screened. Weight, age, Hb A_1C levels, and cholesterol levels were not predictors of being screened for microalbuminuria Table 2.

Because of the low rates of microalbuminuria screening for patients who were eligible and the relatively frequent use of screening in patients who already had evidence of gross proteinuria, we were interested in what clinicians did when a microalbuminuria test result was positive. In the group without evidence of proteinuria and not using ACEI or ARB therapy, 10 of the 31 patients who received screening for microalbuminuria tested positive. However, only 4 (40%) were placed on ACE inhibitor or ARB therapy.

Discussion

Our data suggest that several problems exist in the use and interpretation of microalbuminuria testing in the primary care setting. First, microalbuminuria testing is being performed on only 1 of 5 adult patients with type 2 diabetes. Second, in this practice, testing is not targeted to the patients who are most likely to benefit from the results. Rather, the tests seemed to be used indiscriminately. Finally, even when patients are screened and found to have microalbuminuria, only a small percentage were started on appropriate therapy. At least in this patient population, it appears that ACEI or ARB therapy is reserved for patients with higher blood pressures rather than used for renal protection.

The observation that patients with existing proteinuria or who were on ACEI or ARB therapy were screened just as often as those who were prime candidates for screening contradicts our initial hypothesis. We had assumed that clinicians would not screen patients who were on ACEI or ARB therapy, reducing the overall screening rate. Apparently, this is not the case. At least in this practice, a low screening rate is not due to selective screening.

The lack of optimal use of microalbuminuria testing and the failure to respond appropriately to positive test results suggests that current recommendations have not been embraced by physicians. Also, the complexity of carrying out these recommendations may make it difficult to integrate this screening into routine practice. If the current evidence on ACEI and ARB therapy for the prevention of renal dysfunction is to be translated into practice, either greater emphasis needs to be placed on microalbuminuria screening or more efficient ways to provide renal protection for patients with diabetes should be considered. Other studies have found that between 17% and 30% of patients with type 2 diabetes have microalbuminuria.^1,12,13 Although primary care physicians report that they provide microalbuminuria screening to a large percentage of their patients with diabetes, in fact only a small percentage of those who should be screened actually are screened.¹⁰ Suboptimal screening rates for important conditions seen in primary care are not unique for microalbuminuria. Other studies have documented comparable low screening rates for a wide variety of cancers.¹⁴ Since physicians do not screen reliably for potentially fatal diseases with screening modalities that have been available for decades, it is unlikely that their behavior is likely to improve when asked to screen for microalbuminuria.

Also, recent evidence that ACEI therapy may improve endothelial function in patients with type 2 diabetes suggests that even patients without microalbuminuria may benefit from routine ACEI therapy.¹⁵ Other studies suggest that routine use of ACEIs in middle-aged patients with type 2 diabetes may provide substantial benefits at only modest costs compared with a screening strategy.¹⁶ These data suggest that a more effective strategy would be to advise that all patients with type 2 diabetes start ACEI or ARB therapy along with their medications for diabetes. This strategy would obviate the need for microalbuminuria screening, while assuring that patients receive any additional benefits of ACEI or ARB therapy unrelated to renal protection. However, using this strategy, patients who may not have proteinuria will have to take the medication for a prolonged period, pay for it, and run the risks for any complications associated with using the drug.

Limitations

Our study has several limitations. Only 1 practice was examined, and it was part of a residency training practice. This means that less-experienced clinicians were providing care that could reduce the overall rate of screening. However, the rate of screening observed in this study was very similar to rates found in the practices of clinicians with more experience,^11,12 suggesting that the lack of experience of resident physicians may be balanced by the oversight provided by faculty preceptors.

Another limitation is that it was not possible to account for microalbuminuria screening completed outside the MUSC medical center. Patients who split their care among several providers could have had testing performed in other health care facilities. However, since more than 95% of the referrals from the MUSC Family Medicine Center stay within the university health care system, it is doubtful that many patients would have received testing outside the search capabilities of the hospital laboratory database.

Finally, the study was limited in its power to detect small differences between the groups. We originally conceived our project as an exploratory study to determine how many patients were already taking ACEIs and the potential effect of this on overall screening rates for microalbuminuria. Without any reference for the percentage of patients who were taking ACEIs, we could not perform an ad hoc power analysis. However, a post hoc analysis shows that for a sample in which the groups are matched in a 1-to-3 ratio (approximating the proportion of the 51 patients in our sample taking ACEIs and the 183 not taking these drugs) and given the study sample size, our study had a power of 80% to detect a difference in screening rates between 20% in the baseline group and 5% in the ACE or ARB groups. The actual difference seen in our study was much smaller, which increases the possibility of a type II error.

Conclusions

Because physician use of microalbuminuria screening does not follow established guidelines, consideration should be given to other strategies to prevent nephropathy in persons with type 2 diabetes. One proposed strategy would advise all patients with type 2 diabetes to start ACEI or ARB therapy along with their medications for diabetes. This strategy would obviate the need for microalbuminuria screening, while ensuring that patients receive any additional benefits of ACEI or ARB therapy unrelated to renal protection. It is unknown, however, whether patients would accept universal treatment rather than periodic screening. This is an important question that should be addressed before any population-based strategies are adopted.

Nephropathy is one of the most common long-term side effects of diabetes mellitus and accounts for the largest percentage of patients requiring chronic renal dialysis in the United States and Europe.^1,2 The high prevalence of type 2 diabetes among adults in the United States and the high rate of nephropathy in these individuals pose a great economic burden to the health care system.

Several studies have noted that angiotensin-converting enzyme inhibitors (ACEIs) can delay the progression of renal impairment in patients with type 2 diabetes.^3-7 Patients with diabetic nephropathy generally progress from a stage of normal renal function to microalbuminuria, gross proteinuria, and then renal dysfunction.¹ ACEIs appear to delay or prevent the progression from microalbuminuria to proteinuria. Although there are no controlled trials that show microalbuminuria screening as effective at reducing proteinuria, expert panels of the American Diabetes Association⁸ and National Kidney Foundation⁹ have recommended that patients with type 2 diabetes receive annual screening for microalbuminuria, and if it is detected on 2 of 3 occasions, these patients should be placed on an ACEI or an angiotensin receptor blocker (ARB) for renal protection.

Initial evaluation of data from primary care practices, however, reveals that screening for microalbuminuria is not optimal.^10,11 One reason microalbuminuria screening may happen less often than expected could be that many patients with diabetes mellitus are already being treated with an ACEI or ARB for hypertension, congestive heart failure, or other reasons. Some physicians also might employ an ACEI or ARB prophylactically, starting treatment before recognizing microalbuminuria. Given that these patients are already being treated with an ACEI or ARB, clinicians may not recognize any usefulness in performing a microalbuminuria test.

The purpose of our study was to examine what patient factors are associated with screening for microalbuminuria in patients with type 2 diabetes mellitus. Specifically, we examined how often patients who were not screened were already being treated with an ACEI or ARB. Also, we hoped to characterize the populations being screened more fully to determine if certain patient and disease characteristics were associated with the likelihood of a screening test being performed. A better awareness of these characteristics will help in targeting specific patient groups and changing physician behavior.

Methods

Sample

Our sample was drawn from the primary care practice in the department of family medicine at the Medical University of South Carolina (MUSC) in 1998 and 1999. The department provided care for approximately 18,000 patients who made 42,000 and 48,000 patient visits in 1998 and 1999, respectively, at 2 clinical sites. These 2 sites serve a diverse population of patients in downtown Charleston and a nearby suburban area, which in 1998 had a payer mix distribution that was 26% Medicaid, 27% Medicare, 37% commercially insured, and 10% self-pay.

We identified patients with diabetes at the 2 clinical sites from a search of the problem list in an electronic medical record database that has been used in the department of family medicine since 1992 Table 1. All patients aged between 18 years and 65 years in 2000 and who had an appointment scheduled in 1998 or 1999 were included. The charts that were initially selected for review had diabetes mellitus listed as a problem; after the chart review, we excluded 18 patients from the study because they were using insulin or had not been seen in the practices since 1995, even though they had scheduled an appointment in 1998 or 1999. This left a final sample size of 278.

Data Collection and Variables

Two medical students performed the chart reviews and recorded the following variables when available: age, weight, sex, serum creatinine level, hemoglobin A_1C (Hb A_1C) level, proteinuria on urinalysis testing, blood pressure, total serum cholesterol, and whether a microalbuminuria test was recommended and, if performed, the results. Race was not included because the patient charts do not consistently note the patient’s race. Also, because care is often shared between attending and resident physicians, we did not include the physician training level as a variable in our analysis.

To determine whether patients were on ACEI or ARB therapy, we searched the electronic medical record database for all medications in the previous 5 years. The medical record used during this period required all prescriptions to be entered before a printed version could be generated, so we could determine if a drug had been used in the past. Although this system overlooked prescriptions that might be called in to a pharmacy and not documented in the record, it captured every prescription written by a physician in the practice. When an ACEI or ARB was used, we examined whether the medication had been started before screening was indicated or after a microalbuminuria test was performed.

We searched the laboratory section of the electronic medical record and also the hospital patient database to determine if the hospital laboratory had performed the test. Searching the hospital database would indicate if the test was performed by any other clinician (eg, an endocrinologist) or in another setting (eg, inpatient) in the university medical center. Whether a microalbuminuria test was recommended was recorded, with the returned value (if available) and the date the test was recommended. We considered values greater than 20 mg per L positive for microalbuminuria. Protein-uria tests were considered positive if they returned a 1+ protein or greater result. We also recorded whether the subject was on an ACEI or ARB therapy, and if so at what date it had been prescribed.

To minimize inter-rater variability, the 2 medical students each reviewed a pilot sample of the same 20 charts. Data were compared and differences between the auditors were reviewed to standardize definitions of data elements. After standardization, sets of 10 different charts were selected, and the process was repeated until the data from 40 consecutive charts were recorded identically by both students.

Analysis

When comparing mean values, we performed a Student t test to determine statistical significance. A chi-square test was done to determine statistical significance when comparing proportions. A P value of <.05 was determined to be statistically significant.

Results

Of the 278 eligible patients, 44 (16%) had a urinalysis with 1+ or greater protein result at baseline; 18 (41%) of these were already taking an ACEI or ARB drug. In patients without previous evidence of proteinuria, 51 (18%) patients were using ACEI or ARB therapy. This left 183 patients (66%) who had no evidence of renal disease and who were not using ACEI or ARB therapy and therefore were the prime candidates for microalbuminuria screening Figure 1.

When we examined the demographics and clinical variables of these 3 groups, we found that patients with proteinuria or who were already using drug treatment were older and had higher systolic and diastolic blood pressures than those who were not. Unexpectedly, we also found that patients with existing proteinuria had lower Hb A_1C levels than patients in the other 2 categories.

Of these prime candidates for screening, only 31 (17%) received at least 1 microalbuminuria test between 1995 and 1999. The rate of screening in this group was no different from those who were taking an ACEI or ARB drug (16%, P=.83) or already had gross proteinuria (18%, P=.84).

When we examined the patients who were most likely to benefit from screening and looked at demographic or clinical factors that might influence whether a screening test was performed, we found that patients who received microalbuminuria testing were very similar to those who did not. The only difference we found was that patients who received screening had lower systolic blood pressures than those who were not screened. Weight, age, Hb A_1C levels, and cholesterol levels were not predictors of being screened for microalbuminuria Table 2.

Because of the low rates of microalbuminuria screening for patients who were eligible and the relatively frequent use of screening in patients who already had evidence of gross proteinuria, we were interested in what clinicians did when a microalbuminuria test result was positive. In the group without evidence of proteinuria and not using ACEI or ARB therapy, 10 of the 31 patients who received screening for microalbuminuria tested positive. However, only 4 (40%) were placed on ACE inhibitor or ARB therapy.

Discussion

Our data suggest that several problems exist in the use and interpretation of microalbuminuria testing in the primary care setting. First, microalbuminuria testing is being performed on only 1 of 5 adult patients with type 2 diabetes. Second, in this practice, testing is not targeted to the patients who are most likely to benefit from the results. Rather, the tests seemed to be used indiscriminately. Finally, even when patients are screened and found to have microalbuminuria, only a small percentage were started on appropriate therapy. At least in this patient population, it appears that ACEI or ARB therapy is reserved for patients with higher blood pressures rather than used for renal protection.

The observation that patients with existing proteinuria or who were on ACEI or ARB therapy were screened just as often as those who were prime candidates for screening contradicts our initial hypothesis. We had assumed that clinicians would not screen patients who were on ACEI or ARB therapy, reducing the overall screening rate. Apparently, this is not the case. At least in this practice, a low screening rate is not due to selective screening.

The lack of optimal use of microalbuminuria testing and the failure to respond appropriately to positive test results suggests that current recommendations have not been embraced by physicians. Also, the complexity of carrying out these recommendations may make it difficult to integrate this screening into routine practice. If the current evidence on ACEI and ARB therapy for the prevention of renal dysfunction is to be translated into practice, either greater emphasis needs to be placed on microalbuminuria screening or more efficient ways to provide renal protection for patients with diabetes should be considered. Other studies have found that between 17% and 30% of patients with type 2 diabetes have microalbuminuria.^1,12,13 Although primary care physicians report that they provide microalbuminuria screening to a large percentage of their patients with diabetes, in fact only a small percentage of those who should be screened actually are screened.¹⁰ Suboptimal screening rates for important conditions seen in primary care are not unique for microalbuminuria. Other studies have documented comparable low screening rates for a wide variety of cancers.¹⁴ Since physicians do not screen reliably for potentially fatal diseases with screening modalities that have been available for decades, it is unlikely that their behavior is likely to improve when asked to screen for microalbuminuria.

Also, recent evidence that ACEI therapy may improve endothelial function in patients with type 2 diabetes suggests that even patients without microalbuminuria may benefit from routine ACEI therapy.¹⁵ Other studies suggest that routine use of ACEIs in middle-aged patients with type 2 diabetes may provide substantial benefits at only modest costs compared with a screening strategy.¹⁶ These data suggest that a more effective strategy would be to advise that all patients with type 2 diabetes start ACEI or ARB therapy along with their medications for diabetes. This strategy would obviate the need for microalbuminuria screening, while assuring that patients receive any additional benefits of ACEI or ARB therapy unrelated to renal protection. However, using this strategy, patients who may not have proteinuria will have to take the medication for a prolonged period, pay for it, and run the risks for any complications associated with using the drug.

Limitations

Our study has several limitations. Only 1 practice was examined, and it was part of a residency training practice. This means that less-experienced clinicians were providing care that could reduce the overall rate of screening. However, the rate of screening observed in this study was very similar to rates found in the practices of clinicians with more experience,^11,12 suggesting that the lack of experience of resident physicians may be balanced by the oversight provided by faculty preceptors.

Another limitation is that it was not possible to account for microalbuminuria screening completed outside the MUSC medical center. Patients who split their care among several providers could have had testing performed in other health care facilities. However, since more than 95% of the referrals from the MUSC Family Medicine Center stay within the university health care system, it is doubtful that many patients would have received testing outside the search capabilities of the hospital laboratory database.

Finally, the study was limited in its power to detect small differences between the groups. We originally conceived our project as an exploratory study to determine how many patients were already taking ACEIs and the potential effect of this on overall screening rates for microalbuminuria. Without any reference for the percentage of patients who were taking ACEIs, we could not perform an ad hoc power analysis. However, a post hoc analysis shows that for a sample in which the groups are matched in a 1-to-3 ratio (approximating the proportion of the 51 patients in our sample taking ACEIs and the 183 not taking these drugs) and given the study sample size, our study had a power of 80% to detect a difference in screening rates between 20% in the baseline group and 5% in the ACE or ARB groups. The actual difference seen in our study was much smaller, which increases the possibility of a type II error.

Conclusions

Because physician use of microalbuminuria screening does not follow established guidelines, consideration should be given to other strategies to prevent nephropathy in persons with type 2 diabetes. One proposed strategy would advise all patients with type 2 diabetes to start ACEI or ARB therapy along with their medications for diabetes. This strategy would obviate the need for microalbuminuria screening, while ensuring that patients receive any additional benefits of ACEI or ARB therapy unrelated to renal protection. It is unknown, however, whether patients would accept universal treatment rather than periodic screening. This is an important question that should be addressed before any population-based strategies are adopted.

Nephropathy is one of the most common long-term side effects of diabetes mellitus and accounts for the largest percentage of patients requiring chronic renal dialysis in the United States and Europe.^1,2 The high prevalence of type 2 diabetes among adults in the United States and the high rate of nephropathy in these individuals pose a great economic burden to the health care system.

Several studies have noted that angiotensin-converting enzyme inhibitors (ACEIs) can delay the progression of renal impairment in patients with type 2 diabetes.^3-7 Patients with diabetic nephropathy generally progress from a stage of normal renal function to microalbuminuria, gross proteinuria, and then renal dysfunction.¹ ACEIs appear to delay or prevent the progression from microalbuminuria to proteinuria. Although there are no controlled trials that show microalbuminuria screening as effective at reducing proteinuria, expert panels of the American Diabetes Association⁸ and National Kidney Foundation⁹ have recommended that patients with type 2 diabetes receive annual screening for microalbuminuria, and if it is detected on 2 of 3 occasions, these patients should be placed on an ACEI or an angiotensin receptor blocker (ARB) for renal protection.

Initial evaluation of data from primary care practices, however, reveals that screening for microalbuminuria is not optimal.^10,11 One reason microalbuminuria screening may happen less often than expected could be that many patients with diabetes mellitus are already being treated with an ACEI or ARB for hypertension, congestive heart failure, or other reasons. Some physicians also might employ an ACEI or ARB prophylactically, starting treatment before recognizing microalbuminuria. Given that these patients are already being treated with an ACEI or ARB, clinicians may not recognize any usefulness in performing a microalbuminuria test.

The purpose of our study was to examine what patient factors are associated with screening for microalbuminuria in patients with type 2 diabetes mellitus. Specifically, we examined how often patients who were not screened were already being treated with an ACEI or ARB. Also, we hoped to characterize the populations being screened more fully to determine if certain patient and disease characteristics were associated with the likelihood of a screening test being performed. A better awareness of these characteristics will help in targeting specific patient groups and changing physician behavior.

Methods

Sample

Our sample was drawn from the primary care practice in the department of family medicine at the Medical University of South Carolina (MUSC) in 1998 and 1999. The department provided care for approximately 18,000 patients who made 42,000 and 48,000 patient visits in 1998 and 1999, respectively, at 2 clinical sites. These 2 sites serve a diverse population of patients in downtown Charleston and a nearby suburban area, which in 1998 had a payer mix distribution that was 26% Medicaid, 27% Medicare, 37% commercially insured, and 10% self-pay.

We identified patients with diabetes at the 2 clinical sites from a search of the problem list in an electronic medical record database that has been used in the department of family medicine since 1992 Table 1. All patients aged between 18 years and 65 years in 2000 and who had an appointment scheduled in 1998 or 1999 were included. The charts that were initially selected for review had diabetes mellitus listed as a problem; after the chart review, we excluded 18 patients from the study because they were using insulin or had not been seen in the practices since 1995, even though they had scheduled an appointment in 1998 or 1999. This left a final sample size of 278.

Data Collection and Variables

Two medical students performed the chart reviews and recorded the following variables when available: age, weight, sex, serum creatinine level, hemoglobin A_1C (Hb A_1C) level, proteinuria on urinalysis testing, blood pressure, total serum cholesterol, and whether a microalbuminuria test was recommended and, if performed, the results. Race was not included because the patient charts do not consistently note the patient’s race. Also, because care is often shared between attending and resident physicians, we did not include the physician training level as a variable in our analysis.

To determine whether patients were on ACEI or ARB therapy, we searched the electronic medical record database for all medications in the previous 5 years. The medical record used during this period required all prescriptions to be entered before a printed version could be generated, so we could determine if a drug had been used in the past. Although this system overlooked prescriptions that might be called in to a pharmacy and not documented in the record, it captured every prescription written by a physician in the practice. When an ACEI or ARB was used, we examined whether the medication had been started before screening was indicated or after a microalbuminuria test was performed.

We searched the laboratory section of the electronic medical record and also the hospital patient database to determine if the hospital laboratory had performed the test. Searching the hospital database would indicate if the test was performed by any other clinician (eg, an endocrinologist) or in another setting (eg, inpatient) in the university medical center. Whether a microalbuminuria test was recommended was recorded, with the returned value (if available) and the date the test was recommended. We considered values greater than 20 mg per L positive for microalbuminuria. Protein-uria tests were considered positive if they returned a 1+ protein or greater result. We also recorded whether the subject was on an ACEI or ARB therapy, and if so at what date it had been prescribed.

To minimize inter-rater variability, the 2 medical students each reviewed a pilot sample of the same 20 charts. Data were compared and differences between the auditors were reviewed to standardize definitions of data elements. After standardization, sets of 10 different charts were selected, and the process was repeated until the data from 40 consecutive charts were recorded identically by both students.

Analysis

When comparing mean values, we performed a Student t test to determine statistical significance. A chi-square test was done to determine statistical significance when comparing proportions. A P value of <.05 was determined to be statistically significant.

Results

Of the 278 eligible patients, 44 (16%) had a urinalysis with 1+ or greater protein result at baseline; 18 (41%) of these were already taking an ACEI or ARB drug. In patients without previous evidence of proteinuria, 51 (18%) patients were using ACEI or ARB therapy. This left 183 patients (66%) who had no evidence of renal disease and who were not using ACEI or ARB therapy and therefore were the prime candidates for microalbuminuria screening Figure 1.

When we examined the demographics and clinical variables of these 3 groups, we found that patients with proteinuria or who were already using drug treatment were older and had higher systolic and diastolic blood pressures than those who were not. Unexpectedly, we also found that patients with existing proteinuria had lower Hb A_1C levels than patients in the other 2 categories.

Of these prime candidates for screening, only 31 (17%) received at least 1 microalbuminuria test between 1995 and 1999. The rate of screening in this group was no different from those who were taking an ACEI or ARB drug (16%, P=.83) or already had gross proteinuria (18%, P=.84).

When we examined the patients who were most likely to benefit from screening and looked at demographic or clinical factors that might influence whether a screening test was performed, we found that patients who received microalbuminuria testing were very similar to those who did not. The only difference we found was that patients who received screening had lower systolic blood pressures than those who were not screened. Weight, age, Hb A_1C levels, and cholesterol levels were not predictors of being screened for microalbuminuria Table 2.

Because of the low rates of microalbuminuria screening for patients who were eligible and the relatively frequent use of screening in patients who already had evidence of gross proteinuria, we were interested in what clinicians did when a microalbuminuria test result was positive. In the group without evidence of proteinuria and not using ACEI or ARB therapy, 10 of the 31 patients who received screening for microalbuminuria tested positive. However, only 4 (40%) were placed on ACE inhibitor or ARB therapy.

Discussion

Our data suggest that several problems exist in the use and interpretation of microalbuminuria testing in the primary care setting. First, microalbuminuria testing is being performed on only 1 of 5 adult patients with type 2 diabetes. Second, in this practice, testing is not targeted to the patients who are most likely to benefit from the results. Rather, the tests seemed to be used indiscriminately. Finally, even when patients are screened and found to have microalbuminuria, only a small percentage were started on appropriate therapy. At least in this patient population, it appears that ACEI or ARB therapy is reserved for patients with higher blood pressures rather than used for renal protection.

The observation that patients with existing proteinuria or who were on ACEI or ARB therapy were screened just as often as those who were prime candidates for screening contradicts our initial hypothesis. We had assumed that clinicians would not screen patients who were on ACEI or ARB therapy, reducing the overall screening rate. Apparently, this is not the case. At least in this practice, a low screening rate is not due to selective screening.

The lack of optimal use of microalbuminuria testing and the failure to respond appropriately to positive test results suggests that current recommendations have not been embraced by physicians. Also, the complexity of carrying out these recommendations may make it difficult to integrate this screening into routine practice. If the current evidence on ACEI and ARB therapy for the prevention of renal dysfunction is to be translated into practice, either greater emphasis needs to be placed on microalbuminuria screening or more efficient ways to provide renal protection for patients with diabetes should be considered. Other studies have found that between 17% and 30% of patients with type 2 diabetes have microalbuminuria.^1,12,13 Although primary care physicians report that they provide microalbuminuria screening to a large percentage of their patients with diabetes, in fact only a small percentage of those who should be screened actually are screened.¹⁰ Suboptimal screening rates for important conditions seen in primary care are not unique for microalbuminuria. Other studies have documented comparable low screening rates for a wide variety of cancers.¹⁴ Since physicians do not screen reliably for potentially fatal diseases with screening modalities that have been available for decades, it is unlikely that their behavior is likely to improve when asked to screen for microalbuminuria.

Also, recent evidence that ACEI therapy may improve endothelial function in patients with type 2 diabetes suggests that even patients without microalbuminuria may benefit from routine ACEI therapy.¹⁵ Other studies suggest that routine use of ACEIs in middle-aged patients with type 2 diabetes may provide substantial benefits at only modest costs compared with a screening strategy.¹⁶ These data suggest that a more effective strategy would be to advise that all patients with type 2 diabetes start ACEI or ARB therapy along with their medications for diabetes. This strategy would obviate the need for microalbuminuria screening, while assuring that patients receive any additional benefits of ACEI or ARB therapy unrelated to renal protection. However, using this strategy, patients who may not have proteinuria will have to take the medication for a prolonged period, pay for it, and run the risks for any complications associated with using the drug.

Limitations

Our study has several limitations. Only 1 practice was examined, and it was part of a residency training practice. This means that less-experienced clinicians were providing care that could reduce the overall rate of screening. However, the rate of screening observed in this study was very similar to rates found in the practices of clinicians with more experience,^11,12 suggesting that the lack of experience of resident physicians may be balanced by the oversight provided by faculty preceptors.

Another limitation is that it was not possible to account for microalbuminuria screening completed outside the MUSC medical center. Patients who split their care among several providers could have had testing performed in other health care facilities. However, since more than 95% of the referrals from the MUSC Family Medicine Center stay within the university health care system, it is doubtful that many patients would have received testing outside the search capabilities of the hospital laboratory database.

Finally, the study was limited in its power to detect small differences between the groups. We originally conceived our project as an exploratory study to determine how many patients were already taking ACEIs and the potential effect of this on overall screening rates for microalbuminuria. Without any reference for the percentage of patients who were taking ACEIs, we could not perform an ad hoc power analysis. However, a post hoc analysis shows that for a sample in which the groups are matched in a 1-to-3 ratio (approximating the proportion of the 51 patients in our sample taking ACEIs and the 183 not taking these drugs) and given the study sample size, our study had a power of 80% to detect a difference in screening rates between 20% in the baseline group and 5% in the ACE or ARB groups. The actual difference seen in our study was much smaller, which increases the possibility of a type II error.

Conclusions

Because physician use of microalbuminuria screening does not follow established guidelines, consideration should be given to other strategies to prevent nephropathy in persons with type 2 diabetes. One proposed strategy would advise all patients with type 2 diabetes to start ACEI or ARB therapy along with their medications for diabetes. This strategy would obviate the need for microalbuminuria screening, while ensuring that patients receive any additional benefits of ACEI or ARB therapy unrelated to renal protection. It is unknown, however, whether patients would accept universal treatment rather than periodic screening. This is an important question that should be addressed before any population-based strategies are adopted.

Six major reviews of the natural history, prevention, and treatment of diabetic nephropathy have been published.^1-6 Key points of these overviews with regard to screening are that persistent microalbuminuria (MA) is a reliable marker for the presence of diabetic nephropathy in both patients with type 1 (insulin-dependent) and type 2 (non–insulin-dependent) diabetes mellitus, and that angiotensin-converting enzyme inhibitors (ACEIs) slow or prevent the progression of diabetic nephropathy in patients with MA. The latter benefit occurs in both patients with type 1 and type 2 diabetes and appears to be long-lasting.

At least 6 sets of recommendations^7-13 that advocate routine MA screening in all patients with diabetes have been issued by various physicians’ groups and organizations. Current American Diabetes Association (ADA) guidelines permit 3 types of collection to measure urinary albumin excretion (UAE): 24-hour (<30 mg/24 hrs), timed (<20 mg/minute), and untimed random albumin/creatinine ratio (UACR, <30 mg/mg creatine),*Table 1w^7,8 Dipstick semiquantitative rapid tests are included in the ADA guidelines as alternatives if quantitative assays are not readily available, but they must be confirmed by quantitative methods. Others have suggested that semiquantitative tests should not be considered substitutes for the other methods.⁴ The variability of UAE is considered too high to use urine albumin concentration (UAC) alone. The average intraindividual daily UAE variation is approximately 40%; standing, exercise, illness, and diuresis all increase UAE.¹⁴ Because of this variation, the ADA guidelines recommend that 2 of 3 tests (performed over a 3- to 6-month period) should provide elevated results before the patient is considered to have MA.^7,8

Recommended strategies to overcome diagnostic uncertainty by repeated testing are difficult to follow in primary care settings and do not appear to improve diagnostic accuracy. Published recommendations are based on expert opinion regarding the performance of MA tests. However, no controlled trial of the effectiveness of MA screening has been reported.³ Although concern about diagnostic uncertainty has led to recommendations for repeated testing and the use of timed quantitative MA tests, these strategies may be difficult to follow in primary care settings and may not improve accuracy. With this systematic review we critically examined components of previous recommendations and addressed the question of whether persons with diabetes should be screened for MA.

Methods

We searched the MEDLINE database from 1966 to the present looking for studies describing diagnostic tests for MA. The search strategy included the medical subject headings “albuminuria” and “diabetes mellitus” and the text words “microalbuminuria,” “nephropathy or nephropathies,” and “screening or testing or diagnosis.” The reference lists of relevant articles and the 6 major review articles^1-6 were searched by hand to locate additional pertinent articles. A second search following identical procedures was performed substituting “cost-benefit analysis” for “screening or testing or diagnosis.” Screening articles were included if: (1) the subjects were only patients with diabetes, (2) the studies investigated the use of untimed urine samples (first morning [FAM], morning [AM], random urine sampling [RUS]), (3) tests were performed only in an ambulatory setting, and (4) test performance (sensitivity and specificity) could be determined from the article. Though it was not recommended in the ADA guidelines, we included semiquantitative tests as possible alternatives, because these tests are in clinical use and have been extensively studied. We initially included articles if any 1 of the 4 reviewers thought they met the inclusion criteria. Articles considered relevant by 3 of the 4 reviewers were included in the final round. Only English language articles reporting studies in human beings were selected. We did not seek unpublished data.

The quality of the screening articles was graded by a consensus of 2 reviewers using published criteria.¹⁵ Studies that presented sufficient information were examined to determine if the test characteristics could be pooled to give summary point estimates.¹⁶ We attempted to combine sensitivities and specificities reported for: (1) quantitative tests with a cutoff UAC of 20 mg per L or greater, (2) the same semiquantitative test used with any type of urine sample, or (3) the same semiquantitative test used with 1 type of urine sample. Using the chi-square test, we tested homogeneity among the sensitivities and specificities reported in the each of the studies. Studies were considered homogeneous if P was .05 or greater. Confidence intervals were calculated using the normal approximation to the binomial method.¹⁷

MA screening recommendations were analyzed using the criteria of Frame and Carlson¹⁸ and that of the US Preventive Services Task Force for determining effectiveness.¹⁹ The impact that repeated testing strategy recommended by the ADA⁸ had on diagnostic accuracy was analyzed using a clinical decision-making calculator.²⁰ The positive predictive value (PPV) was calculated by specifying the probability of “true MA,” the sensitivity and specificity of the MA test, and simultaneously varying both the phi coefficients (for cases with “true” MA and without MA) from 0 (independence) to 1 (dependence). The phi coefficient is a measure of the correlation between the dichotomous results of 2 tests in the presence or absence of the target condition. Cost-effectiveness analyses were assessed with the quality checklist of 37 critical features developed by Gold and colleagues.²¹

Results

Literature Yield

We retrieved 105 articles from the initial literature search and excluded 44 general review articles. The reference lists of the remaining 61 articles were reviewed to locate additional relevant articles. No controlled trials of screening to prevent progression to nephropathy or that compared sequential repeated screening strategies were identified. We found 31 articles that reported the performance of 1 MA screening test or more. Of these, 8 reported the characteristics of a quantitative test;^22-29 22 reported the characteristics of a semiquantitative test;^28,30-50 and 1 reported both.²⁸ Our review is unlikely to be affected by publication bias, because a wide range of results were reported from varied international sources.

We used a variety of cutoffs in the studies that reported quantitative UAC or UACR, which precluded pooling test characteristics of most of these studies. Because of the striking heterogeneity among studies and the existence of at least 1 large study for the 2 most commonly studied semiquantitative tests, we did not pool the sensitivities and specificities. The sensitivity ranged from 56% to 100% and specificity from -81% to 98% for UAC of 20 mg per L or greater for quantitative tests Table 1. For morning urine samples, the pooled sensitivity was 75% (95% confidence interval [CI], 59-91) and the pooled specificity was 97% (95% CI, 94-99).^23,26 Test performance was similar for all types of urine samples.

The sensitivity ranged from 51% to 100% and specificity from 21% to 100% for semiquantitative tests. Test performance was similar for all types of urine samples. Micral (Roche; Mannheim, Germany) was the most extensively reported semiquantitative test. A large (n=2228) multicenter study of the Micral II found a sensitivity of 96.7% and specificity of 71% to detect a UAC of 20 mg per L or greater by radioimmunoassay (RIA).⁴² The sensitivity of the Micro-Bumintest (Bayer; Pittsburgh, Pa) ranged from 60% to 100% and the specificity from 21% to 97%. A large (n=1186) population-based study of the Micro-Bumintest reported a sensitivity of 98.6% (95% CI, 97.5-99.6) and specificity of 85.1% (95% CI, 82.4-87.7) to detect a UAC of 30 mg per L or greater by RIA.⁴⁸

There is often considerable interobserver variation in the evaluation of semiquantitative tests that involve colorimetric changes. Mogenson and colleagues⁴² found 93% concordance of Micral results from 538 samples. The sensitivity of the Micral varies when used by different operators: general practitioners, 66%; laboratory technicians, 91%; and trained nurses, 84%. Ten percent of physicians who were less familiar with procedures accounted for 44% of the misread strips.³⁴ The Micral was not influenced by most potential interference factors,⁵¹ though it may be affected by freezing.^38,40 Authors have reported high numbers of false positives⁴⁷ and problems interpreting the results of the Micro-Bumintest tests.^52,53

Frame and Carlson Criteria for Screening Tests

The 6 criteria of Frame and Carlson¹⁸ we applied to MA screening Table 2 were introduced in 1975. There is adequate evidence to suggest that screening for MA meets the first 4 criteria.^4,54,55 Whether the test is acceptable to patients at a reasonable cost (criterion 5) and is cost-effective (criterion 6) is less certain.

Criterion 5: Tests must be acceptable to the patient and available at reasonable cost. A major limitation of any annual screening program is the proportion of false-positive tests that occur. During the first years of an annual screening program in a previously unscreened population with a high prevalence of disease, the proportion of false positives would be low. For example, in the first year of screening a population with a 40% prevalence of MA,¹⁴ using a test that is 90% sensitive and specific, the probability of having true MA after a single positive test would be 86% (the positive predictive value [PPV]). During subsequent years of a screening program, the prevalence of MA should approach the annual incidence of new disease, 1% to 4% per year.¹⁴ Therefore, the PPV of a single screening test in subsequent years could be expected to range from 8% to 27%.

To reduce the number of false positives, the ADA recommends that 2 of 3 screening tests be positive over a 3- to 6-month period before beginning treatment.^7,8,56 However, the degree of improvement that can be expected depends on the correlation between repeated tests. Although the UAE measure (in mg/minutes) has a high variance (coefficient of variation ranging from 33%-52%),⁵⁷ there is no published information on the correlation between errors on repeated screens when each is simply categorized as positive or negative for MA. However, Feldt-Rasmussen⁵⁷ calculated the probability of correct classification above or below 20 mg per minute using 1 sample compared with the median of 3 samples. Using 1 sample, specimens below 11 mg per minute and above 40 mg per minute had a greater than 95% probability of correct classification. By using the median of 3 samples, specimens below 13 mg per minute and above 32 mg per minute had a greater than 95% probability of correct classification. Most would agree that this is a clinically insignificant difference.

We analyzed the performance of a theoretical UAE test repeated up to 3 times according to ADA recommendations (considered negative if the first test is negative, or else the majority of 3 tests).⁸ Assuming an individual test sensitivity of 90%, a specificity of 90%, and a 10% prevalence of MA, we performed a sensitivity analysis of the effect of varying the correlations between repeated tests Figure 1.²⁰ This pretest probability was selected because it was between the estimate of 40% prevalence for the first year of screening and a 1% to 4% annual incidence of new disease. If the tests are completely independent (correlation=0), the probability of true MA if the multiple screen is positive is 84%, an improvement compared with the PPV of 50% for a single positive test. However, as the correlation (phi) between tests increases, the PPV of repeated testing decreases, approaching the PPV for a single test. To keep the PPV of repeated testing as high as 75%, the correlation between tests would have to be lower than approximately 0.1, which is quite unlikely. Thus, although MA screening tests are noninvasive and relatively inexpensive, current recommendations may impose a significant burden on patient management without necessarily improving diagnostic certainty.

Criterion 6: Incidence of disease must justify screening cost. Seven cost-effectiveness analyses of MA screening and treatment with ACEIs to prevent end-stage renal disease (ESRD) have been published Table 3.^58-64 Five of these studies estimated the cost-effectiveness of MA screening in persons with type 1 diabetes.^59-63 Three of these 5 studies^59,60,63 that found screening to be cost-saving assumed perfect testing for MA. In 1 study that considered false-positive tests, the additional cost of screening for MA was $27,042 per quality-adjusted year of life (QALY) saved, compared with simply screening for hypertension or macroalbuminuria.⁶¹

Because the incidence of a costly outcome such as ESRD is higher for persons with type 1 diabetes, MA screening is likely to be cost-effective in this population.⁶² However, the cost-effectiveness of screening persons with type 2 diabetes for MA, only 5% to 10% of whom will develop ESRD, has recently been analyzed.^58,64 These analyses assumed perfect screening characteristics, and one study⁶⁴ considered only Pima Indians, who have a higher incidence of ESRD. MA screening saved QALYs and reduced costs compared with screening for macroalbuminuria, but routine use of ACEIs for all persons with type 2 diabetes was cost-effective ($7500/QALY) when compared with screening.⁵⁸ No cost-effectiveness analysis to date has included false-positive tests and studied a more typical population.

Discussion

We found no controlled trials of screening to prevent progression to nephropathy.³ Recommendations for screening persons with diabetes for MA are based on expert opinion; the evidence to support the specific components of these recommendations is lacking. Several studies have also demonstrated that UACR has little advantage over the measurement of UAC alone.^25,29,47,65 Use of untimed urine samples avoids the need for 2 visits, collection equipment, the problems of inaccurate timing, urine storage at 4 °C, and transfer to laboratories.³⁸

Semiquantitative MA tests are not favored by the ADA8 but have an accuracy similar to quantitative tests. Though they may not be reliable when used by untrained health care providers, high sensitivities and specificities can be obtained by personnel other than laboratory technicians.³⁴ Semiquantitative tests have the important advantages of increased convenience and decreased cost, which may improve adherence to recommendations. Several authors have suggested that semiquantitative MA tests could at least substitute for the first quantitative test in a multiple test strategy,^28,36,45,66 and the ADA position has recently shifted to allow semiquantitative tests if quantitative tests are not readily available.⁶⁷

The Micral is the best studied test, appears reliable, and has a high sensitivity even at low UAC (20 mg/L). A pooled analysis of 10 previous studies of the Micral found a sensitivity of 92.3% and a specificity of 83.2%.⁶⁸ Results from studies were included that investigated 24-hour urine samples; homogeneity among the studies was not tested. Two large studies found a sensitivity of 90.1% to 96.7% and a specificity of 71% to 87%.^42,51 The Micro-Bumintest has good sensitivity but has been evaluated at a slightly higher cutoff UAC (30 mg/L), and the reliability has been questioned.^47,52,53

MA screening clearly meets only 4 of the 6 criteria of Frame and Carlson. Current recommendations for MA screening require repeated testing that is onerous and probably does not improve diagnostic certainty. This strategy has not been compared with simpler strategies in a randomized controlled trial. In our analysis, at low prevalence the theoretical improvement in specificity is minimal and would not seem to justify the need for a criterion of 2 of 3 tests positive.

A number of studies have reported on the poor rate of screening persons with diabetes in primary care.^69,70 In an academic family medicine center, Lawler and Viviani⁷¹ found that the patient-reported rate of MA screening was 43%. In a recent survey of primary care physicians, more than 40% reported screening no persons with type 2 diabetes for MA, and only 17% screened more than 50% of persons with type 1 diabetes.⁷² A recent analysis of insurance claims data for 4623 persons with diabetes found that only 2.1% of those without known nephropathy were tested for MA during the study year.⁷³ This lack of adherence to even single annual screening tests raises questions of whether the screening strategy of repeated screening followed by treatment will effectively prevent diabetic nephropathy. Strategies that incorporate using a semiquantitative test first may mitigate adherence problems, but the feasibility of such strategies has not been evaluated. A practice-based trial comparing screening strategies is needed.

Because of the high incidence of nephropathy and ESRD, MA screening in patients with type 1 diabetes is probably cost-effective. Screening persons with type 2 diabetes for MA is less certain. Analyses have generally not considered imperfect testing or the impact of sequential testing strategies. Based on studies that have demonstrated delayed progression in persons with diabetes who have normoalbuminuria,⁷⁴ 3 cost-effectiveness analyses found that routine use of ACEIs compared favorably with MA screening.^58,62,64 A cost-effective analysis that considered recommended testing strategies and imperfect screening would be useful.

MA is associated with a substantial risk of cardiovascular events.⁷⁵ The recent Heart Outcomes Prevention Evaluation Study found that ACEIs lower the risk of death, heart attack, stroke, and other complications of diabetes mellitus in high-risk patients with known cardiovascular disease.^76,77 Given the difficulties of changing patient and health provider behavior, a more compelling question, which we discuss in a subsequent article, is whether routinely prescribing ACEIs is more desirable than annual screening and treatment when MA is detected.

Acknowledgments

We would like to thank the many people who contributed their time reading and commenting on our manuscript. We also thank Alice Reed and Stacy Wigley for their help assembling and managing the reference databases for this review and for preparing some of the graphics.

Six major reviews of the natural history, prevention, and treatment of diabetic nephropathy have been published.^1-6 Key points of these overviews with regard to screening are that persistent microalbuminuria (MA) is a reliable marker for the presence of diabetic nephropathy in both patients with type 1 (insulin-dependent) and type 2 (non–insulin-dependent) diabetes mellitus, and that angiotensin-converting enzyme inhibitors (ACEIs) slow or prevent the progression of diabetic nephropathy in patients with MA. The latter benefit occurs in both patients with type 1 and type 2 diabetes and appears to be long-lasting.

At least 6 sets of recommendations^7-13 that advocate routine MA screening in all patients with diabetes have been issued by various physicians’ groups and organizations. Current American Diabetes Association (ADA) guidelines permit 3 types of collection to measure urinary albumin excretion (UAE): 24-hour (<30 mg/24 hrs), timed (<20 mg/minute), and untimed random albumin/creatinine ratio (UACR, <30 mg/mg creatine),*Table 1w^7,8 Dipstick semiquantitative rapid tests are included in the ADA guidelines as alternatives if quantitative assays are not readily available, but they must be confirmed by quantitative methods. Others have suggested that semiquantitative tests should not be considered substitutes for the other methods.⁴ The variability of UAE is considered too high to use urine albumin concentration (UAC) alone. The average intraindividual daily UAE variation is approximately 40%; standing, exercise, illness, and diuresis all increase UAE.¹⁴ Because of this variation, the ADA guidelines recommend that 2 of 3 tests (performed over a 3- to 6-month period) should provide elevated results before the patient is considered to have MA.^7,8

Recommended strategies to overcome diagnostic uncertainty by repeated testing are difficult to follow in primary care settings and do not appear to improve diagnostic accuracy. Published recommendations are based on expert opinion regarding the performance of MA tests. However, no controlled trial of the effectiveness of MA screening has been reported.³ Although concern about diagnostic uncertainty has led to recommendations for repeated testing and the use of timed quantitative MA tests, these strategies may be difficult to follow in primary care settings and may not improve accuracy. With this systematic review we critically examined components of previous recommendations and addressed the question of whether persons with diabetes should be screened for MA.

Methods

We searched the MEDLINE database from 1966 to the present looking for studies describing diagnostic tests for MA. The search strategy included the medical subject headings “albuminuria” and “diabetes mellitus” and the text words “microalbuminuria,” “nephropathy or nephropathies,” and “screening or testing or diagnosis.” The reference lists of relevant articles and the 6 major review articles^1-6 were searched by hand to locate additional pertinent articles. A second search following identical procedures was performed substituting “cost-benefit analysis” for “screening or testing or diagnosis.” Screening articles were included if: (1) the subjects were only patients with diabetes, (2) the studies investigated the use of untimed urine samples (first morning [FAM], morning [AM], random urine sampling [RUS]), (3) tests were performed only in an ambulatory setting, and (4) test performance (sensitivity and specificity) could be determined from the article. Though it was not recommended in the ADA guidelines, we included semiquantitative tests as possible alternatives, because these tests are in clinical use and have been extensively studied. We initially included articles if any 1 of the 4 reviewers thought they met the inclusion criteria. Articles considered relevant by 3 of the 4 reviewers were included in the final round. Only English language articles reporting studies in human beings were selected. We did not seek unpublished data.

The quality of the screening articles was graded by a consensus of 2 reviewers using published criteria.¹⁵ Studies that presented sufficient information were examined to determine if the test characteristics could be pooled to give summary point estimates.¹⁶ We attempted to combine sensitivities and specificities reported for: (1) quantitative tests with a cutoff UAC of 20 mg per L or greater, (2) the same semiquantitative test used with any type of urine sample, or (3) the same semiquantitative test used with 1 type of urine sample. Using the chi-square test, we tested homogeneity among the sensitivities and specificities reported in the each of the studies. Studies were considered homogeneous if P was .05 or greater. Confidence intervals were calculated using the normal approximation to the binomial method.¹⁷

MA screening recommendations were analyzed using the criteria of Frame and Carlson¹⁸ and that of the US Preventive Services Task Force for determining effectiveness.¹⁹ The impact that repeated testing strategy recommended by the ADA⁸ had on diagnostic accuracy was analyzed using a clinical decision-making calculator.²⁰ The positive predictive value (PPV) was calculated by specifying the probability of “true MA,” the sensitivity and specificity of the MA test, and simultaneously varying both the phi coefficients (for cases with “true” MA and without MA) from 0 (independence) to 1 (dependence). The phi coefficient is a measure of the correlation between the dichotomous results of 2 tests in the presence or absence of the target condition. Cost-effectiveness analyses were assessed with the quality checklist of 37 critical features developed by Gold and colleagues.²¹

Results

Literature Yield

We retrieved 105 articles from the initial literature search and excluded 44 general review articles. The reference lists of the remaining 61 articles were reviewed to locate additional relevant articles. No controlled trials of screening to prevent progression to nephropathy or that compared sequential repeated screening strategies were identified. We found 31 articles that reported the performance of 1 MA screening test or more. Of these, 8 reported the characteristics of a quantitative test;^22-29 22 reported the characteristics of a semiquantitative test;^28,30-50 and 1 reported both.²⁸ Our review is unlikely to be affected by publication bias, because a wide range of results were reported from varied international sources.

We used a variety of cutoffs in the studies that reported quantitative UAC or UACR, which precluded pooling test characteristics of most of these studies. Because of the striking heterogeneity among studies and the existence of at least 1 large study for the 2 most commonly studied semiquantitative tests, we did not pool the sensitivities and specificities. The sensitivity ranged from 56% to 100% and specificity from -81% to 98% for UAC of 20 mg per L or greater for quantitative tests Table 1. For morning urine samples, the pooled sensitivity was 75% (95% confidence interval [CI], 59-91) and the pooled specificity was 97% (95% CI, 94-99).^23,26 Test performance was similar for all types of urine samples.

The sensitivity ranged from 51% to 100% and specificity from 21% to 100% for semiquantitative tests. Test performance was similar for all types of urine samples. Micral (Roche; Mannheim, Germany) was the most extensively reported semiquantitative test. A large (n=2228) multicenter study of the Micral II found a sensitivity of 96.7% and specificity of 71% to detect a UAC of 20 mg per L or greater by radioimmunoassay (RIA).⁴² The sensitivity of the Micro-Bumintest (Bayer; Pittsburgh, Pa) ranged from 60% to 100% and the specificity from 21% to 97%. A large (n=1186) population-based study of the Micro-Bumintest reported a sensitivity of 98.6% (95% CI, 97.5-99.6) and specificity of 85.1% (95% CI, 82.4-87.7) to detect a UAC of 30 mg per L or greater by RIA.⁴⁸

There is often considerable interobserver variation in the evaluation of semiquantitative tests that involve colorimetric changes. Mogenson and colleagues⁴² found 93% concordance of Micral results from 538 samples. The sensitivity of the Micral varies when used by different operators: general practitioners, 66%; laboratory technicians, 91%; and trained nurses, 84%. Ten percent of physicians who were less familiar with procedures accounted for 44% of the misread strips.³⁴ The Micral was not influenced by most potential interference factors,⁵¹ though it may be affected by freezing.^38,40 Authors have reported high numbers of false positives⁴⁷ and problems interpreting the results of the Micro-Bumintest tests.^52,53

Frame and Carlson Criteria for Screening Tests

The 6 criteria of Frame and Carlson¹⁸ we applied to MA screening Table 2 were introduced in 1975. There is adequate evidence to suggest that screening for MA meets the first 4 criteria.^4,54,55 Whether the test is acceptable to patients at a reasonable cost (criterion 5) and is cost-effective (criterion 6) is less certain.

Criterion 5: Tests must be acceptable to the patient and available at reasonable cost. A major limitation of any annual screening program is the proportion of false-positive tests that occur. During the first years of an annual screening program in a previously unscreened population with a high prevalence of disease, the proportion of false positives would be low. For example, in the first year of screening a population with a 40% prevalence of MA,¹⁴ using a test that is 90% sensitive and specific, the probability of having true MA after a single positive test would be 86% (the positive predictive value [PPV]). During subsequent years of a screening program, the prevalence of MA should approach the annual incidence of new disease, 1% to 4% per year.¹⁴ Therefore, the PPV of a single screening test in subsequent years could be expected to range from 8% to 27%.

To reduce the number of false positives, the ADA recommends that 2 of 3 screening tests be positive over a 3- to 6-month period before beginning treatment.^7,8,56 However, the degree of improvement that can be expected depends on the correlation between repeated tests. Although the UAE measure (in mg/minutes) has a high variance (coefficient of variation ranging from 33%-52%),⁵⁷ there is no published information on the correlation between errors on repeated screens when each is simply categorized as positive or negative for MA. However, Feldt-Rasmussen⁵⁷ calculated the probability of correct classification above or below 20 mg per minute using 1 sample compared with the median of 3 samples. Using 1 sample, specimens below 11 mg per minute and above 40 mg per minute had a greater than 95% probability of correct classification. By using the median of 3 samples, specimens below 13 mg per minute and above 32 mg per minute had a greater than 95% probability of correct classification. Most would agree that this is a clinically insignificant difference.

We analyzed the performance of a theoretical UAE test repeated up to 3 times according to ADA recommendations (considered negative if the first test is negative, or else the majority of 3 tests).⁸ Assuming an individual test sensitivity of 90%, a specificity of 90%, and a 10% prevalence of MA, we performed a sensitivity analysis of the effect of varying the correlations between repeated tests Figure 1.²⁰ This pretest probability was selected because it was between the estimate of 40% prevalence for the first year of screening and a 1% to 4% annual incidence of new disease. If the tests are completely independent (correlation=0), the probability of true MA if the multiple screen is positive is 84%, an improvement compared with the PPV of 50% for a single positive test. However, as the correlation (phi) between tests increases, the PPV of repeated testing decreases, approaching the PPV for a single test. To keep the PPV of repeated testing as high as 75%, the correlation between tests would have to be lower than approximately 0.1, which is quite unlikely. Thus, although MA screening tests are noninvasive and relatively inexpensive, current recommendations may impose a significant burden on patient management without necessarily improving diagnostic certainty.

Criterion 6: Incidence of disease must justify screening cost. Seven cost-effectiveness analyses of MA screening and treatment with ACEIs to prevent end-stage renal disease (ESRD) have been published Table 3.^58-64 Five of these studies estimated the cost-effectiveness of MA screening in persons with type 1 diabetes.^59-63 Three of these 5 studies^59,60,63 that found screening to be cost-saving assumed perfect testing for MA. In 1 study that considered false-positive tests, the additional cost of screening for MA was $27,042 per quality-adjusted year of life (QALY) saved, compared with simply screening for hypertension or macroalbuminuria.⁶¹

Because the incidence of a costly outcome such as ESRD is higher for persons with type 1 diabetes, MA screening is likely to be cost-effective in this population.⁶² However, the cost-effectiveness of screening persons with type 2 diabetes for MA, only 5% to 10% of whom will develop ESRD, has recently been analyzed.^58,64 These analyses assumed perfect screening characteristics, and one study⁶⁴ considered only Pima Indians, who have a higher incidence of ESRD. MA screening saved QALYs and reduced costs compared with screening for macroalbuminuria, but routine use of ACEIs for all persons with type 2 diabetes was cost-effective ($7500/QALY) when compared with screening.⁵⁸ No cost-effectiveness analysis to date has included false-positive tests and studied a more typical population.

Discussion

We found no controlled trials of screening to prevent progression to nephropathy.³ Recommendations for screening persons with diabetes for MA are based on expert opinion; the evidence to support the specific components of these recommendations is lacking. Several studies have also demonstrated that UACR has little advantage over the measurement of UAC alone.^25,29,47,65 Use of untimed urine samples avoids the need for 2 visits, collection equipment, the problems of inaccurate timing, urine storage at 4 °C, and transfer to laboratories.³⁸

Semiquantitative MA tests are not favored by the ADA8 but have an accuracy similar to quantitative tests. Though they may not be reliable when used by untrained health care providers, high sensitivities and specificities can be obtained by personnel other than laboratory technicians.³⁴ Semiquantitative tests have the important advantages of increased convenience and decreased cost, which may improve adherence to recommendations. Several authors have suggested that semiquantitative MA tests could at least substitute for the first quantitative test in a multiple test strategy,^28,36,45,66 and the ADA position has recently shifted to allow semiquantitative tests if quantitative tests are not readily available.⁶⁷

The Micral is the best studied test, appears reliable, and has a high sensitivity even at low UAC (20 mg/L). A pooled analysis of 10 previous studies of the Micral found a sensitivity of 92.3% and a specificity of 83.2%.⁶⁸ Results from studies were included that investigated 24-hour urine samples; homogeneity among the studies was not tested. Two large studies found a sensitivity of 90.1% to 96.7% and a specificity of 71% to 87%.^42,51 The Micro-Bumintest has good sensitivity but has been evaluated at a slightly higher cutoff UAC (30 mg/L), and the reliability has been questioned.^47,52,53

MA screening clearly meets only 4 of the 6 criteria of Frame and Carlson. Current recommendations for MA screening require repeated testing that is onerous and probably does not improve diagnostic certainty. This strategy has not been compared with simpler strategies in a randomized controlled trial. In our analysis, at low prevalence the theoretical improvement in specificity is minimal and would not seem to justify the need for a criterion of 2 of 3 tests positive.

A number of studies have reported on the poor rate of screening persons with diabetes in primary care.^69,70 In an academic family medicine center, Lawler and Viviani⁷¹ found that the patient-reported rate of MA screening was 43%. In a recent survey of primary care physicians, more than 40% reported screening no persons with type 2 diabetes for MA, and only 17% screened more than 50% of persons with type 1 diabetes.⁷² A recent analysis of insurance claims data for 4623 persons with diabetes found that only 2.1% of those without known nephropathy were tested for MA during the study year.⁷³ This lack of adherence to even single annual screening tests raises questions of whether the screening strategy of repeated screening followed by treatment will effectively prevent diabetic nephropathy. Strategies that incorporate using a semiquantitative test first may mitigate adherence problems, but the feasibility of such strategies has not been evaluated. A practice-based trial comparing screening strategies is needed.

Because of the high incidence of nephropathy and ESRD, MA screening in patients with type 1 diabetes is probably cost-effective. Screening persons with type 2 diabetes for MA is less certain. Analyses have generally not considered imperfect testing or the impact of sequential testing strategies. Based on studies that have demonstrated delayed progression in persons with diabetes who have normoalbuminuria,⁷⁴ 3 cost-effectiveness analyses found that routine use of ACEIs compared favorably with MA screening.^58,62,64 A cost-effective analysis that considered recommended testing strategies and imperfect screening would be useful.

MA is associated with a substantial risk of cardiovascular events.⁷⁵ The recent Heart Outcomes Prevention Evaluation Study found that ACEIs lower the risk of death, heart attack, stroke, and other complications of diabetes mellitus in high-risk patients with known cardiovascular disease.^76,77 Given the difficulties of changing patient and health provider behavior, a more compelling question, which we discuss in a subsequent article, is whether routinely prescribing ACEIs is more desirable than annual screening and treatment when MA is detected.

Acknowledgments

We would like to thank the many people who contributed their time reading and commenting on our manuscript. We also thank Alice Reed and Stacy Wigley for their help assembling and managing the reference databases for this review and for preparing some of the graphics.

Six major reviews of the natural history, prevention, and treatment of diabetic nephropathy have been published.^1-6 Key points of these overviews with regard to screening are that persistent microalbuminuria (MA) is a reliable marker for the presence of diabetic nephropathy in both patients with type 1 (insulin-dependent) and type 2 (non–insulin-dependent) diabetes mellitus, and that angiotensin-converting enzyme inhibitors (ACEIs) slow or prevent the progression of diabetic nephropathy in patients with MA. The latter benefit occurs in both patients with type 1 and type 2 diabetes and appears to be long-lasting.

At least 6 sets of recommendations^7-13 that advocate routine MA screening in all patients with diabetes have been issued by various physicians’ groups and organizations. Current American Diabetes Association (ADA) guidelines permit 3 types of collection to measure urinary albumin excretion (UAE): 24-hour (<30 mg/24 hrs), timed (<20 mg/minute), and untimed random albumin/creatinine ratio (UACR, <30 mg/mg creatine),*Table 1w^7,8 Dipstick semiquantitative rapid tests are included in the ADA guidelines as alternatives if quantitative assays are not readily available, but they must be confirmed by quantitative methods. Others have suggested that semiquantitative tests should not be considered substitutes for the other methods.⁴ The variability of UAE is considered too high to use urine albumin concentration (UAC) alone. The average intraindividual daily UAE variation is approximately 40%; standing, exercise, illness, and diuresis all increase UAE.¹⁴ Because of this variation, the ADA guidelines recommend that 2 of 3 tests (performed over a 3- to 6-month period) should provide elevated results before the patient is considered to have MA.^7,8

Recommended strategies to overcome diagnostic uncertainty by repeated testing are difficult to follow in primary care settings and do not appear to improve diagnostic accuracy. Published recommendations are based on expert opinion regarding the performance of MA tests. However, no controlled trial of the effectiveness of MA screening has been reported.³ Although concern about diagnostic uncertainty has led to recommendations for repeated testing and the use of timed quantitative MA tests, these strategies may be difficult to follow in primary care settings and may not improve accuracy. With this systematic review we critically examined components of previous recommendations and addressed the question of whether persons with diabetes should be screened for MA.

Methods

We searched the MEDLINE database from 1966 to the present looking for studies describing diagnostic tests for MA. The search strategy included the medical subject headings “albuminuria” and “diabetes mellitus” and the text words “microalbuminuria,” “nephropathy or nephropathies,” and “screening or testing or diagnosis.” The reference lists of relevant articles and the 6 major review articles^1-6 were searched by hand to locate additional pertinent articles. A second search following identical procedures was performed substituting “cost-benefit analysis” for “screening or testing or diagnosis.” Screening articles were included if: (1) the subjects were only patients with diabetes, (2) the studies investigated the use of untimed urine samples (first morning [FAM], morning [AM], random urine sampling [RUS]), (3) tests were performed only in an ambulatory setting, and (4) test performance (sensitivity and specificity) could be determined from the article. Though it was not recommended in the ADA guidelines, we included semiquantitative tests as possible alternatives, because these tests are in clinical use and have been extensively studied. We initially included articles if any 1 of the 4 reviewers thought they met the inclusion criteria. Articles considered relevant by 3 of the 4 reviewers were included in the final round. Only English language articles reporting studies in human beings were selected. We did not seek unpublished data.

The quality of the screening articles was graded by a consensus of 2 reviewers using published criteria.¹⁵ Studies that presented sufficient information were examined to determine if the test characteristics could be pooled to give summary point estimates.¹⁶ We attempted to combine sensitivities and specificities reported for: (1) quantitative tests with a cutoff UAC of 20 mg per L or greater, (2) the same semiquantitative test used with any type of urine sample, or (3) the same semiquantitative test used with 1 type of urine sample. Using the chi-square test, we tested homogeneity among the sensitivities and specificities reported in the each of the studies. Studies were considered homogeneous if P was .05 or greater. Confidence intervals were calculated using the normal approximation to the binomial method.¹⁷

MA screening recommendations were analyzed using the criteria of Frame and Carlson¹⁸ and that of the US Preventive Services Task Force for determining effectiveness.¹⁹ The impact that repeated testing strategy recommended by the ADA⁸ had on diagnostic accuracy was analyzed using a clinical decision-making calculator.²⁰ The positive predictive value (PPV) was calculated by specifying the probability of “true MA,” the sensitivity and specificity of the MA test, and simultaneously varying both the phi coefficients (for cases with “true” MA and without MA) from 0 (independence) to 1 (dependence). The phi coefficient is a measure of the correlation between the dichotomous results of 2 tests in the presence or absence of the target condition. Cost-effectiveness analyses were assessed with the quality checklist of 37 critical features developed by Gold and colleagues.²¹

Results

Literature Yield

We retrieved 105 articles from the initial literature search and excluded 44 general review articles. The reference lists of the remaining 61 articles were reviewed to locate additional relevant articles. No controlled trials of screening to prevent progression to nephropathy or that compared sequential repeated screening strategies were identified. We found 31 articles that reported the performance of 1 MA screening test or more. Of these, 8 reported the characteristics of a quantitative test;^22-29 22 reported the characteristics of a semiquantitative test;^28,30-50 and 1 reported both.²⁸ Our review is unlikely to be affected by publication bias, because a wide range of results were reported from varied international sources.

We used a variety of cutoffs in the studies that reported quantitative UAC or UACR, which precluded pooling test characteristics of most of these studies. Because of the striking heterogeneity among studies and the existence of at least 1 large study for the 2 most commonly studied semiquantitative tests, we did not pool the sensitivities and specificities. The sensitivity ranged from 56% to 100% and specificity from -81% to 98% for UAC of 20 mg per L or greater for quantitative tests Table 1. For morning urine samples, the pooled sensitivity was 75% (95% confidence interval [CI], 59-91) and the pooled specificity was 97% (95% CI, 94-99).^23,26 Test performance was similar for all types of urine samples.

The sensitivity ranged from 51% to 100% and specificity from 21% to 100% for semiquantitative tests. Test performance was similar for all types of urine samples. Micral (Roche; Mannheim, Germany) was the most extensively reported semiquantitative test. A large (n=2228) multicenter study of the Micral II found a sensitivity of 96.7% and specificity of 71% to detect a UAC of 20 mg per L or greater by radioimmunoassay (RIA).⁴² The sensitivity of the Micro-Bumintest (Bayer; Pittsburgh, Pa) ranged from 60% to 100% and the specificity from 21% to 97%. A large (n=1186) population-based study of the Micro-Bumintest reported a sensitivity of 98.6% (95% CI, 97.5-99.6) and specificity of 85.1% (95% CI, 82.4-87.7) to detect a UAC of 30 mg per L or greater by RIA.⁴⁸

There is often considerable interobserver variation in the evaluation of semiquantitative tests that involve colorimetric changes. Mogenson and colleagues⁴² found 93% concordance of Micral results from 538 samples. The sensitivity of the Micral varies when used by different operators: general practitioners, 66%; laboratory technicians, 91%; and trained nurses, 84%. Ten percent of physicians who were less familiar with procedures accounted for 44% of the misread strips.³⁴ The Micral was not influenced by most potential interference factors,⁵¹ though it may be affected by freezing.^38,40 Authors have reported high numbers of false positives⁴⁷ and problems interpreting the results of the Micro-Bumintest tests.^52,53

Frame and Carlson Criteria for Screening Tests

The 6 criteria of Frame and Carlson¹⁸ we applied to MA screening Table 2 were introduced in 1975. There is adequate evidence to suggest that screening for MA meets the first 4 criteria.^4,54,55 Whether the test is acceptable to patients at a reasonable cost (criterion 5) and is cost-effective (criterion 6) is less certain.

Criterion 5: Tests must be acceptable to the patient and available at reasonable cost. A major limitation of any annual screening program is the proportion of false-positive tests that occur. During the first years of an annual screening program in a previously unscreened population with a high prevalence of disease, the proportion of false positives would be low. For example, in the first year of screening a population with a 40% prevalence of MA,¹⁴ using a test that is 90% sensitive and specific, the probability of having true MA after a single positive test would be 86% (the positive predictive value [PPV]). During subsequent years of a screening program, the prevalence of MA should approach the annual incidence of new disease, 1% to 4% per year.¹⁴ Therefore, the PPV of a single screening test in subsequent years could be expected to range from 8% to 27%.

To reduce the number of false positives, the ADA recommends that 2 of 3 screening tests be positive over a 3- to 6-month period before beginning treatment.^7,8,56 However, the degree of improvement that can be expected depends on the correlation between repeated tests. Although the UAE measure (in mg/minutes) has a high variance (coefficient of variation ranging from 33%-52%),⁵⁷ there is no published information on the correlation between errors on repeated screens when each is simply categorized as positive or negative for MA. However, Feldt-Rasmussen⁵⁷ calculated the probability of correct classification above or below 20 mg per minute using 1 sample compared with the median of 3 samples. Using 1 sample, specimens below 11 mg per minute and above 40 mg per minute had a greater than 95% probability of correct classification. By using the median of 3 samples, specimens below 13 mg per minute and above 32 mg per minute had a greater than 95% probability of correct classification. Most would agree that this is a clinically insignificant difference.

We analyzed the performance of a theoretical UAE test repeated up to 3 times according to ADA recommendations (considered negative if the first test is negative, or else the majority of 3 tests).⁸ Assuming an individual test sensitivity of 90%, a specificity of 90%, and a 10% prevalence of MA, we performed a sensitivity analysis of the effect of varying the correlations between repeated tests Figure 1.²⁰ This pretest probability was selected because it was between the estimate of 40% prevalence for the first year of screening and a 1% to 4% annual incidence of new disease. If the tests are completely independent (correlation=0), the probability of true MA if the multiple screen is positive is 84%, an improvement compared with the PPV of 50% for a single positive test. However, as the correlation (phi) between tests increases, the PPV of repeated testing decreases, approaching the PPV for a single test. To keep the PPV of repeated testing as high as 75%, the correlation between tests would have to be lower than approximately 0.1, which is quite unlikely. Thus, although MA screening tests are noninvasive and relatively inexpensive, current recommendations may impose a significant burden on patient management without necessarily improving diagnostic certainty.

Criterion 6: Incidence of disease must justify screening cost. Seven cost-effectiveness analyses of MA screening and treatment with ACEIs to prevent end-stage renal disease (ESRD) have been published Table 3.^58-64 Five of these studies estimated the cost-effectiveness of MA screening in persons with type 1 diabetes.^59-63 Three of these 5 studies^59,60,63 that found screening to be cost-saving assumed perfect testing for MA. In 1 study that considered false-positive tests, the additional cost of screening for MA was $27,042 per quality-adjusted year of life (QALY) saved, compared with simply screening for hypertension or macroalbuminuria.⁶¹

Because the incidence of a costly outcome such as ESRD is higher for persons with type 1 diabetes, MA screening is likely to be cost-effective in this population.⁶² However, the cost-effectiveness of screening persons with type 2 diabetes for MA, only 5% to 10% of whom will develop ESRD, has recently been analyzed.^58,64 These analyses assumed perfect screening characteristics, and one study⁶⁴ considered only Pima Indians, who have a higher incidence of ESRD. MA screening saved QALYs and reduced costs compared with screening for macroalbuminuria, but routine use of ACEIs for all persons with type 2 diabetes was cost-effective ($7500/QALY) when compared with screening.⁵⁸ No cost-effectiveness analysis to date has included false-positive tests and studied a more typical population.

Discussion

We found no controlled trials of screening to prevent progression to nephropathy.³ Recommendations for screening persons with diabetes for MA are based on expert opinion; the evidence to support the specific components of these recommendations is lacking. Several studies have also demonstrated that UACR has little advantage over the measurement of UAC alone.^25,29,47,65 Use of untimed urine samples avoids the need for 2 visits, collection equipment, the problems of inaccurate timing, urine storage at 4 °C, and transfer to laboratories.³⁸

Semiquantitative MA tests are not favored by the ADA8 but have an accuracy similar to quantitative tests. Though they may not be reliable when used by untrained health care providers, high sensitivities and specificities can be obtained by personnel other than laboratory technicians.³⁴ Semiquantitative tests have the important advantages of increased convenience and decreased cost, which may improve adherence to recommendations. Several authors have suggested that semiquantitative MA tests could at least substitute for the first quantitative test in a multiple test strategy,^28,36,45,66 and the ADA position has recently shifted to allow semiquantitative tests if quantitative tests are not readily available.⁶⁷

The Micral is the best studied test, appears reliable, and has a high sensitivity even at low UAC (20 mg/L). A pooled analysis of 10 previous studies of the Micral found a sensitivity of 92.3% and a specificity of 83.2%.⁶⁸ Results from studies were included that investigated 24-hour urine samples; homogeneity among the studies was not tested. Two large studies found a sensitivity of 90.1% to 96.7% and a specificity of 71% to 87%.^42,51 The Micro-Bumintest has good sensitivity but has been evaluated at a slightly higher cutoff UAC (30 mg/L), and the reliability has been questioned.^47,52,53

MA screening clearly meets only 4 of the 6 criteria of Frame and Carlson. Current recommendations for MA screening require repeated testing that is onerous and probably does not improve diagnostic certainty. This strategy has not been compared with simpler strategies in a randomized controlled trial. In our analysis, at low prevalence the theoretical improvement in specificity is minimal and would not seem to justify the need for a criterion of 2 of 3 tests positive.

A number of studies have reported on the poor rate of screening persons with diabetes in primary care.^69,70 In an academic family medicine center, Lawler and Viviani⁷¹ found that the patient-reported rate of MA screening was 43%. In a recent survey of primary care physicians, more than 40% reported screening no persons with type 2 diabetes for MA, and only 17% screened more than 50% of persons with type 1 diabetes.⁷² A recent analysis of insurance claims data for 4623 persons with diabetes found that only 2.1% of those without known nephropathy were tested for MA during the study year.⁷³ This lack of adherence to even single annual screening tests raises questions of whether the screening strategy of repeated screening followed by treatment will effectively prevent diabetic nephropathy. Strategies that incorporate using a semiquantitative test first may mitigate adherence problems, but the feasibility of such strategies has not been evaluated. A practice-based trial comparing screening strategies is needed.

Because of the high incidence of nephropathy and ESRD, MA screening in patients with type 1 diabetes is probably cost-effective. Screening persons with type 2 diabetes for MA is less certain. Analyses have generally not considered imperfect testing or the impact of sequential testing strategies. Based on studies that have demonstrated delayed progression in persons with diabetes who have normoalbuminuria,⁷⁴ 3 cost-effectiveness analyses found that routine use of ACEIs compared favorably with MA screening.^58,62,64 A cost-effective analysis that considered recommended testing strategies and imperfect screening would be useful.

MA is associated with a substantial risk of cardiovascular events.⁷⁵ The recent Heart Outcomes Prevention Evaluation Study found that ACEIs lower the risk of death, heart attack, stroke, and other complications of diabetes mellitus in high-risk patients with known cardiovascular disease.^76,77 Given the difficulties of changing patient and health provider behavior, a more compelling question, which we discuss in a subsequent article, is whether routinely prescribing ACEIs is more desirable than annual screening and treatment when MA is detected.

Acknowledgments

We would like to thank the many people who contributed their time reading and commenting on our manuscript. We also thank Alice Reed and Stacy Wigley for their help assembling and managing the reference databases for this review and for preparing some of the graphics.

Since ancient times, athletes have ingested substances to gain a competitive edge.¹ The use of such supplements, however, is not restricted to athletes. In 1996, Americans spent $6.5 billion on dietary supplements.² Individuals cite many reasons for using such supplements, including to ensure good nutrition, prevent illness, improve performance, ward off fatigue, and enhance personal appearance.³

Since 1998, interest in the ergogenic effects of products currently marketed as nutritional supplements, in particular creatine and androstenedione, has increased. Creatine is currently the most popular dietary ergogenic supplement.⁴ The reported benefits of creatine include increased energy during short-term intense exercise, increased muscle mass, increased strength, increased lean body mass, and decreased lactate accumulation during intense exercise. Although it is clear that supplementation raises intramuscular creatine stores,⁵ it remains unclear how effective creatine is as an ergogenic aid. Generally, it is felt that creatine supplementation may be useful for repeated bouts of high-intensity short-duration exercise.⁶ Claims of increased strength and muscle mass have not, however, been unequivocally proven.

Androstenedione is a steroid hormone. Many sporting communities, including the International Olympic Committee, have banned its use. By itself, androstenedione is weakly androgenic. To date, the largest controlled trial examining the effectiveness of androstenedione as an ergogenic aid showed no significant gains in muscular strength compared with a standard program of resistance training.⁷ Methodologic concerns have been raised about this particular study,⁸ because supplement manufacturers are not subject to United States Food and Drug Administration approval if no unsupported claims of efficacy are made. Therefore, sales of ergogenic supplements remain brisk.

Within this context, our study was designed to examine the prevalence of ergogenic supplement use in a young population. We also sought to determine the extent to which supplement use was associated with personal health risk behaviors and to determine how effective ergogenic supplements were perceived to be.

Methods

Sample Recruitment

Our study was conducted at Lackland Air Force Base, Texas, and Parris Island, South Carolina, from June through September, 1999. All men and women entering military basic training at these sites were eligible to participate. Individuals were randomly selected to receive the survey. There were no specific exclusion criteria.

Survey Administration

The Youth Risk Behavior Survey (YRBS)⁹ was modified for use in our study. Specifically, questions about nutritional ergogenic supplement use were added to the standard instrument that is used to examine health risk and preventive behaviors, such as tobacco use, alcohol use, seat belt use, helmet use, and suicide. The subjects were specifically asked to report use of creatine and androstenedione. They were also given an open-ended opportunity to report any other ergogenic supplement use.

Data Analysis

We used basic descriptive statistics for categorical variables. Proportions were compared using the chi-square test of contingency. The Fisher exact test was used for rare responses. Continuous variables were compared using the Student t test for independent samples. We calculated adjusted odds ratios with a multivariate logistic regression analysis controlling for sex, age, and body mass index.

Results

Of 550 individuals recruited for participation, 499 (91%) completed the survey. The respondents ranged in age from 17 to 35 years (mean age = 21 years). The study sample was 88% men (n=439) and 12% women (n=60). The lifetime prevalence of ergogenic supplement use in our study population was 41%. Creatine was the most commonly used ergogenic supplement (n=117; 23% of the study sample), followed by androstenedione (n=38; 8%).

Ergogenic supplement use was significantly associated with several health risk behaviors. Individuals who used ergogenic supplements were more likely to drink alcohol, more likely to drink heavily, more likely to ride in a vehicle with someone who had been drinking, more likely to drive after drinking, and more likely to have been in a physical fight compared with those who had not used supplements Table 1.

Men were more likely to use ergogenic nutritional supplements than women (P <.001). No difference in the pattern of supplement use was noted relative to age or body mass index. An equal number of respondents (70%) felt that creatine, androstenedione, and anabolic steroids were all effective at building muscle strength. Men, respondents younger than 20 years, and individuals who reported using ergogenic supplements were all more likely to believe in the effectiveness of creatine, androstenedione, and steroids compared with others (P <.05).

Discussion

Our study has 3 important findings: (1) a significant number of young people have used ergogenic supplements; (2) a similar majority of respondents believe creatine, androstenedione, and anabolic steroids are effective with respect to gains in muscle strength and mass; and (3) use of nutritional ergogenic supplements is associated with certain high-risk health behaviors.

Although it has been reported that 38% of high school athletes use vitamin supplements,¹⁰ the prevalence rates of creatine or androstenedione use in adolescents are not known. Our study suggests that a significant number of young persons use ergogenic supplements, creatine in particular. The self-reported prevalence rate of 41% of individuals who have used ergogenic supplements represents a baseline that needs to be verified by further studies.

A majority of respondents (70%) believed that creatine and androstenedione are effective in increasing muscle strength. There is limited evidence to support this in the medical literature. This is important given the credibility gap the medical community created when discussing anabolic steroids with athletes and adolescents in the past. Physicians routinely counseled that steroids were ineffective in building muscle strength and mass. Athletes’ experiences in the gym and on the playing field proved otherwise,¹¹ however, and a pervading sense of mistrust developed among athletes toward the medical community.¹² Our study shows that many young individuals believe that ergogenic supplements are effective. Some of this belief may stem from word of mouth,¹³ some from savvy marketing, and some from perceived experience. Whatever the case, physicians must be careful to provide accurate clinical information, while not alienating patients, when discussing ergogenic supplements. Our results also suggest that individuals who have used ergogenics are more likely to believe in their effectiveness. This agrees with the model suggesting that users of “natural” products are more likely to believe in their safety and efficacy than nonusers.¹⁴

Finally, and perhaps most important, our study indicates that supplement use is associated with certain health risk behaviors. Specifically, ergogenic supplement users are more likely to drink alcohol heavily, ride as a passenger with a driver who has been drinking, drive after drinking, and get involved in a physical fight. A similar pattern of behavior has been described among adolescents who use anabolic steroids.¹⁵ Although the prevalence of anabolic steroid use in the general adolescent population has been estimated at 3% to 10%,^16,17 our study shows a much higher rate of ergogenic supplement use. This establishes an important link in behavioral data between nutritional supplement users and anabolic steroid users. While steroids are illegal, ergogenic supplements are readily available. If, as our results suggest, ergogenic use is associated with a risk-behavior syndrome similar to that seen with steroid use, this is a valuable marker for physicians to use to provide targeted preventive counseling.

Limitations

Our study, however, has several important limitations. The respondents were all military recruits. Caution must therefore be used before generalizing the results to an unselected population. The prevalence of risk behaviors reported by the respondents relative to data from the 1999 YRBS¹⁸ also show several important differences Table 1. Our respondents appear more likely to drink, less likely to ride with someone who had been drinking, more likely to smoke, more likely to wear a seatbelt, less likely to fight, and less likely to report symptoms of depression compared with general data from the YRBS. The prevalence of other risk behaviors appears to be similar. It is uncertain whether these behavioral differences are particular to a military population. Another limitation of our study is the use of self-reported data, which potentially introduces a significant recall bias.

Conclusions

Our study suggests that in a young healthy military population there is widespread belief that creatine and androstenedione are effective ergogenic agents. The prevalence of supplement use in this population is significant, and such use is associated with certain health risk behaviors. This pattern of behaviors appears similar to a risk-behavior syndrome previously noted with anabolic steroid use. Clinicians should routinely ask adolescents and athletes about their use of dietary supplements. They should ask which supplements their patients are using, why they are using supplements, and what doses they are taking. Clinicians should be alert so they can provide targeted preventive counseling for individuals who are identified as users of ergogenic supplements.

Acknowledgments

Our study was supported by Uniformed Services Academy of Family Physicians grant G-1879.

Related resources

• FDA Dietary Supplement Consumer Information Homepage: http://www.cfsan.fda.gov/~dms/supplmnt.html
• USDA Food and Nutrition Information Center http://www.nalusda.gov/fnic/etext/000015.html
• Australian Academy of Sciences—Drugs in Sport http://www.science.org.au/nova/055/055sit.htm

Since ancient times, athletes have ingested substances to gain a competitive edge.¹ The use of such supplements, however, is not restricted to athletes. In 1996, Americans spent $6.5 billion on dietary supplements.² Individuals cite many reasons for using such supplements, including to ensure good nutrition, prevent illness, improve performance, ward off fatigue, and enhance personal appearance.³

Since 1998, interest in the ergogenic effects of products currently marketed as nutritional supplements, in particular creatine and androstenedione, has increased. Creatine is currently the most popular dietary ergogenic supplement.⁴ The reported benefits of creatine include increased energy during short-term intense exercise, increased muscle mass, increased strength, increased lean body mass, and decreased lactate accumulation during intense exercise. Although it is clear that supplementation raises intramuscular creatine stores,⁵ it remains unclear how effective creatine is as an ergogenic aid. Generally, it is felt that creatine supplementation may be useful for repeated bouts of high-intensity short-duration exercise.⁶ Claims of increased strength and muscle mass have not, however, been unequivocally proven.

Androstenedione is a steroid hormone. Many sporting communities, including the International Olympic Committee, have banned its use. By itself, androstenedione is weakly androgenic. To date, the largest controlled trial examining the effectiveness of androstenedione as an ergogenic aid showed no significant gains in muscular strength compared with a standard program of resistance training.⁷ Methodologic concerns have been raised about this particular study,⁸ because supplement manufacturers are not subject to United States Food and Drug Administration approval if no unsupported claims of efficacy are made. Therefore, sales of ergogenic supplements remain brisk.

Within this context, our study was designed to examine the prevalence of ergogenic supplement use in a young population. We also sought to determine the extent to which supplement use was associated with personal health risk behaviors and to determine how effective ergogenic supplements were perceived to be.

Methods

Sample Recruitment

Our study was conducted at Lackland Air Force Base, Texas, and Parris Island, South Carolina, from June through September, 1999. All men and women entering military basic training at these sites were eligible to participate. Individuals were randomly selected to receive the survey. There were no specific exclusion criteria.

Survey Administration

The Youth Risk Behavior Survey (YRBS)⁹ was modified for use in our study. Specifically, questions about nutritional ergogenic supplement use were added to the standard instrument that is used to examine health risk and preventive behaviors, such as tobacco use, alcohol use, seat belt use, helmet use, and suicide. The subjects were specifically asked to report use of creatine and androstenedione. They were also given an open-ended opportunity to report any other ergogenic supplement use.

Data Analysis

We used basic descriptive statistics for categorical variables. Proportions were compared using the chi-square test of contingency. The Fisher exact test was used for rare responses. Continuous variables were compared using the Student t test for independent samples. We calculated adjusted odds ratios with a multivariate logistic regression analysis controlling for sex, age, and body mass index.

Results

Of 550 individuals recruited for participation, 499 (91%) completed the survey. The respondents ranged in age from 17 to 35 years (mean age = 21 years). The study sample was 88% men (n=439) and 12% women (n=60). The lifetime prevalence of ergogenic supplement use in our study population was 41%. Creatine was the most commonly used ergogenic supplement (n=117; 23% of the study sample), followed by androstenedione (n=38; 8%).

Ergogenic supplement use was significantly associated with several health risk behaviors. Individuals who used ergogenic supplements were more likely to drink alcohol, more likely to drink heavily, more likely to ride in a vehicle with someone who had been drinking, more likely to drive after drinking, and more likely to have been in a physical fight compared with those who had not used supplements Table 1.

Men were more likely to use ergogenic nutritional supplements than women (P <.001). No difference in the pattern of supplement use was noted relative to age or body mass index. An equal number of respondents (70%) felt that creatine, androstenedione, and anabolic steroids were all effective at building muscle strength. Men, respondents younger than 20 years, and individuals who reported using ergogenic supplements were all more likely to believe in the effectiveness of creatine, androstenedione, and steroids compared with others (P <.05).

Discussion

Our study has 3 important findings: (1) a significant number of young people have used ergogenic supplements; (2) a similar majority of respondents believe creatine, androstenedione, and anabolic steroids are effective with respect to gains in muscle strength and mass; and (3) use of nutritional ergogenic supplements is associated with certain high-risk health behaviors.

Although it has been reported that 38% of high school athletes use vitamin supplements,¹⁰ the prevalence rates of creatine or androstenedione use in adolescents are not known. Our study suggests that a significant number of young persons use ergogenic supplements, creatine in particular. The self-reported prevalence rate of 41% of individuals who have used ergogenic supplements represents a baseline that needs to be verified by further studies.

A majority of respondents (70%) believed that creatine and androstenedione are effective in increasing muscle strength. There is limited evidence to support this in the medical literature. This is important given the credibility gap the medical community created when discussing anabolic steroids with athletes and adolescents in the past. Physicians routinely counseled that steroids were ineffective in building muscle strength and mass. Athletes’ experiences in the gym and on the playing field proved otherwise,¹¹ however, and a pervading sense of mistrust developed among athletes toward the medical community.¹² Our study shows that many young individuals believe that ergogenic supplements are effective. Some of this belief may stem from word of mouth,¹³ some from savvy marketing, and some from perceived experience. Whatever the case, physicians must be careful to provide accurate clinical information, while not alienating patients, when discussing ergogenic supplements. Our results also suggest that individuals who have used ergogenics are more likely to believe in their effectiveness. This agrees with the model suggesting that users of “natural” products are more likely to believe in their safety and efficacy than nonusers.¹⁴

Finally, and perhaps most important, our study indicates that supplement use is associated with certain health risk behaviors. Specifically, ergogenic supplement users are more likely to drink alcohol heavily, ride as a passenger with a driver who has been drinking, drive after drinking, and get involved in a physical fight. A similar pattern of behavior has been described among adolescents who use anabolic steroids.¹⁵ Although the prevalence of anabolic steroid use in the general adolescent population has been estimated at 3% to 10%,^16,17 our study shows a much higher rate of ergogenic supplement use. This establishes an important link in behavioral data between nutritional supplement users and anabolic steroid users. While steroids are illegal, ergogenic supplements are readily available. If, as our results suggest, ergogenic use is associated with a risk-behavior syndrome similar to that seen with steroid use, this is a valuable marker for physicians to use to provide targeted preventive counseling.

Limitations

Our study, however, has several important limitations. The respondents were all military recruits. Caution must therefore be used before generalizing the results to an unselected population. The prevalence of risk behaviors reported by the respondents relative to data from the 1999 YRBS¹⁸ also show several important differences Table 1. Our respondents appear more likely to drink, less likely to ride with someone who had been drinking, more likely to smoke, more likely to wear a seatbelt, less likely to fight, and less likely to report symptoms of depression compared with general data from the YRBS. The prevalence of other risk behaviors appears to be similar. It is uncertain whether these behavioral differences are particular to a military population. Another limitation of our study is the use of self-reported data, which potentially introduces a significant recall bias.

Conclusions

Our study suggests that in a young healthy military population there is widespread belief that creatine and androstenedione are effective ergogenic agents. The prevalence of supplement use in this population is significant, and such use is associated with certain health risk behaviors. This pattern of behaviors appears similar to a risk-behavior syndrome previously noted with anabolic steroid use. Clinicians should routinely ask adolescents and athletes about their use of dietary supplements. They should ask which supplements their patients are using, why they are using supplements, and what doses they are taking. Clinicians should be alert so they can provide targeted preventive counseling for individuals who are identified as users of ergogenic supplements.

Acknowledgments

Our study was supported by Uniformed Services Academy of Family Physicians grant G-1879.

Related resources

• FDA Dietary Supplement Consumer Information Homepage: http://www.cfsan.fda.gov/~dms/supplmnt.html
• USDA Food and Nutrition Information Center http://www.nalusda.gov/fnic/etext/000015.html
• Australian Academy of Sciences—Drugs in Sport http://www.science.org.au/nova/055/055sit.htm

Since ancient times, athletes have ingested substances to gain a competitive edge.¹ The use of such supplements, however, is not restricted to athletes. In 1996, Americans spent $6.5 billion on dietary supplements.² Individuals cite many reasons for using such supplements, including to ensure good nutrition, prevent illness, improve performance, ward off fatigue, and enhance personal appearance.³

Since 1998, interest in the ergogenic effects of products currently marketed as nutritional supplements, in particular creatine and androstenedione, has increased. Creatine is currently the most popular dietary ergogenic supplement.⁴ The reported benefits of creatine include increased energy during short-term intense exercise, increased muscle mass, increased strength, increased lean body mass, and decreased lactate accumulation during intense exercise. Although it is clear that supplementation raises intramuscular creatine stores,⁵ it remains unclear how effective creatine is as an ergogenic aid. Generally, it is felt that creatine supplementation may be useful for repeated bouts of high-intensity short-duration exercise.⁶ Claims of increased strength and muscle mass have not, however, been unequivocally proven.

Androstenedione is a steroid hormone. Many sporting communities, including the International Olympic Committee, have banned its use. By itself, androstenedione is weakly androgenic. To date, the largest controlled trial examining the effectiveness of androstenedione as an ergogenic aid showed no significant gains in muscular strength compared with a standard program of resistance training.⁷ Methodologic concerns have been raised about this particular study,⁸ because supplement manufacturers are not subject to United States Food and Drug Administration approval if no unsupported claims of efficacy are made. Therefore, sales of ergogenic supplements remain brisk.

Within this context, our study was designed to examine the prevalence of ergogenic supplement use in a young population. We also sought to determine the extent to which supplement use was associated with personal health risk behaviors and to determine how effective ergogenic supplements were perceived to be.

Methods

Sample Recruitment

Our study was conducted at Lackland Air Force Base, Texas, and Parris Island, South Carolina, from June through September, 1999. All men and women entering military basic training at these sites were eligible to participate. Individuals were randomly selected to receive the survey. There were no specific exclusion criteria.

Survey Administration

The Youth Risk Behavior Survey (YRBS)⁹ was modified for use in our study. Specifically, questions about nutritional ergogenic supplement use were added to the standard instrument that is used to examine health risk and preventive behaviors, such as tobacco use, alcohol use, seat belt use, helmet use, and suicide. The subjects were specifically asked to report use of creatine and androstenedione. They were also given an open-ended opportunity to report any other ergogenic supplement use.

Data Analysis

We used basic descriptive statistics for categorical variables. Proportions were compared using the chi-square test of contingency. The Fisher exact test was used for rare responses. Continuous variables were compared using the Student t test for independent samples. We calculated adjusted odds ratios with a multivariate logistic regression analysis controlling for sex, age, and body mass index.

Results

Of 550 individuals recruited for participation, 499 (91%) completed the survey. The respondents ranged in age from 17 to 35 years (mean age = 21 years). The study sample was 88% men (n=439) and 12% women (n=60). The lifetime prevalence of ergogenic supplement use in our study population was 41%. Creatine was the most commonly used ergogenic supplement (n=117; 23% of the study sample), followed by androstenedione (n=38; 8%).

Ergogenic supplement use was significantly associated with several health risk behaviors. Individuals who used ergogenic supplements were more likely to drink alcohol, more likely to drink heavily, more likely to ride in a vehicle with someone who had been drinking, more likely to drive after drinking, and more likely to have been in a physical fight compared with those who had not used supplements Table 1.

Men were more likely to use ergogenic nutritional supplements than women (P <.001). No difference in the pattern of supplement use was noted relative to age or body mass index. An equal number of respondents (70%) felt that creatine, androstenedione, and anabolic steroids were all effective at building muscle strength. Men, respondents younger than 20 years, and individuals who reported using ergogenic supplements were all more likely to believe in the effectiveness of creatine, androstenedione, and steroids compared with others (P <.05).

Discussion

Our study has 3 important findings: (1) a significant number of young people have used ergogenic supplements; (2) a similar majority of respondents believe creatine, androstenedione, and anabolic steroids are effective with respect to gains in muscle strength and mass; and (3) use of nutritional ergogenic supplements is associated with certain high-risk health behaviors.

Although it has been reported that 38% of high school athletes use vitamin supplements,¹⁰ the prevalence rates of creatine or androstenedione use in adolescents are not known. Our study suggests that a significant number of young persons use ergogenic supplements, creatine in particular. The self-reported prevalence rate of 41% of individuals who have used ergogenic supplements represents a baseline that needs to be verified by further studies.

A majority of respondents (70%) believed that creatine and androstenedione are effective in increasing muscle strength. There is limited evidence to support this in the medical literature. This is important given the credibility gap the medical community created when discussing anabolic steroids with athletes and adolescents in the past. Physicians routinely counseled that steroids were ineffective in building muscle strength and mass. Athletes’ experiences in the gym and on the playing field proved otherwise,¹¹ however, and a pervading sense of mistrust developed among athletes toward the medical community.¹² Our study shows that many young individuals believe that ergogenic supplements are effective. Some of this belief may stem from word of mouth,¹³ some from savvy marketing, and some from perceived experience. Whatever the case, physicians must be careful to provide accurate clinical information, while not alienating patients, when discussing ergogenic supplements. Our results also suggest that individuals who have used ergogenics are more likely to believe in their effectiveness. This agrees with the model suggesting that users of “natural” products are more likely to believe in their safety and efficacy than nonusers.¹⁴

Finally, and perhaps most important, our study indicates that supplement use is associated with certain health risk behaviors. Specifically, ergogenic supplement users are more likely to drink alcohol heavily, ride as a passenger with a driver who has been drinking, drive after drinking, and get involved in a physical fight. A similar pattern of behavior has been described among adolescents who use anabolic steroids.¹⁵ Although the prevalence of anabolic steroid use in the general adolescent population has been estimated at 3% to 10%,^16,17 our study shows a much higher rate of ergogenic supplement use. This establishes an important link in behavioral data between nutritional supplement users and anabolic steroid users. While steroids are illegal, ergogenic supplements are readily available. If, as our results suggest, ergogenic use is associated with a risk-behavior syndrome similar to that seen with steroid use, this is a valuable marker for physicians to use to provide targeted preventive counseling.

Limitations

Our study, however, has several important limitations. The respondents were all military recruits. Caution must therefore be used before generalizing the results to an unselected population. The prevalence of risk behaviors reported by the respondents relative to data from the 1999 YRBS¹⁸ also show several important differences Table 1. Our respondents appear more likely to drink, less likely to ride with someone who had been drinking, more likely to smoke, more likely to wear a seatbelt, less likely to fight, and less likely to report symptoms of depression compared with general data from the YRBS. The prevalence of other risk behaviors appears to be similar. It is uncertain whether these behavioral differences are particular to a military population. Another limitation of our study is the use of self-reported data, which potentially introduces a significant recall bias.

Conclusions

Our study suggests that in a young healthy military population there is widespread belief that creatine and androstenedione are effective ergogenic agents. The prevalence of supplement use in this population is significant, and such use is associated with certain health risk behaviors. This pattern of behaviors appears similar to a risk-behavior syndrome previously noted with anabolic steroid use. Clinicians should routinely ask adolescents and athletes about their use of dietary supplements. They should ask which supplements their patients are using, why they are using supplements, and what doses they are taking. Clinicians should be alert so they can provide targeted preventive counseling for individuals who are identified as users of ergogenic supplements.

Acknowledgments

Our study was supported by Uniformed Services Academy of Family Physicians grant G-1879.

Related resources

• FDA Dietary Supplement Consumer Information Homepage: http://www.cfsan.fda.gov/~dms/supplmnt.html
• USDA Food and Nutrition Information Center http://www.nalusda.gov/fnic/etext/000015.html
• Australian Academy of Sciences—Drugs in Sport http://www.science.org.au/nova/055/055sit.htm