Original Research

Research using self-developed questionnaires is a popular study design in family practice and is frequently used for gathering data on knowledge, beliefs, attitudes, and behaviors. A Medline literature search from 1966 to 2000 identified 53,101 articles related to questionnaires, of which 2088 were directly related to family practice. Despite the large number of questionnaire-related articles, however, only 2 in the general medical literature^1,2 and 1 in the family practice literature³ were directly related to research methodology.

To obtain guidance on survey research methodology, novice family practice researchers often must go through volumes of information by specialists in other disciplines. For example, a search of a psychology database (PsychInfo)⁴ from 1966 to 2000 produced 45 articles about questionnaire methodology. The goal of this article is to synthesize pertinent survey research methodology tenets-from other disciplines as well as from family practice-in a manner that is meaningful to novice family practice researchers as well as to research consumers. This article is not aimed at answering all questions, but rather is meant to serve as a general guideline for those with little formal research training who seek guidance in developing and administering questionnaires.

Avoiding common pitfalls in survey research

Although constructing a questionnaire is not exceedingly complex, simple mistakes can be avoided by following some basic rules and guidelines. The Figure is a checklist for conducting a survey research project that combines guidelines and suggestions from published survey research literature,^5-9 and the cumulative experience of the authors. Two of the authors (M.J.D. and K.C.O.) are experienced survey researchers who have published, in peer-reviewed journals, numerous studies that used questionnaires.^10-19 One of the authors (MJD) has been teaching research to residents and junior faculty for over a decade, and has been an advisor on scores of resident, student, and faculty research projects. The perspective of the novice researcher is represented by 1 author (C.R.W.).

Getting started

The “quick and dirty” approach is perhaps the most common pitfall in survey research. Because of the ease of administration and the relatively low cost of survey research, questionnaires can be developed and administered quickly. The researcher, however, should be sure to consider whether or not a survey is the most appropriate method to answer a research question. Adequate time must be given to thoroughly searching the relevant literature, developing and focusing on an appropriate research question, and defining the target population for the study (see Figure A, Getting Started). Large, multisite surveys are more likely to be generalizeable and to be published in peer-reviewed journals.

One way to avoid undertaking a project too rapidly and giving inadequate attention to the survey research process is for novice researchers to avoid independent research. Those with little or no experience must realize that researchers in both family practice and other fields perform research in teams, with the various participants bringing specific skills to the process.²⁰ Oversights, mistakes, and biases in the design of questionnaires can always occur, whether a researcher is working independently or as a member of a team. It seems reasonable to assume, however, that significant problems are much less likely to occur when a multidisciplinary team approach is involved rather than an individual researcher undertaking a study independently.

Ideally, a research team should include a statistician, a professional with experience in the content areas of the study, and a senior investigator.²¹ The desirable area of expertise, however, is often not readily available to family physicians, especially those in community-based settings. Individuals with some training in research who are interested in being involved can usually be found in colleges and universities, hospitals, and at the local public health department. Psychologists, sociologists, health services researchers, public health epidemiologists, and nursing educators are all potential resources and possible collaborators. Establishing the necessary relationships to form an ad hoc research team is certainly more time and labor intensive than undertaking research independently, but generally results in the collection of more useful information.

Novices should consult survey methodology books before and during the study.^5-9 Excellent resources are available that provide a comprehensive overview of survey methods,²² means for improving response rates,²³ and methods for constructing relatively brief but thorough survey questions.⁵ Academic family practice fellowships often provide training in survey methodology. In addition, many family practice researchers respond favorably to requests for information or advice requested by telephone or email contact. The novice author of this article reports excellent success in contacting experts in this manner. With the advent of the Internet, a “cyberspace” team comprised of experts in the topic and the methodology is a reasonable and helpful option for the novice.

Survey content and structure

Novice researchers often assume that developing a questionnaire is an intuitive process attainable by virtually anyone, regardless of their level of research training. While it is true that questionnaires are relatively simple to construct, developing an instrument that is valid and reliable is not intuitive. An instrument is valid if it actually measures what we think it is measuring, and it is reliable if it measures the phenomenon consistently in repeated applications.²⁴ By following a few basic guidelines, those with limited research training can develop survey instruments capable of producing valid and reliable information. The 3 primary concerns for developing appropriate questions (items) are: (1) response format; (2) content; and (3) wording and placement (see Figure B, Survey Questions; and Figure C, Designing and Formatting the Survey).

Format

Questionnaires generally use a closed-ended format rather than an open-ended format. Closed formats spell out response options instead of asking study subjects to respond in their own words. Although there are many reasons for using closed formats, their primary advantages over open formats is that they are more specific and provide the same frame of reference to all respondents, and they allow quantitative analysis. A disadvantage is that they limit the possible range of responses envisioned by the investigators. Questionnaires with closed formats are therefore not as helpful as qualitative methods in the early, exploratory phases of a research project.

Closed-ended items can be formatted into several different categories (classes) of measurement, based on the relationship of the response categories to one another. Nominal measurements are responses that are sorted into unordered categories, such as demographic variables (ie, sex, ethnicity). Ordinal measurements are similar to nominal, except that there is a definite order to the categories. For example, ordinal items may ask respondents to rank their preferences among a list of options from the least desirable to the most desirable.

Survey items that ask for respondents(delete apostrophe) to rank order preferences are often a more useful than items that state, “check all that apply.” While checking all relevant responses may be necessary for certain items, such questions often lose valuable information as they can only supply raw percentages without supplying any comparison between responses. If a survey uses a rank order response, it enables determining the relative importance of the different categories during data analysis Table 1.

Two additional tools used on questionnaires are continuous variables and scales. Continuous variables can be simple counts (eg, the number of times something occurred) or physical attributes (eg, age or weight). A general rule when collecting information on continuous variables is to avoid obtaining the information in ranges of categories unless absolutely necessary. Response categories that reflect ranges of responses can always be constructed after the information is gathered, but if the information is gathered in ranges from the start, it cannot later be expanded to reflect specific values.

Scales are used by survey researchers to assess the intensity of respondents’ attitudes about a specific issue or issues. Likert scales are probably the best known and most widely used for measuring attitudes. These scales typically present respondents with a statement and ask them to indicate whether they “strongly agree,” “agree,” “neither agree nor disagree,” “disagree,” or “strongly disagree.” The wording of the response categories can be changed to reflect other concepts (eg, approval or disapproval), and the standard 5-response format can be expanded or abbreviated if necessary.

There are no hard and fast rules for determining the number of response categories to use for scaled items, or whether to use a neutral category or one that reflects uncertainty. Research indicates that the reliability of respondents’ ratings declines when using more than 9 rating scale points.²⁵ However, the reliability of a scale increases when the number of rating scale points is increased, with maximum benefit achieved with 5 or 7 scale points.^25,26 Since the objective of using scales is to gauge respondent’s preferences, it is sometimes argued that a middle point or category of uncertainty category should not be used. Odd-numbered rating scales, however, conform better with the underlying tenets of many statistical tests, suggesting the need for including this category.²⁹ As the number of rating scale points increases, respondents’ use of the midpoint category decreases substantially. ³⁰ Thus, based on the available literature, it is generally advisable to use between 5 and 7 response categories and an uncertainty category, unless there is a compelling reason to force respondents to choose between 2 competing perspectives or alternatives.

Content

Items should not be included on questionnaires when the only justification for inclusion is that the investigator feels the information “would be really interesting to know.” Rather, for each item, you should ask yourself how it addresses the study’s research question and how it will be used in the data analysis stage of the study. Researchers should develop a data analysis plan in advance of administering a questionnaire to determine exactly how each question will be used in the analysis. When the relationship between a particular item and the study’s research question is unclear, or it is not known how an item will be used in the analysis, the item should be removed from the questionnaire.

Wording and placement

The wording of questions should be kept simple, regardless of the education level of the respondents. Questions should be kept as short and direct as possible since shorter surveys tend to have higher response rates.^31,32 Each question should be scrutinized to ensure it is appropriate for the respondents and does not require or assume an inappropriate level of knowledge about a topic. Since first impressions are important for setting the tone of a questionnaire, never begin with sensitive or threatening questions.³³ Questionnaires should begin with simple, introductory (“warm-up”)“questions to help establish trust and an appropriate frame of mind for respondents.³⁴ Other successful strategies are: (1) when addressing multiple topics, insert an introductory statement immediately preceding each topic (eg, “In the next section we would like to ask you about …”); (2) request demographic information at the end of the questionnaire; and (3) always provide explicit instructions to avoid any confusion on the part of respondents.³⁵

Additional, clear information on survey content and structure is available in 2 books from Sage Publications.^5,36 By following simple guidelines and common sense, most family practice researchers can construct valid and reliable questionnaires. As a final safeguard, once a final draft of the questionnaire is completed, the researcher should always be the first respondent. By placing yourself in the respondent’s role and taking the time to think about and respond to each question, problems with the instrument that were overlooked are sometimes identified.

Analyzing surveys

It is not within the scope of this project to address statistical analysis of survey data. Before attempting data analysis, investigators should receive appropriate training or consult with a qualified professional. There are 3 topics that can and should be understood by novice researchers related to data analysis (Figure D, Developing a Framework for Analysis).

Coding

Before analyzing survey data it is necessary to assign numbers (codes) to the responses obtained. Since the computer program that is used for analyzing data does not know what the numbers mean, the researcher assigns meaning to the codes so that the results can be interpreted correctly. Coding refers to the process of developing the codes, assigning them to responses, and documenting the decision rules used for assigning specific codes to specific response categories. For example, almost all questionnaires contain missing values when respondents elect to not answer an item. Unique codes need to be assigned to distinguish between an item’s missing values, items that may not be applicable to a particular respondent, and responses that have a “none” or “no opinion” category.

Data can be entered into appropriate data files once codes have been assigned to responses and a codebook compiled that defines the codes and their corresponding response categories. It is important to ensure that the data are free of errors (are clean) prior to performing data analysis. Although many methods can be used for data cleaning (ie, data can be entered twice and results compared consistency), at a minimum all of the codes should be checked to ensure only legitimate codes appear.

Frequency distributions are tables produced by statistical software that display the number of respondents in each response category for each item (variable) used in the analysis. By carefully examining frequency tables, the researcher can check for illegitimate codes. Frequency tables also display the relative distribution of responses and allow identification of items that do not conform to expectations given what is known about the study population.

Sample size

Since it is usually not possible to study all of the members of the group (population) of interest in a study, a subset (sample) of the population is generally selected for study from the sampling frame. Sampling is the process by which study subjects are selected from the target population, while the sample frame is the members of a population who have a chance of being included in the survey. In probability samples, each member of the sampling frame has a known probability of being selected for the study, whereas in nonprobability samples, the probability of selection is unknown. When a high degree of precision in sampling is needed to unambiguously identify the magnitude of a problem in a population or the factors that cause the problem, then probability sampling techniques must be used.

When conducting an analytical study that examines precisely whether statistically significant differences exist between groups in a population, power analysis is used to determine what size sample is needed to detect the differences. Estimates of sample size based on power are inversely related to the expected size of the differences “(effect size)”-that is, detecting smaller differences requires a larger sample. If an analytical study is undertaken to determine the magnitude of the differences between 2 groups, it is necessary to work with a statistician or other methodology expert to perform the appropriate power analysis. For a basic but valuable description of sample size estimation, see chapter 13 of Hulley and Cummings.²¹

In contrast to analytical studies, exploratory and descriptive studies can frequently be conducted without the need for a power analysis. While some descriptive studies may require the use of probability techniques and precise sample estimates, this often is not the case for studies that establish the existence of a problem or estimating its dimensions. When conducting an exploratory or descriptive study using a survey design and a nonprobability sampling technique, considerations other than effect size or precision are used to determine sample size. For example, the availability of eligible respondents, limitations of time and resources, and the need for pilot study data can all contribute to selecting a nonprobability sample. When these types of sampling techniques are used, however, it is important to remember that the validity and reliability of the findings are not assured, and the findings cannot be used to demonstrate the existence of differences between groups. The findings of these types of studies are only suggestive and have limited application beyond the specific study setting.

Response rate

The response rate is a measure indicating the percentage of the identified sample that completed and returned the questionnaire. It is calculated by dividing the number of completed questionnaires by the total sample size identified for the study. For example, if a study is mailed to 500 physicians questionnaires and 100 returned a completed questionnaire, the response rate would be 20% (100/500).

The response rate for mailed questionnaires is extremely variable. Charities are generally content with a 1% to 3% response rate, the US Census Bureau expects to achieve a 99% rate, and among the general population, a 10% response rate is not uncommon. Although an 80% response rate is possible from an extremely motivated population, a rate of 70% is generally considered excellent.³⁴

The effect of nonresponse on the results of a survey depend on the degree to which those not responding are systematically different from the population from which they are drawn.²⁴ When the response rate is high (ie, 95%), the results obtained from the sample will likely provide accurate information about the target population (sampling frame) even if the nonrespondents are distinctly different. However, if nonrespondents differ in a systematic way from the target population and the response rate is low, bias in how much the survey results accurately reflect the true characteristics of the target population is likely.

When calculating the response rate, participants who have died or retired can be removed from the denominator as appropriate. Nonrespondents, however, who refuse to participate, do not return the survey, or have moved should be included. This bias tends to be more problematic in “sensitive” areas of research³⁷ than in studies of common, nonthreatening topics.³⁸ Imputing values for missing data from nonrespondents is complex and generally should not be undertaken.³⁹

Given the importance of response rate, every effort must be made to obtain as many completed questionnaires as possible and strategies to maximize the response rate should be integrated into the study design (see Dillman²³ for a useful discussion of successful strategies). Some simple means for improving response rates include constructing a short questionnaire, sending a well-written and personalized cover letter containing your signature, and emphasizing the importance of the study and the confidentiality of responses. It is also advisable to include a self-addressed, stamped envelope for return responses, and sometimes a small incentive is worthwhile. The National Center for Education Statistics notes that all surveys require some follow-up to achieve desirable response rates.⁴⁰ Survey researchers, therefore, should develop procedures for monitoring responses and implement follow-up plans shortly after the survey begins.

Generally, 2 or 3 mailings are used to maximize response rates. Use of post card reminders is an inexpensive, though untested, method to increase response. Several randomized studies have reported an increase in response rate from physicians in private practice with the use of monetary incentives, although the optimum amount is debated. Everett et al⁴⁰ compared the use of a $1 incentive vs no monetary incentive and found a significant increase with the incentive group (response rates: 63% in the $1 group; 45% in the no incentive group; P < .0001).⁴¹ Other studies have compared $2, $5, $10, $20, and $25 incentives and found that $2 or $5 incentives are most cost effective.⁴²⁴⁵ Similar findings have been reported for physician surveys in other countries.^31,46 In an assessment of incentive for enrollees in a health plan, a $2 incentive was more cost effective than a $5 incentive.⁴⁷ A $1 incentive was as effective as $2 in significantly increasing response rate in a low-income population.⁴⁸ Quality of responses have not varied by use of incentives and there does not appear to be an incentive-bias.

Use of lottery appears to also increase response rate in both physicians and the lay public, although there are no studies comparing lottery to a monetary incentive enclosed for all participants.^31,49 Use of either certified or priority return mail appears to increase response rates, and may be more cost effective when used for the second mailing.^45,48

Pilot testing

Though pilot testing is generally included in the development of a survey, it is often inadequately conducted Figure F Final Preparation). Frequently, investigators are eager to answer their research question and pilot testing is synonymous with letting a few colleagues take a quick look and make a few comments. Table 2 illustrates a problem that could have been avoided with proper pilot testing.¹⁰ One of the questions in the survey asked about how time is allotted for faculty to pursue scholarly activities and research (Format A). Unfortunately, the question mixes 2 types of time in 1 question: extended time away from the institution (sabbatical and mini-sabbatical) and time in the routine schedule. This was confusing to respondents and could have been avoided by separating the content into 2 separate questions (Format B).

Investigators should consider carefully whom to include in the pilot testing. Not only should this include the project team and survey “experts”, but it should also include a sample of the target audience. Pilot testing among multiple groups provides feedback about the wording and clarity of questions, appropriateness of the questions for the target population, and the presence of redundant or unnecessary items.

Conclusions

One of the authors (C.R.W.) recently worked on her first questionnaire project. Among the many lessons she learned was the value of a team in providing assistance, the importance of considering if the time spent on a particular activity makes it cost effective, and the need to be flexible depending on circumstances. She found that establishing good communication with the team cuts down on errors and wasted effort. Rewarding the team for all of their hard work improves morale and provides a positive model for future projects.

The mailed self-administered questionnaire is an important tool in primary care research. For family practice to continue its maturation as a research discipline, family practitioners need to be conversant in survey methodology and familiar with its pitfalls. We hope this primer-designed specifically for use in the family practice setting-will provide not only basic guidelines for novices but will also inspire further investigation.

Acknowledgments

The authors thank Laura Snell, MPH, for her thoughtful review of the manuscript. We also thank Olive Chen, PhD, for research assistance and Janice Rookstool for manuscript preparation.

Research using self-developed questionnaires is a popular study design in family practice and is frequently used for gathering data on knowledge, beliefs, attitudes, and behaviors. A Medline literature search from 1966 to 2000 identified 53,101 articles related to questionnaires, of which 2088 were directly related to family practice. Despite the large number of questionnaire-related articles, however, only 2 in the general medical literature^1,2 and 1 in the family practice literature³ were directly related to research methodology.

To obtain guidance on survey research methodology, novice family practice researchers often must go through volumes of information by specialists in other disciplines. For example, a search of a psychology database (PsychInfo)⁴ from 1966 to 2000 produced 45 articles about questionnaire methodology. The goal of this article is to synthesize pertinent survey research methodology tenets-from other disciplines as well as from family practice-in a manner that is meaningful to novice family practice researchers as well as to research consumers. This article is not aimed at answering all questions, but rather is meant to serve as a general guideline for those with little formal research training who seek guidance in developing and administering questionnaires.

Avoiding common pitfalls in survey research

Although constructing a questionnaire is not exceedingly complex, simple mistakes can be avoided by following some basic rules and guidelines. The Figure is a checklist for conducting a survey research project that combines guidelines and suggestions from published survey research literature,^5-9 and the cumulative experience of the authors. Two of the authors (M.J.D. and K.C.O.) are experienced survey researchers who have published, in peer-reviewed journals, numerous studies that used questionnaires.^10-19 One of the authors (MJD) has been teaching research to residents and junior faculty for over a decade, and has been an advisor on scores of resident, student, and faculty research projects. The perspective of the novice researcher is represented by 1 author (C.R.W.).

Getting started

The “quick and dirty” approach is perhaps the most common pitfall in survey research. Because of the ease of administration and the relatively low cost of survey research, questionnaires can be developed and administered quickly. The researcher, however, should be sure to consider whether or not a survey is the most appropriate method to answer a research question. Adequate time must be given to thoroughly searching the relevant literature, developing and focusing on an appropriate research question, and defining the target population for the study (see Figure A, Getting Started). Large, multisite surveys are more likely to be generalizeable and to be published in peer-reviewed journals.

One way to avoid undertaking a project too rapidly and giving inadequate attention to the survey research process is for novice researchers to avoid independent research. Those with little or no experience must realize that researchers in both family practice and other fields perform research in teams, with the various participants bringing specific skills to the process.²⁰ Oversights, mistakes, and biases in the design of questionnaires can always occur, whether a researcher is working independently or as a member of a team. It seems reasonable to assume, however, that significant problems are much less likely to occur when a multidisciplinary team approach is involved rather than an individual researcher undertaking a study independently.

Ideally, a research team should include a statistician, a professional with experience in the content areas of the study, and a senior investigator.²¹ The desirable area of expertise, however, is often not readily available to family physicians, especially those in community-based settings. Individuals with some training in research who are interested in being involved can usually be found in colleges and universities, hospitals, and at the local public health department. Psychologists, sociologists, health services researchers, public health epidemiologists, and nursing educators are all potential resources and possible collaborators. Establishing the necessary relationships to form an ad hoc research team is certainly more time and labor intensive than undertaking research independently, but generally results in the collection of more useful information.

Novices should consult survey methodology books before and during the study.^5-9 Excellent resources are available that provide a comprehensive overview of survey methods,²² means for improving response rates,²³ and methods for constructing relatively brief but thorough survey questions.⁵ Academic family practice fellowships often provide training in survey methodology. In addition, many family practice researchers respond favorably to requests for information or advice requested by telephone or email contact. The novice author of this article reports excellent success in contacting experts in this manner. With the advent of the Internet, a “cyberspace” team comprised of experts in the topic and the methodology is a reasonable and helpful option for the novice.

Survey content and structure

Novice researchers often assume that developing a questionnaire is an intuitive process attainable by virtually anyone, regardless of their level of research training. While it is true that questionnaires are relatively simple to construct, developing an instrument that is valid and reliable is not intuitive. An instrument is valid if it actually measures what we think it is measuring, and it is reliable if it measures the phenomenon consistently in repeated applications.²⁴ By following a few basic guidelines, those with limited research training can develop survey instruments capable of producing valid and reliable information. The 3 primary concerns for developing appropriate questions (items) are: (1) response format; (2) content; and (3) wording and placement (see Figure B, Survey Questions; and Figure C, Designing and Formatting the Survey).

Format

Questionnaires generally use a closed-ended format rather than an open-ended format. Closed formats spell out response options instead of asking study subjects to respond in their own words. Although there are many reasons for using closed formats, their primary advantages over open formats is that they are more specific and provide the same frame of reference to all respondents, and they allow quantitative analysis. A disadvantage is that they limit the possible range of responses envisioned by the investigators. Questionnaires with closed formats are therefore not as helpful as qualitative methods in the early, exploratory phases of a research project.

Closed-ended items can be formatted into several different categories (classes) of measurement, based on the relationship of the response categories to one another. Nominal measurements are responses that are sorted into unordered categories, such as demographic variables (ie, sex, ethnicity). Ordinal measurements are similar to nominal, except that there is a definite order to the categories. For example, ordinal items may ask respondents to rank their preferences among a list of options from the least desirable to the most desirable.

Survey items that ask for respondents(delete apostrophe) to rank order preferences are often a more useful than items that state, “check all that apply.” While checking all relevant responses may be necessary for certain items, such questions often lose valuable information as they can only supply raw percentages without supplying any comparison between responses. If a survey uses a rank order response, it enables determining the relative importance of the different categories during data analysis Table 1.

Two additional tools used on questionnaires are continuous variables and scales. Continuous variables can be simple counts (eg, the number of times something occurred) or physical attributes (eg, age or weight). A general rule when collecting information on continuous variables is to avoid obtaining the information in ranges of categories unless absolutely necessary. Response categories that reflect ranges of responses can always be constructed after the information is gathered, but if the information is gathered in ranges from the start, it cannot later be expanded to reflect specific values.

Scales are used by survey researchers to assess the intensity of respondents’ attitudes about a specific issue or issues. Likert scales are probably the best known and most widely used for measuring attitudes. These scales typically present respondents with a statement and ask them to indicate whether they “strongly agree,” “agree,” “neither agree nor disagree,” “disagree,” or “strongly disagree.” The wording of the response categories can be changed to reflect other concepts (eg, approval or disapproval), and the standard 5-response format can be expanded or abbreviated if necessary.

There are no hard and fast rules for determining the number of response categories to use for scaled items, or whether to use a neutral category or one that reflects uncertainty. Research indicates that the reliability of respondents’ ratings declines when using more than 9 rating scale points.²⁵ However, the reliability of a scale increases when the number of rating scale points is increased, with maximum benefit achieved with 5 or 7 scale points.^25,26 Since the objective of using scales is to gauge respondent’s preferences, it is sometimes argued that a middle point or category of uncertainty category should not be used. Odd-numbered rating scales, however, conform better with the underlying tenets of many statistical tests, suggesting the need for including this category.²⁹ As the number of rating scale points increases, respondents’ use of the midpoint category decreases substantially. ³⁰ Thus, based on the available literature, it is generally advisable to use between 5 and 7 response categories and an uncertainty category, unless there is a compelling reason to force respondents to choose between 2 competing perspectives or alternatives.

Content

Items should not be included on questionnaires when the only justification for inclusion is that the investigator feels the information “would be really interesting to know.” Rather, for each item, you should ask yourself how it addresses the study’s research question and how it will be used in the data analysis stage of the study. Researchers should develop a data analysis plan in advance of administering a questionnaire to determine exactly how each question will be used in the analysis. When the relationship between a particular item and the study’s research question is unclear, or it is not known how an item will be used in the analysis, the item should be removed from the questionnaire.

Wording and placement

The wording of questions should be kept simple, regardless of the education level of the respondents. Questions should be kept as short and direct as possible since shorter surveys tend to have higher response rates.^31,32 Each question should be scrutinized to ensure it is appropriate for the respondents and does not require or assume an inappropriate level of knowledge about a topic. Since first impressions are important for setting the tone of a questionnaire, never begin with sensitive or threatening questions.³³ Questionnaires should begin with simple, introductory (“warm-up”)“questions to help establish trust and an appropriate frame of mind for respondents.³⁴ Other successful strategies are: (1) when addressing multiple topics, insert an introductory statement immediately preceding each topic (eg, “In the next section we would like to ask you about …”); (2) request demographic information at the end of the questionnaire; and (3) always provide explicit instructions to avoid any confusion on the part of respondents.³⁵

Additional, clear information on survey content and structure is available in 2 books from Sage Publications.^5,36 By following simple guidelines and common sense, most family practice researchers can construct valid and reliable questionnaires. As a final safeguard, once a final draft of the questionnaire is completed, the researcher should always be the first respondent. By placing yourself in the respondent’s role and taking the time to think about and respond to each question, problems with the instrument that were overlooked are sometimes identified.

Analyzing surveys

It is not within the scope of this project to address statistical analysis of survey data. Before attempting data analysis, investigators should receive appropriate training or consult with a qualified professional. There are 3 topics that can and should be understood by novice researchers related to data analysis (Figure D, Developing a Framework for Analysis).

Coding

Before analyzing survey data it is necessary to assign numbers (codes) to the responses obtained. Since the computer program that is used for analyzing data does not know what the numbers mean, the researcher assigns meaning to the codes so that the results can be interpreted correctly. Coding refers to the process of developing the codes, assigning them to responses, and documenting the decision rules used for assigning specific codes to specific response categories. For example, almost all questionnaires contain missing values when respondents elect to not answer an item. Unique codes need to be assigned to distinguish between an item’s missing values, items that may not be applicable to a particular respondent, and responses that have a “none” or “no opinion” category.

Data can be entered into appropriate data files once codes have been assigned to responses and a codebook compiled that defines the codes and their corresponding response categories. It is important to ensure that the data are free of errors (are clean) prior to performing data analysis. Although many methods can be used for data cleaning (ie, data can be entered twice and results compared consistency), at a minimum all of the codes should be checked to ensure only legitimate codes appear.

Frequency distributions are tables produced by statistical software that display the number of respondents in each response category for each item (variable) used in the analysis. By carefully examining frequency tables, the researcher can check for illegitimate codes. Frequency tables also display the relative distribution of responses and allow identification of items that do not conform to expectations given what is known about the study population.

Sample size

Since it is usually not possible to study all of the members of the group (population) of interest in a study, a subset (sample) of the population is generally selected for study from the sampling frame. Sampling is the process by which study subjects are selected from the target population, while the sample frame is the members of a population who have a chance of being included in the survey. In probability samples, each member of the sampling frame has a known probability of being selected for the study, whereas in nonprobability samples, the probability of selection is unknown. When a high degree of precision in sampling is needed to unambiguously identify the magnitude of a problem in a population or the factors that cause the problem, then probability sampling techniques must be used.

When conducting an analytical study that examines precisely whether statistically significant differences exist between groups in a population, power analysis is used to determine what size sample is needed to detect the differences. Estimates of sample size based on power are inversely related to the expected size of the differences “(effect size)”-that is, detecting smaller differences requires a larger sample. If an analytical study is undertaken to determine the magnitude of the differences between 2 groups, it is necessary to work with a statistician or other methodology expert to perform the appropriate power analysis. For a basic but valuable description of sample size estimation, see chapter 13 of Hulley and Cummings.²¹

In contrast to analytical studies, exploratory and descriptive studies can frequently be conducted without the need for a power analysis. While some descriptive studies may require the use of probability techniques and precise sample estimates, this often is not the case for studies that establish the existence of a problem or estimating its dimensions. When conducting an exploratory or descriptive study using a survey design and a nonprobability sampling technique, considerations other than effect size or precision are used to determine sample size. For example, the availability of eligible respondents, limitations of time and resources, and the need for pilot study data can all contribute to selecting a nonprobability sample. When these types of sampling techniques are used, however, it is important to remember that the validity and reliability of the findings are not assured, and the findings cannot be used to demonstrate the existence of differences between groups. The findings of these types of studies are only suggestive and have limited application beyond the specific study setting.

Response rate

The response rate is a measure indicating the percentage of the identified sample that completed and returned the questionnaire. It is calculated by dividing the number of completed questionnaires by the total sample size identified for the study. For example, if a study is mailed to 500 physicians questionnaires and 100 returned a completed questionnaire, the response rate would be 20% (100/500).

The response rate for mailed questionnaires is extremely variable. Charities are generally content with a 1% to 3% response rate, the US Census Bureau expects to achieve a 99% rate, and among the general population, a 10% response rate is not uncommon. Although an 80% response rate is possible from an extremely motivated population, a rate of 70% is generally considered excellent.³⁴

The effect of nonresponse on the results of a survey depend on the degree to which those not responding are systematically different from the population from which they are drawn.²⁴ When the response rate is high (ie, 95%), the results obtained from the sample will likely provide accurate information about the target population (sampling frame) even if the nonrespondents are distinctly different. However, if nonrespondents differ in a systematic way from the target population and the response rate is low, bias in how much the survey results accurately reflect the true characteristics of the target population is likely.

When calculating the response rate, participants who have died or retired can be removed from the denominator as appropriate. Nonrespondents, however, who refuse to participate, do not return the survey, or have moved should be included. This bias tends to be more problematic in “sensitive” areas of research³⁷ than in studies of common, nonthreatening topics.³⁸ Imputing values for missing data from nonrespondents is complex and generally should not be undertaken.³⁹

Given the importance of response rate, every effort must be made to obtain as many completed questionnaires as possible and strategies to maximize the response rate should be integrated into the study design (see Dillman²³ for a useful discussion of successful strategies). Some simple means for improving response rates include constructing a short questionnaire, sending a well-written and personalized cover letter containing your signature, and emphasizing the importance of the study and the confidentiality of responses. It is also advisable to include a self-addressed, stamped envelope for return responses, and sometimes a small incentive is worthwhile. The National Center for Education Statistics notes that all surveys require some follow-up to achieve desirable response rates.⁴⁰ Survey researchers, therefore, should develop procedures for monitoring responses and implement follow-up plans shortly after the survey begins.

Generally, 2 or 3 mailings are used to maximize response rates. Use of post card reminders is an inexpensive, though untested, method to increase response. Several randomized studies have reported an increase in response rate from physicians in private practice with the use of monetary incentives, although the optimum amount is debated. Everett et al⁴⁰ compared the use of a $1 incentive vs no monetary incentive and found a significant increase with the incentive group (response rates: 63% in the $1 group; 45% in the no incentive group; P < .0001).⁴¹ Other studies have compared $2, $5, $10, $20, and $25 incentives and found that $2 or $5 incentives are most cost effective.⁴²⁴⁵ Similar findings have been reported for physician surveys in other countries.^31,46 In an assessment of incentive for enrollees in a health plan, a $2 incentive was more cost effective than a $5 incentive.⁴⁷ A $1 incentive was as effective as $2 in significantly increasing response rate in a low-income population.⁴⁸ Quality of responses have not varied by use of incentives and there does not appear to be an incentive-bias.

Use of lottery appears to also increase response rate in both physicians and the lay public, although there are no studies comparing lottery to a monetary incentive enclosed for all participants.^31,49 Use of either certified or priority return mail appears to increase response rates, and may be more cost effective when used for the second mailing.^45,48

Pilot testing

Though pilot testing is generally included in the development of a survey, it is often inadequately conducted Figure F Final Preparation). Frequently, investigators are eager to answer their research question and pilot testing is synonymous with letting a few colleagues take a quick look and make a few comments. Table 2 illustrates a problem that could have been avoided with proper pilot testing.¹⁰ One of the questions in the survey asked about how time is allotted for faculty to pursue scholarly activities and research (Format A). Unfortunately, the question mixes 2 types of time in 1 question: extended time away from the institution (sabbatical and mini-sabbatical) and time in the routine schedule. This was confusing to respondents and could have been avoided by separating the content into 2 separate questions (Format B).

Investigators should consider carefully whom to include in the pilot testing. Not only should this include the project team and survey “experts”, but it should also include a sample of the target audience. Pilot testing among multiple groups provides feedback about the wording and clarity of questions, appropriateness of the questions for the target population, and the presence of redundant or unnecessary items.

Conclusions

One of the authors (C.R.W.) recently worked on her first questionnaire project. Among the many lessons she learned was the value of a team in providing assistance, the importance of considering if the time spent on a particular activity makes it cost effective, and the need to be flexible depending on circumstances. She found that establishing good communication with the team cuts down on errors and wasted effort. Rewarding the team for all of their hard work improves morale and provides a positive model for future projects.

The mailed self-administered questionnaire is an important tool in primary care research. For family practice to continue its maturation as a research discipline, family practitioners need to be conversant in survey methodology and familiar with its pitfalls. We hope this primer-designed specifically for use in the family practice setting-will provide not only basic guidelines for novices but will also inspire further investigation.

Acknowledgments

The authors thank Laura Snell, MPH, for her thoughtful review of the manuscript. We also thank Olive Chen, PhD, for research assistance and Janice Rookstool for manuscript preparation.

Research using self-developed questionnaires is a popular study design in family practice and is frequently used for gathering data on knowledge, beliefs, attitudes, and behaviors. A Medline literature search from 1966 to 2000 identified 53,101 articles related to questionnaires, of which 2088 were directly related to family practice. Despite the large number of questionnaire-related articles, however, only 2 in the general medical literature^1,2 and 1 in the family practice literature³ were directly related to research methodology.

To obtain guidance on survey research methodology, novice family practice researchers often must go through volumes of information by specialists in other disciplines. For example, a search of a psychology database (PsychInfo)⁴ from 1966 to 2000 produced 45 articles about questionnaire methodology. The goal of this article is to synthesize pertinent survey research methodology tenets-from other disciplines as well as from family practice-in a manner that is meaningful to novice family practice researchers as well as to research consumers. This article is not aimed at answering all questions, but rather is meant to serve as a general guideline for those with little formal research training who seek guidance in developing and administering questionnaires.

Avoiding common pitfalls in survey research

Although constructing a questionnaire is not exceedingly complex, simple mistakes can be avoided by following some basic rules and guidelines. The Figure is a checklist for conducting a survey research project that combines guidelines and suggestions from published survey research literature,^5-9 and the cumulative experience of the authors. Two of the authors (M.J.D. and K.C.O.) are experienced survey researchers who have published, in peer-reviewed journals, numerous studies that used questionnaires.^10-19 One of the authors (MJD) has been teaching research to residents and junior faculty for over a decade, and has been an advisor on scores of resident, student, and faculty research projects. The perspective of the novice researcher is represented by 1 author (C.R.W.).

Getting started

The “quick and dirty” approach is perhaps the most common pitfall in survey research. Because of the ease of administration and the relatively low cost of survey research, questionnaires can be developed and administered quickly. The researcher, however, should be sure to consider whether or not a survey is the most appropriate method to answer a research question. Adequate time must be given to thoroughly searching the relevant literature, developing and focusing on an appropriate research question, and defining the target population for the study (see Figure A, Getting Started). Large, multisite surveys are more likely to be generalizeable and to be published in peer-reviewed journals.

One way to avoid undertaking a project too rapidly and giving inadequate attention to the survey research process is for novice researchers to avoid independent research. Those with little or no experience must realize that researchers in both family practice and other fields perform research in teams, with the various participants bringing specific skills to the process.²⁰ Oversights, mistakes, and biases in the design of questionnaires can always occur, whether a researcher is working independently or as a member of a team. It seems reasonable to assume, however, that significant problems are much less likely to occur when a multidisciplinary team approach is involved rather than an individual researcher undertaking a study independently.

Ideally, a research team should include a statistician, a professional with experience in the content areas of the study, and a senior investigator.²¹ The desirable area of expertise, however, is often not readily available to family physicians, especially those in community-based settings. Individuals with some training in research who are interested in being involved can usually be found in colleges and universities, hospitals, and at the local public health department. Psychologists, sociologists, health services researchers, public health epidemiologists, and nursing educators are all potential resources and possible collaborators. Establishing the necessary relationships to form an ad hoc research team is certainly more time and labor intensive than undertaking research independently, but generally results in the collection of more useful information.

Novices should consult survey methodology books before and during the study.^5-9 Excellent resources are available that provide a comprehensive overview of survey methods,²² means for improving response rates,²³ and methods for constructing relatively brief but thorough survey questions.⁵ Academic family practice fellowships often provide training in survey methodology. In addition, many family practice researchers respond favorably to requests for information or advice requested by telephone or email contact. The novice author of this article reports excellent success in contacting experts in this manner. With the advent of the Internet, a “cyberspace” team comprised of experts in the topic and the methodology is a reasonable and helpful option for the novice.

Survey content and structure

Novice researchers often assume that developing a questionnaire is an intuitive process attainable by virtually anyone, regardless of their level of research training. While it is true that questionnaires are relatively simple to construct, developing an instrument that is valid and reliable is not intuitive. An instrument is valid if it actually measures what we think it is measuring, and it is reliable if it measures the phenomenon consistently in repeated applications.²⁴ By following a few basic guidelines, those with limited research training can develop survey instruments capable of producing valid and reliable information. The 3 primary concerns for developing appropriate questions (items) are: (1) response format; (2) content; and (3) wording and placement (see Figure B, Survey Questions; and Figure C, Designing and Formatting the Survey).

Format

Questionnaires generally use a closed-ended format rather than an open-ended format. Closed formats spell out response options instead of asking study subjects to respond in their own words. Although there are many reasons for using closed formats, their primary advantages over open formats is that they are more specific and provide the same frame of reference to all respondents, and they allow quantitative analysis. A disadvantage is that they limit the possible range of responses envisioned by the investigators. Questionnaires with closed formats are therefore not as helpful as qualitative methods in the early, exploratory phases of a research project.

Closed-ended items can be formatted into several different categories (classes) of measurement, based on the relationship of the response categories to one another. Nominal measurements are responses that are sorted into unordered categories, such as demographic variables (ie, sex, ethnicity). Ordinal measurements are similar to nominal, except that there is a definite order to the categories. For example, ordinal items may ask respondents to rank their preferences among a list of options from the least desirable to the most desirable.

Survey items that ask for respondents(delete apostrophe) to rank order preferences are often a more useful than items that state, “check all that apply.” While checking all relevant responses may be necessary for certain items, such questions often lose valuable information as they can only supply raw percentages without supplying any comparison between responses. If a survey uses a rank order response, it enables determining the relative importance of the different categories during data analysis Table 1.

Two additional tools used on questionnaires are continuous variables and scales. Continuous variables can be simple counts (eg, the number of times something occurred) or physical attributes (eg, age or weight). A general rule when collecting information on continuous variables is to avoid obtaining the information in ranges of categories unless absolutely necessary. Response categories that reflect ranges of responses can always be constructed after the information is gathered, but if the information is gathered in ranges from the start, it cannot later be expanded to reflect specific values.

Scales are used by survey researchers to assess the intensity of respondents’ attitudes about a specific issue or issues. Likert scales are probably the best known and most widely used for measuring attitudes. These scales typically present respondents with a statement and ask them to indicate whether they “strongly agree,” “agree,” “neither agree nor disagree,” “disagree,” or “strongly disagree.” The wording of the response categories can be changed to reflect other concepts (eg, approval or disapproval), and the standard 5-response format can be expanded or abbreviated if necessary.

There are no hard and fast rules for determining the number of response categories to use for scaled items, or whether to use a neutral category or one that reflects uncertainty. Research indicates that the reliability of respondents’ ratings declines when using more than 9 rating scale points.²⁵ However, the reliability of a scale increases when the number of rating scale points is increased, with maximum benefit achieved with 5 or 7 scale points.^25,26 Since the objective of using scales is to gauge respondent’s preferences, it is sometimes argued that a middle point or category of uncertainty category should not be used. Odd-numbered rating scales, however, conform better with the underlying tenets of many statistical tests, suggesting the need for including this category.²⁹ As the number of rating scale points increases, respondents’ use of the midpoint category decreases substantially. ³⁰ Thus, based on the available literature, it is generally advisable to use between 5 and 7 response categories and an uncertainty category, unless there is a compelling reason to force respondents to choose between 2 competing perspectives or alternatives.

Content

Items should not be included on questionnaires when the only justification for inclusion is that the investigator feels the information “would be really interesting to know.” Rather, for each item, you should ask yourself how it addresses the study’s research question and how it will be used in the data analysis stage of the study. Researchers should develop a data analysis plan in advance of administering a questionnaire to determine exactly how each question will be used in the analysis. When the relationship between a particular item and the study’s research question is unclear, or it is not known how an item will be used in the analysis, the item should be removed from the questionnaire.

Wording and placement

The wording of questions should be kept simple, regardless of the education level of the respondents. Questions should be kept as short and direct as possible since shorter surveys tend to have higher response rates.^31,32 Each question should be scrutinized to ensure it is appropriate for the respondents and does not require or assume an inappropriate level of knowledge about a topic. Since first impressions are important for setting the tone of a questionnaire, never begin with sensitive or threatening questions.³³ Questionnaires should begin with simple, introductory (“warm-up”)“questions to help establish trust and an appropriate frame of mind for respondents.³⁴ Other successful strategies are: (1) when addressing multiple topics, insert an introductory statement immediately preceding each topic (eg, “In the next section we would like to ask you about …”); (2) request demographic information at the end of the questionnaire; and (3) always provide explicit instructions to avoid any confusion on the part of respondents.³⁵

Additional, clear information on survey content and structure is available in 2 books from Sage Publications.^5,36 By following simple guidelines and common sense, most family practice researchers can construct valid and reliable questionnaires. As a final safeguard, once a final draft of the questionnaire is completed, the researcher should always be the first respondent. By placing yourself in the respondent’s role and taking the time to think about and respond to each question, problems with the instrument that were overlooked are sometimes identified.

Analyzing surveys

It is not within the scope of this project to address statistical analysis of survey data. Before attempting data analysis, investigators should receive appropriate training or consult with a qualified professional. There are 3 topics that can and should be understood by novice researchers related to data analysis (Figure D, Developing a Framework for Analysis).

Coding

Before analyzing survey data it is necessary to assign numbers (codes) to the responses obtained. Since the computer program that is used for analyzing data does not know what the numbers mean, the researcher assigns meaning to the codes so that the results can be interpreted correctly. Coding refers to the process of developing the codes, assigning them to responses, and documenting the decision rules used for assigning specific codes to specific response categories. For example, almost all questionnaires contain missing values when respondents elect to not answer an item. Unique codes need to be assigned to distinguish between an item’s missing values, items that may not be applicable to a particular respondent, and responses that have a “none” or “no opinion” category.

Data can be entered into appropriate data files once codes have been assigned to responses and a codebook compiled that defines the codes and their corresponding response categories. It is important to ensure that the data are free of errors (are clean) prior to performing data analysis. Although many methods can be used for data cleaning (ie, data can be entered twice and results compared consistency), at a minimum all of the codes should be checked to ensure only legitimate codes appear.

Frequency distributions are tables produced by statistical software that display the number of respondents in each response category for each item (variable) used in the analysis. By carefully examining frequency tables, the researcher can check for illegitimate codes. Frequency tables also display the relative distribution of responses and allow identification of items that do not conform to expectations given what is known about the study population.

Sample size

Since it is usually not possible to study all of the members of the group (population) of interest in a study, a subset (sample) of the population is generally selected for study from the sampling frame. Sampling is the process by which study subjects are selected from the target population, while the sample frame is the members of a population who have a chance of being included in the survey. In probability samples, each member of the sampling frame has a known probability of being selected for the study, whereas in nonprobability samples, the probability of selection is unknown. When a high degree of precision in sampling is needed to unambiguously identify the magnitude of a problem in a population or the factors that cause the problem, then probability sampling techniques must be used.

When conducting an analytical study that examines precisely whether statistically significant differences exist between groups in a population, power analysis is used to determine what size sample is needed to detect the differences. Estimates of sample size based on power are inversely related to the expected size of the differences “(effect size)”-that is, detecting smaller differences requires a larger sample. If an analytical study is undertaken to determine the magnitude of the differences between 2 groups, it is necessary to work with a statistician or other methodology expert to perform the appropriate power analysis. For a basic but valuable description of sample size estimation, see chapter 13 of Hulley and Cummings.²¹

In contrast to analytical studies, exploratory and descriptive studies can frequently be conducted without the need for a power analysis. While some descriptive studies may require the use of probability techniques and precise sample estimates, this often is not the case for studies that establish the existence of a problem or estimating its dimensions. When conducting an exploratory or descriptive study using a survey design and a nonprobability sampling technique, considerations other than effect size or precision are used to determine sample size. For example, the availability of eligible respondents, limitations of time and resources, and the need for pilot study data can all contribute to selecting a nonprobability sample. When these types of sampling techniques are used, however, it is important to remember that the validity and reliability of the findings are not assured, and the findings cannot be used to demonstrate the existence of differences between groups. The findings of these types of studies are only suggestive and have limited application beyond the specific study setting.

Response rate

The response rate is a measure indicating the percentage of the identified sample that completed and returned the questionnaire. It is calculated by dividing the number of completed questionnaires by the total sample size identified for the study. For example, if a study is mailed to 500 physicians questionnaires and 100 returned a completed questionnaire, the response rate would be 20% (100/500).

The response rate for mailed questionnaires is extremely variable. Charities are generally content with a 1% to 3% response rate, the US Census Bureau expects to achieve a 99% rate, and among the general population, a 10% response rate is not uncommon. Although an 80% response rate is possible from an extremely motivated population, a rate of 70% is generally considered excellent.³⁴

The effect of nonresponse on the results of a survey depend on the degree to which those not responding are systematically different from the population from which they are drawn.²⁴ When the response rate is high (ie, 95%), the results obtained from the sample will likely provide accurate information about the target population (sampling frame) even if the nonrespondents are distinctly different. However, if nonrespondents differ in a systematic way from the target population and the response rate is low, bias in how much the survey results accurately reflect the true characteristics of the target population is likely.

When calculating the response rate, participants who have died or retired can be removed from the denominator as appropriate. Nonrespondents, however, who refuse to participate, do not return the survey, or have moved should be included. This bias tends to be more problematic in “sensitive” areas of research³⁷ than in studies of common, nonthreatening topics.³⁸ Imputing values for missing data from nonrespondents is complex and generally should not be undertaken.³⁹

Given the importance of response rate, every effort must be made to obtain as many completed questionnaires as possible and strategies to maximize the response rate should be integrated into the study design (see Dillman²³ for a useful discussion of successful strategies). Some simple means for improving response rates include constructing a short questionnaire, sending a well-written and personalized cover letter containing your signature, and emphasizing the importance of the study and the confidentiality of responses. It is also advisable to include a self-addressed, stamped envelope for return responses, and sometimes a small incentive is worthwhile. The National Center for Education Statistics notes that all surveys require some follow-up to achieve desirable response rates.⁴⁰ Survey researchers, therefore, should develop procedures for monitoring responses and implement follow-up plans shortly after the survey begins.

Generally, 2 or 3 mailings are used to maximize response rates. Use of post card reminders is an inexpensive, though untested, method to increase response. Several randomized studies have reported an increase in response rate from physicians in private practice with the use of monetary incentives, although the optimum amount is debated. Everett et al⁴⁰ compared the use of a $1 incentive vs no monetary incentive and found a significant increase with the incentive group (response rates: 63% in the $1 group; 45% in the no incentive group; P < .0001).⁴¹ Other studies have compared $2, $5, $10, $20, and $25 incentives and found that $2 or $5 incentives are most cost effective.⁴²⁴⁵ Similar findings have been reported for physician surveys in other countries.^31,46 In an assessment of incentive for enrollees in a health plan, a $2 incentive was more cost effective than a $5 incentive.⁴⁷ A $1 incentive was as effective as $2 in significantly increasing response rate in a low-income population.⁴⁸ Quality of responses have not varied by use of incentives and there does not appear to be an incentive-bias.

Use of lottery appears to also increase response rate in both physicians and the lay public, although there are no studies comparing lottery to a monetary incentive enclosed for all participants.^31,49 Use of either certified or priority return mail appears to increase response rates, and may be more cost effective when used for the second mailing.^45,48

Pilot testing

Though pilot testing is generally included in the development of a survey, it is often inadequately conducted Figure F Final Preparation). Frequently, investigators are eager to answer their research question and pilot testing is synonymous with letting a few colleagues take a quick look and make a few comments. Table 2 illustrates a problem that could have been avoided with proper pilot testing.¹⁰ One of the questions in the survey asked about how time is allotted for faculty to pursue scholarly activities and research (Format A). Unfortunately, the question mixes 2 types of time in 1 question: extended time away from the institution (sabbatical and mini-sabbatical) and time in the routine schedule. This was confusing to respondents and could have been avoided by separating the content into 2 separate questions (Format B).

Investigators should consider carefully whom to include in the pilot testing. Not only should this include the project team and survey “experts”, but it should also include a sample of the target audience. Pilot testing among multiple groups provides feedback about the wording and clarity of questions, appropriateness of the questions for the target population, and the presence of redundant or unnecessary items.

Conclusions

One of the authors (C.R.W.) recently worked on her first questionnaire project. Among the many lessons she learned was the value of a team in providing assistance, the importance of considering if the time spent on a particular activity makes it cost effective, and the need to be flexible depending on circumstances. She found that establishing good communication with the team cuts down on errors and wasted effort. Rewarding the team for all of their hard work improves morale and provides a positive model for future projects.

The mailed self-administered questionnaire is an important tool in primary care research. For family practice to continue its maturation as a research discipline, family practitioners need to be conversant in survey methodology and familiar with its pitfalls. We hope this primer-designed specifically for use in the family practice setting-will provide not only basic guidelines for novices but will also inspire further investigation.

Acknowledgments

The authors thank Laura Snell, MPH, for her thoughtful review of the manuscript. We also thank Olive Chen, PhD, for research assistance and Janice Rookstool for manuscript preparation.

Substantial disparities in children’s health and health care continue to exist across racial and ethnic groups in the United States.^1-4 With the release of Healthy People 2010, the United States has reaffirmed its commitment to eliminating these growing racial and economic disparities in children’s health. Healthy People 2010 calls for greater and more equitable delivery of high-quality primary care* and prevention to reduce these disparities. A strong primary care-oriented health care system is associated with more frequent and complete delivery of preventive services for children,^7-10 fewer complications from chronic illnesses,^11-14 and better health outcomes.^15-17

However, intensifying pressures to contain medical care costs in the US health care system have meant that even children with financial access to care are not guaranteed to receive high quality primary care. The for-profit nature of many health care delivery systems continues to raise serious concerns about quality because of the financial interest to reduce use of services. Safety-net providers such as community health centers are being forced to compete in the market-driven system and may have to compromise the quality of care they provide to vulnerable, primarily minority populations.¹⁸ Because of these concerns, consumers, providers, purchasers, and federal and state agencies are demanding better monitoring of the quality of primary care, particularly for vulnerable populations.¹⁹

Previous studies have identified significant racial and ethnic disparities in the quality of primary care among adults. Shi used nationally representative data from the Medical Expenditure Panel Survey and found that racial and ethnic minority adults experience worse primary care across most of its cardinal attributes, with the greatest differences being in the accessibility of medical care.²⁰ Murray-Garcia and colleagues²¹ also found that Asian American adults tend to report the lowest quality of primary care among racial and ethnic groups, although the results may be attributable in part to differences in patient expectations or survey response practices rather than to actual differences in quality. In 2 separate studies, Taira and colleagues^22,23 previously demonstrated similar findings regarding lower reported quality of primary care among Asian American adults. Finally, Morales et al²⁴ found that, with the exception of Asians and Pacific Islanders, there were few differences between minorities and whites in satisfaction with primary care and ratings of access and communication.

Studies of primary care among adults, however, are not readily applicable to children for several reasons.²⁵ First, a unique set of primary care financing, organization, and delivery systems has been developed for children. Examples include the recent State Children’s Health Insurance Program and nontraditional delivery settings such as school-based health centers. Second, childhood is a unique developmental stage of life during which children’s health care experiences strongly influence their future health and health care utilization. Third, primary care for children emphasizes preventive care rather than acute care as for adults and therefore must be evaluated differently.

Racial and ethnic disparities in primary care quality have rarely been studied for children.²⁶ Although several studies have evaluated racial differences in children’s use of primary care services, few have evaluated racial differences in more qualitative primary care experiences. Weech-Maldonado and colleagues, conducting the only study of this type, found that Asian, black, and Hispanic children experienced poorer access, timeliness to care, and communication with providers compared to whites. However, language appeared to play an important factor in these disparities.²⁷

The purpose of this study was to examine racial and ethnic differences in the quality of primary care specifically for children. Primary care was uniquely assessed, pursuant to the Institute of Medicine’s definition, with the use of a reliable and valid instrument asking parents to report on, rather than rate, the quality of care for their children. The study sought to identify deficits in primary care quality among children to lay groundwork for the development of clinical strategies and health care policies to eliminate health disparities.

Methods

Study design and setting

A cross-sectional community-based survey was conducted in a random sample of 1200 parents of elementary school children in 1 school district. The district spans 3 large suburban communities in San Bernardino County, California, near Los Angeles. The area encompasses a population of about 300,000 and approximately 17% of the population live in poverty. In San Bernardino County, there are 72.5 health care providers per 100,000 inhabitants; this rate is lower than the overall rate of 90 providers per 100,000 for the State of California.²⁸ Because the county has several rural areas (with low physician presence) that are not served by the school district, the physician ratio is likely to be an underestimate for the more urban geographic area under study.

A school district was selected as the setting for this study because it provides the single most comprehensive list of children in a community. A community sample avoids the biases associated with research based in provider settings that generally include only the most frequent users of health services.

The school district serves a population of 18,000 racially and socioeconomically diverse elementary school children in 20 elementary schools (kindergarten through grade 6). The racial and ethnic makeup of the population is approximately 43% Hispanic; 42% white; 10% Asian, Filipino, and Pacific Islander; 5% black; and fewer than 1% American Indian. The sampling frame was sorted and systematically sampled by the child’s sex, grade level, and school strata to ensure that the sample was representative of the community. To improve the analytic capacity of the sample, Asian and black subgroups were oversampled at 4 times the rate for Asians (compared with whites) and 16 times the rate for blacks to obtain approximately equal numbers of respondents across racial and ethnic groups.

In San Bernardino County, as in a growing number of other counties in California and throughout the United States, non-white racial and ethnic groups are beginning to represent more of the population. In this study, Hispanics are the numerical majority, but we continue to use the term “minority” to represent Asian, black, and Hispanic racial and ethnic groups because in most areas of the United States these groups continue to be the numerical minority.

Data collection

The Johns Hopkins University Office for Research Subjects approved the survey instrument and administration procedures. Questionnaires were administered through structured telephone interviews between November 2000 and January 2001.

Two rounds of informational mailers were sent to parents in advance of contact by telephone. To maintain legal privacy protections for parents, clerks employed by the school district made initial contact with families to schedule appointments for interviewers to complete the telephone interview. Reminder letters were mailed to parents who had scheduled an appointment but were not reached by telephone contact.

Of the original sample of 1200 children, 289 families had moved or left the school district, disconnected their telephone number, or had a telephone number that was busy or not answered on repeated (10+) attempts; 59 families were unable to participate because of language difficulties. Parents who reported to the study clerks that they were unable to complete the survey in English or Spanish were excluded from participation. Negative terming of 2 questions and alternate wording of 2 similar questions were used to check comprehension. Concern was raised in 1 case, but this was resolved through follow-up questioning.

Interviews were completed with the families of 413 children. After subtracting the unreachable families from the original sampling frame, the overall response rate was 49%. Children without a regular source of care were excluded from the analyses, leaving 403 respondents in the analytic sample.

Nonrespondents were similar to respondents in terms of child’s sex, race and ethnicity, and school in which the child was enrolled. Respondents were slightly more likely than nonrespondents (P < .05) to have a younger child (mean age = 8.1 vs 8.4 years). Data for these comparisons were available through an administrative data set provided by the school district and assembled each school year through an enrollment form completed by parents.

Measurement

Race and ethnicity. Race data were available through the parent-completed school enrollment files provided by the school district. The categories of race or ethnicity were white (non-Hispanic), Hispanic, black (non-Hispanic), Asian, Filipino, Pacific Islander, and American Indian. To ensure a sufficiently large sample size, we combined Asian, Filipino, and Pacific Islander into a single category called Asian. We also excluded American Indian from the study sample because of extremely small numbers.

Primary care quality. For this study we used the Pediatric Primary Care Assessment Tool (PCAT) developed by the Johns Hopkins Primary Care Policy Center for the Underserved to evaluate 4 cardinal attributes of primary care quality: first-contact care, longitudinality, comprehensiveness, and coordination Table 1. Scale scores were generated for each attribute based on summed responses to questions, with 4 Likert-type response choices: definitely (score = 4), probably (score = 3), probably not (score = 2), and definitely not (score = 1). “Don’t know” responses were coded as the middle score (2.5) because we assumed that not knowing about an important feature of primary care signified some partial failure to convey the availability of that particular feature. For example, parents’ not knowing whether their child could receive immunizations from the provider signified some partial communication failure on the part of the provider. Both child and adult versions of the instrument have been developed, the reliability and validity of which are reported elsewhere.^29,30

Within each cardinal attribute, the PCAT assesses structural characteristics of the facility or provider that reflect the capacity to achieve quality primary care and processes of care that indicate the achievement of the function in actual practice. Only patients who reported a regular source of care (n = 403) were asked to assess the quality of primary care.

First-contact care. Two subdomains of first-contact care are measured by the PCAT: accessibility of the provider and the degree to which the provider is used as a single point of entry into the medical care system. Accessibility is evaluated with 8 questions about characteristics of the health system that facilitate access (eg, If the facility is open on weekends, would the provider see the child the same day?). The utilization subdomain is scored with an algorithm that assigns a higher score for each type of service (acute illness, regular check-up, and immunizations) that is sought from the parent-identified regular source of care.

Longitudinality. Two subdomains of longitudinality are measured by the PCAT: interpersonal relationship with the provider and extent of affiliation. The relationship subdomain is evaluated with 14 questions concerning the parents’ perception of the “person orientation” of the interactions between provider, parents and child (eg, the degree of interest the provider has in the child as a person rather than as someone with a medical problem). The extent of affiliation subdomain addresses the extent of the child’s relationship with a specific provider. This is scored with an algorithm that assigns a higher score if the provider identified as the regular source of care also knows the child best and is the provider from whom care would be sought for a new problem.

Comprehensiveness. Two subdomains of comprehensiveness are measured by the PCAT: services available and services provided. Six questions address the availability of specific primary care services (eg, immunizations and tests for lead poisoning). Another 5 questions address the services received from the primary care source (eg, discussions of ways to stay healthy such as eating nutritious foods and getting enough sleep).

Coordination. For children who have visited a specialist (n = 135), 7 questions address the degree of interaction and integration between the primary care physician and specialist services (eg, Did the primary care provider know that you made the visit to the specialist?).

Covariates. We selected covariates based on previous studies demonstrating a relation between the variables and aspects of primary care quality such as accessibility and continuity of care. We controlled for socioeconomic status (income, employment, and education), characteristics of the health care system (provider specialty, practice setting, and cost-sharing requirements), and demographics (child’s age, sex, general health status, and insurance coverage). Because of extensive managed care penetration in California, we clarified the range of practice settings by using names of local managed care clinics, public health centers, and group practices as examples.

Analysis

The independent variable was racial and ethnic background, and its analytic categories included Asian, black, Hispanic, and white. Comparisons were made between race or ethnicity and the study covariates and scores for the primary care subdomains. Frequencies of the study covariates were compared across racial and ethnic groups, and the significance of these differences was assessed with chi-squared tests of association. Generalized linear model procedures were used to assess differences in primary care quality across racial and ethnic groups after adjusting for study covariates. Bonferroni t tests were used to test significance and account for multiple comparisons.

Two total primary care scores were generated by summing the mean scores for the primary care subdomains. The first total primary care score (A) included coordination of care, a domain that was answered only by a subset of the population that reported they had visited a specialist since they first saw their regular provider. Therefore, total score A was limited to 1 subset of the population (n = 135). The second total primary care score (B) did not include coordination of care and thus included the full study sample.

Multiple linear regression analyses were conducted to predict primary care quality. Four models were constructed incrementally, with the first including only dummy variables of race and ethnicity (with white “race” as the reference group). Additional models controlled for (1) socioeconomic status covariates, (2) health system characteristics, and (3) socioeconomic status, health system characteristics, and demographics. The models were constructed separately for primary care scores A and B. Regression coefficients and respective P values are reported for race/ethnicity categories and study covariates. The coefficient of determination (R² and adjusted R²) is reported for each model to describe how much of the variance in primary care quality was explained by the study variables.

Table 2 compares the unadjusted socioeconomic status, health system characteristics, and demographic factors of our analytic sample across racial and ethnic groups. As per the sampling strategy, respondents were nearly equally divided among the 4 categories of race and ethnicity. Most respondents (74.3%) had family incomes greater than $36,000/year, although a significantly smaller proportion of black (64.2%) and Hispanic (67.7%) families had incomes above this amount compared with whites (90.2%) and Asians (84.0%; P < .001). Racial and ethnic groups also differed in maternal education and employment, with Asians reporting the highest proportion with a high school education or greater (P < .01) and blacks reporting the highest employment among mothers (P < .001).

With regard to health system factors, Asians and whites were most likely to report seeking care at a doctor’s office (58.8% and 57.0%, respectively) compared to seeking care from a health maintenance organization clinic or other setting (P < .001). Hispanics reported the largest proportion of children receiving care from a health maintenance organization clinic setting (39.4%), and Asians reported the smallest proportion (20.6%; P < .05). White respondents had the highest proportion covered by private health insurance (86.6%) and Hispanic respondents had the lowest (79.1%; P < .05). Hispanics were most likely to be uninsured (13.13%; P < .05).

Asians were most likely to report having any cost sharing such as a deductible or co-payment (83.2%; P < .01). There were no significant differences in child’s age, sex, or health status across racial and ethnic groups.

Table 3 compares adjusted primary care quality scores across racial and ethnic groups. The attribute scales were standardized by summing the responses to each question in the attribute and dividing by the number of questions (range, 1-4). In general, Asian, black, and Hispanic parents reported slightly lower quality of primary care than did whites. Minority parents reported lower scores for 6 of the 7 subdomains, although only some of the findings were significant. Asian respondents reported the lowest (or statistically equivalent to the lowest) primary care quality for 5 of the 7 domains, reflecting differences of approximately 5% to 10%. These scores were significantly lower than those reported by whites for first-contact accessibility (P < .05), first-contact utilization (P < .01), interpersonal relationship (P < .05), and comprehensiveness of services received (P < .001). Moreover, Asians reported significantly lower mean scores than did whites for both total primary care scales. Black respondents reported significantly worse first-contact utilization but slightly greater accessibility than did whites, although the difference was not significant.

Table 4 presents 4 multiple regression models for the 2 versions of total primary care quality after successive adjustment for socioeconomic status, health system characteristics, and demographics. In model 1A (not adjusted for any covariates) Asian and black races (vs whites) were significant negative predictors of primary care quality. In model 1B (ie, coordination of care domain excluded) all minority groups (vs whites) were significant negative predictors of quality (P < .01). In models 2B and 3B, after controlling for socioeconomic status and health system characteristics, respectively, 2 additional covariates positively predicted total primary care quality. These were family income greater than $36,000 (P < .01) and having a pediatrician as opposed to any other type of family practitioner or generalist primary care provider (P < .001). Despite the addition of these covariates to models 2 and 3, minority racial and ethnic groups remained significant negative predictors for both versions of primary care quality (P < .05). Asian race remained a particularly significant predictor of quality (P < .001).

In model 4A, which controlled for all covariates, older child age and, nonintuitively, being uninsured were significant positive predictors of quality (likely due to the small number of uninsured respondents). In model 4B, health status was a significant predictor of quality (P < .05). With the addition of the full complement of covariates, Hispanic and black race and ethnicity became nonsignificant in both models despite small changes in the magnitude of the coefficients and P values (.07 and .06, respectively). The loss of significance in this model is likely attributable to the number of variables that was controlled for given the moderate sample size rather than to any confounding effects of specific covariates. In model 4A, Asian race remained a significant negative predictor of quality (P < .05). In model 4B, Asian race remained a strong negative predictor of quality and having a pediatrician remained a strong positive predictor (P < .001 for both). After adjustment for the natural rise in R² associated with the inclusion of additional covariates, the final models for both total scales (A and B) explained only about 8% and 11%, respectively, of the variation in primary care quality.

Discussion

This community-based study advances the literature by demonstrating that Asian, black, and Hispanic children experience poorer quality of primary care than whites, even after controlling for many differences in socioeconomic status, health system factors, and demographics. This suggests that racial and ethnic differences in quality of care are not simple reflections of ability to pay, health disparities, or other sociodemographics.

The findings in this study that parents of minority children, in particular Asian Americans, report lower quality of primary care is consistent with previous research among adults but has not been demonstrated previously for children.^20-24 This finding is particularly important because of the growing numbers of Asian Americans in the United States and because Asian children, despite their family’s higher education, are more likely than whites and some other ethnic groups to be in fair or poor health, underimmunized, and at risk for contracting preventable illnesses such as hepatitis B.^31-33 These differences in health and health risk may be remedied in part by the receipt of high-quality primary care.³⁴

Of the primary care measures, the greatest difference between Asians and whites was in comprehensiveness of services received. This domain covered the range of services that patients could receive from their regular provider and included items such as preventive counseling and discussions about growth and development. Although language was unlikely to be a determinant of quality in this study (because we excluded those unable to complete the survey in English or Spanish), it does not discount the potential of undetected or unstudied language difficulties to enhance disparities in health care. For example, even though Asian families in our study were able to communicate sufficiently in English, they might have rated the patient-provider relationship lower because of trouble finding a provider who spoke their language. Regardless, the finding suggests the importance of making services more widely available to minority groups, including improvements in communication about existing primary care services.

An interesting secondary finding was that parents who reported a pediatrician as the child’s regular source of care reported higher quality primary care than did parents reporting other generalist providers. In particular, pediatricians appeared to perform better than other providers on 3 of the attributes: utilization of services (P < .02), patient-provider relationship (P < .0001), and services provided (P < .0001; data not shown). The differences may be attributable in part to the greater frequency of visits to pediatricians for well-child care; thus, greater opportunities may exist for delivery of preventive services and the development of the patient-provider relationship. Future research should explore the experience of minority patients receiving care from various provider specialties.

Despite significant findings, the most comprehensive regression models explained only a small proportion of the variation in primary care quality (about 11%). Other factors that may play a role in determining primary care quality, but were not included in this analysis, include health insurance plan restrictions (discussed in an upcoming paper), practice arrangements, racial concordance between the patient and provider, family mobility, and perhaps provider-specific factors such as training or years in practice.

This study has several limitations. First, the cross-sectional design and analysis allowed the demonstration of association and not of causality. Second, this study examined only 4 broad classifications of race and ethnicity that do not capture within-group variations in ethnicity or culture that could be associated with differences in quality of care received. Standard measurements of race and ethnicity also do not fully capture biologic, cultural, socioeconomic, and political aspects of multiculturalism that may interact and produce more complex findings than those reported.³⁵

Third, because of the moderate response rate, the respondents in this study may not be fully representative of the population under study. Although respondents were demographically similar to nonrespondents, participants may have been more likely than nonrespondents to have children in poorer health status or have more negative experiences with the health care system. Although this does not threaten the internal validity of this study (because response rates did not differ substantially across racial groups), such bias could lead to lower overall estimates of primary care quality regardless of racial group.

Fourth, studies that rely on patient reporting to compare quality of care across racial groups often may capture racial and ethnic group variations in perceptions of care or different standards for assessing care. In this study, we used an instrument for assessing quality of care that relies heavily on factual reporting (eg, waiting times and receipt of particular services) rather than on satisfaction or performance ratings, so our study was less subject to these biases.

In conclusion, this study demonstrated significant differences in the quality of primary care for children across racial and ethnic groups. These findings in part suggest that ensuring adequate health insurance coverage may not be sufficient to reduce racial and ethnic disparities in quality of care. Although the cause or mechanism of these disparities in quality is not entirely established, the findings encourage careful additional monitoring of the delivery of primary care, in particular to minority children. At a minimum, health care providers and organizations should make primary care services more accessible to minority families, provide the services in a culturally and linguistically competent manner (to encourage the development of the physician-patient relationship), and communicate more effectively with families about the range of child health services offered.

*The Institute of Medicine defines primary care as “the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.”⁵ The practice of primary care is best characterized as a set of attributes or functions that, only when performed together, constitute the delivery of primary care. Empirical studies have further delineated and operationalized 4 core attributes of primary care: first-contact care with a designated primary care physician; longitudinality, or ongoing care, with a physician or place of care; comprehensiveness of services; and coordination or integration of those services⁶Table 1.

Acknowledgments

The authors thank Barbara Starfield, MD, Lisa Cooper, MD, and Maria Trent, MD, for their thoughtful review of this manuscript. They also thank Jane Lyon for her generous support of and help in coordinating the study in the school district.

Substantial disparities in children’s health and health care continue to exist across racial and ethnic groups in the United States.^1-4 With the release of Healthy People 2010, the United States has reaffirmed its commitment to eliminating these growing racial and economic disparities in children’s health. Healthy People 2010 calls for greater and more equitable delivery of high-quality primary care* and prevention to reduce these disparities. A strong primary care-oriented health care system is associated with more frequent and complete delivery of preventive services for children,^7-10 fewer complications from chronic illnesses,^11-14 and better health outcomes.^15-17

However, intensifying pressures to contain medical care costs in the US health care system have meant that even children with financial access to care are not guaranteed to receive high quality primary care. The for-profit nature of many health care delivery systems continues to raise serious concerns about quality because of the financial interest to reduce use of services. Safety-net providers such as community health centers are being forced to compete in the market-driven system and may have to compromise the quality of care they provide to vulnerable, primarily minority populations.¹⁸ Because of these concerns, consumers, providers, purchasers, and federal and state agencies are demanding better monitoring of the quality of primary care, particularly for vulnerable populations.¹⁹

Previous studies have identified significant racial and ethnic disparities in the quality of primary care among adults. Shi used nationally representative data from the Medical Expenditure Panel Survey and found that racial and ethnic minority adults experience worse primary care across most of its cardinal attributes, with the greatest differences being in the accessibility of medical care.²⁰ Murray-Garcia and colleagues²¹ also found that Asian American adults tend to report the lowest quality of primary care among racial and ethnic groups, although the results may be attributable in part to differences in patient expectations or survey response practices rather than to actual differences in quality. In 2 separate studies, Taira and colleagues^22,23 previously demonstrated similar findings regarding lower reported quality of primary care among Asian American adults. Finally, Morales et al²⁴ found that, with the exception of Asians and Pacific Islanders, there were few differences between minorities and whites in satisfaction with primary care and ratings of access and communication.

Studies of primary care among adults, however, are not readily applicable to children for several reasons.²⁵ First, a unique set of primary care financing, organization, and delivery systems has been developed for children. Examples include the recent State Children’s Health Insurance Program and nontraditional delivery settings such as school-based health centers. Second, childhood is a unique developmental stage of life during which children’s health care experiences strongly influence their future health and health care utilization. Third, primary care for children emphasizes preventive care rather than acute care as for adults and therefore must be evaluated differently.

Racial and ethnic disparities in primary care quality have rarely been studied for children.²⁶ Although several studies have evaluated racial differences in children’s use of primary care services, few have evaluated racial differences in more qualitative primary care experiences. Weech-Maldonado and colleagues, conducting the only study of this type, found that Asian, black, and Hispanic children experienced poorer access, timeliness to care, and communication with providers compared to whites. However, language appeared to play an important factor in these disparities.²⁷

The purpose of this study was to examine racial and ethnic differences in the quality of primary care specifically for children. Primary care was uniquely assessed, pursuant to the Institute of Medicine’s definition, with the use of a reliable and valid instrument asking parents to report on, rather than rate, the quality of care for their children. The study sought to identify deficits in primary care quality among children to lay groundwork for the development of clinical strategies and health care policies to eliminate health disparities.

Methods

Study design and setting

A cross-sectional community-based survey was conducted in a random sample of 1200 parents of elementary school children in 1 school district. The district spans 3 large suburban communities in San Bernardino County, California, near Los Angeles. The area encompasses a population of about 300,000 and approximately 17% of the population live in poverty. In San Bernardino County, there are 72.5 health care providers per 100,000 inhabitants; this rate is lower than the overall rate of 90 providers per 100,000 for the State of California.²⁸ Because the county has several rural areas (with low physician presence) that are not served by the school district, the physician ratio is likely to be an underestimate for the more urban geographic area under study.

A school district was selected as the setting for this study because it provides the single most comprehensive list of children in a community. A community sample avoids the biases associated with research based in provider settings that generally include only the most frequent users of health services.

The school district serves a population of 18,000 racially and socioeconomically diverse elementary school children in 20 elementary schools (kindergarten through grade 6). The racial and ethnic makeup of the population is approximately 43% Hispanic; 42% white; 10% Asian, Filipino, and Pacific Islander; 5% black; and fewer than 1% American Indian. The sampling frame was sorted and systematically sampled by the child’s sex, grade level, and school strata to ensure that the sample was representative of the community. To improve the analytic capacity of the sample, Asian and black subgroups were oversampled at 4 times the rate for Asians (compared with whites) and 16 times the rate for blacks to obtain approximately equal numbers of respondents across racial and ethnic groups.

In San Bernardino County, as in a growing number of other counties in California and throughout the United States, non-white racial and ethnic groups are beginning to represent more of the population. In this study, Hispanics are the numerical majority, but we continue to use the term “minority” to represent Asian, black, and Hispanic racial and ethnic groups because in most areas of the United States these groups continue to be the numerical minority.

Data collection

The Johns Hopkins University Office for Research Subjects approved the survey instrument and administration procedures. Questionnaires were administered through structured telephone interviews between November 2000 and January 2001.

Two rounds of informational mailers were sent to parents in advance of contact by telephone. To maintain legal privacy protections for parents, clerks employed by the school district made initial contact with families to schedule appointments for interviewers to complete the telephone interview. Reminder letters were mailed to parents who had scheduled an appointment but were not reached by telephone contact.

Of the original sample of 1200 children, 289 families had moved or left the school district, disconnected their telephone number, or had a telephone number that was busy or not answered on repeated (10+) attempts; 59 families were unable to participate because of language difficulties. Parents who reported to the study clerks that they were unable to complete the survey in English or Spanish were excluded from participation. Negative terming of 2 questions and alternate wording of 2 similar questions were used to check comprehension. Concern was raised in 1 case, but this was resolved through follow-up questioning.

Interviews were completed with the families of 413 children. After subtracting the unreachable families from the original sampling frame, the overall response rate was 49%. Children without a regular source of care were excluded from the analyses, leaving 403 respondents in the analytic sample.

Nonrespondents were similar to respondents in terms of child’s sex, race and ethnicity, and school in which the child was enrolled. Respondents were slightly more likely than nonrespondents (P < .05) to have a younger child (mean age = 8.1 vs 8.4 years). Data for these comparisons were available through an administrative data set provided by the school district and assembled each school year through an enrollment form completed by parents.

Measurement

Race and ethnicity. Race data were available through the parent-completed school enrollment files provided by the school district. The categories of race or ethnicity were white (non-Hispanic), Hispanic, black (non-Hispanic), Asian, Filipino, Pacific Islander, and American Indian. To ensure a sufficiently large sample size, we combined Asian, Filipino, and Pacific Islander into a single category called Asian. We also excluded American Indian from the study sample because of extremely small numbers.

Primary care quality. For this study we used the Pediatric Primary Care Assessment Tool (PCAT) developed by the Johns Hopkins Primary Care Policy Center for the Underserved to evaluate 4 cardinal attributes of primary care quality: first-contact care, longitudinality, comprehensiveness, and coordination Table 1. Scale scores were generated for each attribute based on summed responses to questions, with 4 Likert-type response choices: definitely (score = 4), probably (score = 3), probably not (score = 2), and definitely not (score = 1). “Don’t know” responses were coded as the middle score (2.5) because we assumed that not knowing about an important feature of primary care signified some partial failure to convey the availability of that particular feature. For example, parents’ not knowing whether their child could receive immunizations from the provider signified some partial communication failure on the part of the provider. Both child and adult versions of the instrument have been developed, the reliability and validity of which are reported elsewhere.^29,30

Within each cardinal attribute, the PCAT assesses structural characteristics of the facility or provider that reflect the capacity to achieve quality primary care and processes of care that indicate the achievement of the function in actual practice. Only patients who reported a regular source of care (n = 403) were asked to assess the quality of primary care.

First-contact care. Two subdomains of first-contact care are measured by the PCAT: accessibility of the provider and the degree to which the provider is used as a single point of entry into the medical care system. Accessibility is evaluated with 8 questions about characteristics of the health system that facilitate access (eg, If the facility is open on weekends, would the provider see the child the same day?). The utilization subdomain is scored with an algorithm that assigns a higher score for each type of service (acute illness, regular check-up, and immunizations) that is sought from the parent-identified regular source of care.

Longitudinality. Two subdomains of longitudinality are measured by the PCAT: interpersonal relationship with the provider and extent of affiliation. The relationship subdomain is evaluated with 14 questions concerning the parents’ perception of the “person orientation” of the interactions between provider, parents and child (eg, the degree of interest the provider has in the child as a person rather than as someone with a medical problem). The extent of affiliation subdomain addresses the extent of the child’s relationship with a specific provider. This is scored with an algorithm that assigns a higher score if the provider identified as the regular source of care also knows the child best and is the provider from whom care would be sought for a new problem.

Comprehensiveness. Two subdomains of comprehensiveness are measured by the PCAT: services available and services provided. Six questions address the availability of specific primary care services (eg, immunizations and tests for lead poisoning). Another 5 questions address the services received from the primary care source (eg, discussions of ways to stay healthy such as eating nutritious foods and getting enough sleep).

Coordination. For children who have visited a specialist (n = 135), 7 questions address the degree of interaction and integration between the primary care physician and specialist services (eg, Did the primary care provider know that you made the visit to the specialist?).

Covariates. We selected covariates based on previous studies demonstrating a relation between the variables and aspects of primary care quality such as accessibility and continuity of care. We controlled for socioeconomic status (income, employment, and education), characteristics of the health care system (provider specialty, practice setting, and cost-sharing requirements), and demographics (child’s age, sex, general health status, and insurance coverage). Because of extensive managed care penetration in California, we clarified the range of practice settings by using names of local managed care clinics, public health centers, and group practices as examples.

Analysis

The independent variable was racial and ethnic background, and its analytic categories included Asian, black, Hispanic, and white. Comparisons were made between race or ethnicity and the study covariates and scores for the primary care subdomains. Frequencies of the study covariates were compared across racial and ethnic groups, and the significance of these differences was assessed with chi-squared tests of association. Generalized linear model procedures were used to assess differences in primary care quality across racial and ethnic groups after adjusting for study covariates. Bonferroni t tests were used to test significance and account for multiple comparisons.

Two total primary care scores were generated by summing the mean scores for the primary care subdomains. The first total primary care score (A) included coordination of care, a domain that was answered only by a subset of the population that reported they had visited a specialist since they first saw their regular provider. Therefore, total score A was limited to 1 subset of the population (n = 135). The second total primary care score (B) did not include coordination of care and thus included the full study sample.

Multiple linear regression analyses were conducted to predict primary care quality. Four models were constructed incrementally, with the first including only dummy variables of race and ethnicity (with white “race” as the reference group). Additional models controlled for (1) socioeconomic status covariates, (2) health system characteristics, and (3) socioeconomic status, health system characteristics, and demographics. The models were constructed separately for primary care scores A and B. Regression coefficients and respective P values are reported for race/ethnicity categories and study covariates. The coefficient of determination (R² and adjusted R²) is reported for each model to describe how much of the variance in primary care quality was explained by the study variables.

Table 2 compares the unadjusted socioeconomic status, health system characteristics, and demographic factors of our analytic sample across racial and ethnic groups. As per the sampling strategy, respondents were nearly equally divided among the 4 categories of race and ethnicity. Most respondents (74.3%) had family incomes greater than $36,000/year, although a significantly smaller proportion of black (64.2%) and Hispanic (67.7%) families had incomes above this amount compared with whites (90.2%) and Asians (84.0%; P < .001). Racial and ethnic groups also differed in maternal education and employment, with Asians reporting the highest proportion with a high school education or greater (P < .01) and blacks reporting the highest employment among mothers (P < .001).

With regard to health system factors, Asians and whites were most likely to report seeking care at a doctor’s office (58.8% and 57.0%, respectively) compared to seeking care from a health maintenance organization clinic or other setting (P < .001). Hispanics reported the largest proportion of children receiving care from a health maintenance organization clinic setting (39.4%), and Asians reported the smallest proportion (20.6%; P < .05). White respondents had the highest proportion covered by private health insurance (86.6%) and Hispanic respondents had the lowest (79.1%; P < .05). Hispanics were most likely to be uninsured (13.13%; P < .05).

Asians were most likely to report having any cost sharing such as a deductible or co-payment (83.2%; P < .01). There were no significant differences in child’s age, sex, or health status across racial and ethnic groups.

Table 3 compares adjusted primary care quality scores across racial and ethnic groups. The attribute scales were standardized by summing the responses to each question in the attribute and dividing by the number of questions (range, 1-4). In general, Asian, black, and Hispanic parents reported slightly lower quality of primary care than did whites. Minority parents reported lower scores for 6 of the 7 subdomains, although only some of the findings were significant. Asian respondents reported the lowest (or statistically equivalent to the lowest) primary care quality for 5 of the 7 domains, reflecting differences of approximately 5% to 10%. These scores were significantly lower than those reported by whites for first-contact accessibility (P < .05), first-contact utilization (P < .01), interpersonal relationship (P < .05), and comprehensiveness of services received (P < .001). Moreover, Asians reported significantly lower mean scores than did whites for both total primary care scales. Black respondents reported significantly worse first-contact utilization but slightly greater accessibility than did whites, although the difference was not significant.

Table 4 presents 4 multiple regression models for the 2 versions of total primary care quality after successive adjustment for socioeconomic status, health system characteristics, and demographics. In model 1A (not adjusted for any covariates) Asian and black races (vs whites) were significant negative predictors of primary care quality. In model 1B (ie, coordination of care domain excluded) all minority groups (vs whites) were significant negative predictors of quality (P < .01). In models 2B and 3B, after controlling for socioeconomic status and health system characteristics, respectively, 2 additional covariates positively predicted total primary care quality. These were family income greater than $36,000 (P < .01) and having a pediatrician as opposed to any other type of family practitioner or generalist primary care provider (P < .001). Despite the addition of these covariates to models 2 and 3, minority racial and ethnic groups remained significant negative predictors for both versions of primary care quality (P < .05). Asian race remained a particularly significant predictor of quality (P < .001).

In model 4A, which controlled for all covariates, older child age and, nonintuitively, being uninsured were significant positive predictors of quality (likely due to the small number of uninsured respondents). In model 4B, health status was a significant predictor of quality (P < .05). With the addition of the full complement of covariates, Hispanic and black race and ethnicity became nonsignificant in both models despite small changes in the magnitude of the coefficients and P values (.07 and .06, respectively). The loss of significance in this model is likely attributable to the number of variables that was controlled for given the moderate sample size rather than to any confounding effects of specific covariates. In model 4A, Asian race remained a significant negative predictor of quality (P < .05). In model 4B, Asian race remained a strong negative predictor of quality and having a pediatrician remained a strong positive predictor (P < .001 for both). After adjustment for the natural rise in R² associated with the inclusion of additional covariates, the final models for both total scales (A and B) explained only about 8% and 11%, respectively, of the variation in primary care quality.

Discussion

This community-based study advances the literature by demonstrating that Asian, black, and Hispanic children experience poorer quality of primary care than whites, even after controlling for many differences in socioeconomic status, health system factors, and demographics. This suggests that racial and ethnic differences in quality of care are not simple reflections of ability to pay, health disparities, or other sociodemographics.

The findings in this study that parents of minority children, in particular Asian Americans, report lower quality of primary care is consistent with previous research among adults but has not been demonstrated previously for children.^20-24 This finding is particularly important because of the growing numbers of Asian Americans in the United States and because Asian children, despite their family’s higher education, are more likely than whites and some other ethnic groups to be in fair or poor health, underimmunized, and at risk for contracting preventable illnesses such as hepatitis B.^31-33 These differences in health and health risk may be remedied in part by the receipt of high-quality primary care.³⁴

Of the primary care measures, the greatest difference between Asians and whites was in comprehensiveness of services received. This domain covered the range of services that patients could receive from their regular provider and included items such as preventive counseling and discussions about growth and development. Although language was unlikely to be a determinant of quality in this study (because we excluded those unable to complete the survey in English or Spanish), it does not discount the potential of undetected or unstudied language difficulties to enhance disparities in health care. For example, even though Asian families in our study were able to communicate sufficiently in English, they might have rated the patient-provider relationship lower because of trouble finding a provider who spoke their language. Regardless, the finding suggests the importance of making services more widely available to minority groups, including improvements in communication about existing primary care services.

An interesting secondary finding was that parents who reported a pediatrician as the child’s regular source of care reported higher quality primary care than did parents reporting other generalist providers. In particular, pediatricians appeared to perform better than other providers on 3 of the attributes: utilization of services (P < .02), patient-provider relationship (P < .0001), and services provided (P < .0001; data not shown). The differences may be attributable in part to the greater frequency of visits to pediatricians for well-child care; thus, greater opportunities may exist for delivery of preventive services and the development of the patient-provider relationship. Future research should explore the experience of minority patients receiving care from various provider specialties.

Despite significant findings, the most comprehensive regression models explained only a small proportion of the variation in primary care quality (about 11%). Other factors that may play a role in determining primary care quality, but were not included in this analysis, include health insurance plan restrictions (discussed in an upcoming paper), practice arrangements, racial concordance between the patient and provider, family mobility, and perhaps provider-specific factors such as training or years in practice.

This study has several limitations. First, the cross-sectional design and analysis allowed the demonstration of association and not of causality. Second, this study examined only 4 broad classifications of race and ethnicity that do not capture within-group variations in ethnicity or culture that could be associated with differences in quality of care received. Standard measurements of race and ethnicity also do not fully capture biologic, cultural, socioeconomic, and political aspects of multiculturalism that may interact and produce more complex findings than those reported.³⁵

Third, because of the moderate response rate, the respondents in this study may not be fully representative of the population under study. Although respondents were demographically similar to nonrespondents, participants may have been more likely than nonrespondents to have children in poorer health status or have more negative experiences with the health care system. Although this does not threaten the internal validity of this study (because response rates did not differ substantially across racial groups), such bias could lead to lower overall estimates of primary care quality regardless of racial group.

Fourth, studies that rely on patient reporting to compare quality of care across racial groups often may capture racial and ethnic group variations in perceptions of care or different standards for assessing care. In this study, we used an instrument for assessing quality of care that relies heavily on factual reporting (eg, waiting times and receipt of particular services) rather than on satisfaction or performance ratings, so our study was less subject to these biases.

In conclusion, this study demonstrated significant differences in the quality of primary care for children across racial and ethnic groups. These findings in part suggest that ensuring adequate health insurance coverage may not be sufficient to reduce racial and ethnic disparities in quality of care. Although the cause or mechanism of these disparities in quality is not entirely established, the findings encourage careful additional monitoring of the delivery of primary care, in particular to minority children. At a minimum, health care providers and organizations should make primary care services more accessible to minority families, provide the services in a culturally and linguistically competent manner (to encourage the development of the physician-patient relationship), and communicate more effectively with families about the range of child health services offered.

*The Institute of Medicine defines primary care as “the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.”⁵ The practice of primary care is best characterized as a set of attributes or functions that, only when performed together, constitute the delivery of primary care. Empirical studies have further delineated and operationalized 4 core attributes of primary care: first-contact care with a designated primary care physician; longitudinality, or ongoing care, with a physician or place of care; comprehensiveness of services; and coordination or integration of those services⁶Table 1.

Acknowledgments

The authors thank Barbara Starfield, MD, Lisa Cooper, MD, and Maria Trent, MD, for their thoughtful review of this manuscript. They also thank Jane Lyon for her generous support of and help in coordinating the study in the school district.

Substantial disparities in children’s health and health care continue to exist across racial and ethnic groups in the United States.^1-4 With the release of Healthy People 2010, the United States has reaffirmed its commitment to eliminating these growing racial and economic disparities in children’s health. Healthy People 2010 calls for greater and more equitable delivery of high-quality primary care* and prevention to reduce these disparities. A strong primary care-oriented health care system is associated with more frequent and complete delivery of preventive services for children,^7-10 fewer complications from chronic illnesses,^11-14 and better health outcomes.^15-17

However, intensifying pressures to contain medical care costs in the US health care system have meant that even children with financial access to care are not guaranteed to receive high quality primary care. The for-profit nature of many health care delivery systems continues to raise serious concerns about quality because of the financial interest to reduce use of services. Safety-net providers such as community health centers are being forced to compete in the market-driven system and may have to compromise the quality of care they provide to vulnerable, primarily minority populations.¹⁸ Because of these concerns, consumers, providers, purchasers, and federal and state agencies are demanding better monitoring of the quality of primary care, particularly for vulnerable populations.¹⁹

Previous studies have identified significant racial and ethnic disparities in the quality of primary care among adults. Shi used nationally representative data from the Medical Expenditure Panel Survey and found that racial and ethnic minority adults experience worse primary care across most of its cardinal attributes, with the greatest differences being in the accessibility of medical care.²⁰ Murray-Garcia and colleagues²¹ also found that Asian American adults tend to report the lowest quality of primary care among racial and ethnic groups, although the results may be attributable in part to differences in patient expectations or survey response practices rather than to actual differences in quality. In 2 separate studies, Taira and colleagues^22,23 previously demonstrated similar findings regarding lower reported quality of primary care among Asian American adults. Finally, Morales et al²⁴ found that, with the exception of Asians and Pacific Islanders, there were few differences between minorities and whites in satisfaction with primary care and ratings of access and communication.

Studies of primary care among adults, however, are not readily applicable to children for several reasons.²⁵ First, a unique set of primary care financing, organization, and delivery systems has been developed for children. Examples include the recent State Children’s Health Insurance Program and nontraditional delivery settings such as school-based health centers. Second, childhood is a unique developmental stage of life during which children’s health care experiences strongly influence their future health and health care utilization. Third, primary care for children emphasizes preventive care rather than acute care as for adults and therefore must be evaluated differently.

Racial and ethnic disparities in primary care quality have rarely been studied for children.²⁶ Although several studies have evaluated racial differences in children’s use of primary care services, few have evaluated racial differences in more qualitative primary care experiences. Weech-Maldonado and colleagues, conducting the only study of this type, found that Asian, black, and Hispanic children experienced poorer access, timeliness to care, and communication with providers compared to whites. However, language appeared to play an important factor in these disparities.²⁷

The purpose of this study was to examine racial and ethnic differences in the quality of primary care specifically for children. Primary care was uniquely assessed, pursuant to the Institute of Medicine’s definition, with the use of a reliable and valid instrument asking parents to report on, rather than rate, the quality of care for their children. The study sought to identify deficits in primary care quality among children to lay groundwork for the development of clinical strategies and health care policies to eliminate health disparities.

Methods

Study design and setting

A cross-sectional community-based survey was conducted in a random sample of 1200 parents of elementary school children in 1 school district. The district spans 3 large suburban communities in San Bernardino County, California, near Los Angeles. The area encompasses a population of about 300,000 and approximately 17% of the population live in poverty. In San Bernardino County, there are 72.5 health care providers per 100,000 inhabitants; this rate is lower than the overall rate of 90 providers per 100,000 for the State of California.²⁸ Because the county has several rural areas (with low physician presence) that are not served by the school district, the physician ratio is likely to be an underestimate for the more urban geographic area under study.

A school district was selected as the setting for this study because it provides the single most comprehensive list of children in a community. A community sample avoids the biases associated with research based in provider settings that generally include only the most frequent users of health services.

The school district serves a population of 18,000 racially and socioeconomically diverse elementary school children in 20 elementary schools (kindergarten through grade 6). The racial and ethnic makeup of the population is approximately 43% Hispanic; 42% white; 10% Asian, Filipino, and Pacific Islander; 5% black; and fewer than 1% American Indian. The sampling frame was sorted and systematically sampled by the child’s sex, grade level, and school strata to ensure that the sample was representative of the community. To improve the analytic capacity of the sample, Asian and black subgroups were oversampled at 4 times the rate for Asians (compared with whites) and 16 times the rate for blacks to obtain approximately equal numbers of respondents across racial and ethnic groups.

In San Bernardino County, as in a growing number of other counties in California and throughout the United States, non-white racial and ethnic groups are beginning to represent more of the population. In this study, Hispanics are the numerical majority, but we continue to use the term “minority” to represent Asian, black, and Hispanic racial and ethnic groups because in most areas of the United States these groups continue to be the numerical minority.

Data collection

The Johns Hopkins University Office for Research Subjects approved the survey instrument and administration procedures. Questionnaires were administered through structured telephone interviews between November 2000 and January 2001.

Two rounds of informational mailers were sent to parents in advance of contact by telephone. To maintain legal privacy protections for parents, clerks employed by the school district made initial contact with families to schedule appointments for interviewers to complete the telephone interview. Reminder letters were mailed to parents who had scheduled an appointment but were not reached by telephone contact.

Of the original sample of 1200 children, 289 families had moved or left the school district, disconnected their telephone number, or had a telephone number that was busy or not answered on repeated (10+) attempts; 59 families were unable to participate because of language difficulties. Parents who reported to the study clerks that they were unable to complete the survey in English or Spanish were excluded from participation. Negative terming of 2 questions and alternate wording of 2 similar questions were used to check comprehension. Concern was raised in 1 case, but this was resolved through follow-up questioning.

Interviews were completed with the families of 413 children. After subtracting the unreachable families from the original sampling frame, the overall response rate was 49%. Children without a regular source of care were excluded from the analyses, leaving 403 respondents in the analytic sample.

Nonrespondents were similar to respondents in terms of child’s sex, race and ethnicity, and school in which the child was enrolled. Respondents were slightly more likely than nonrespondents (P < .05) to have a younger child (mean age = 8.1 vs 8.4 years). Data for these comparisons were available through an administrative data set provided by the school district and assembled each school year through an enrollment form completed by parents.

Measurement

Race and ethnicity. Race data were available through the parent-completed school enrollment files provided by the school district. The categories of race or ethnicity were white (non-Hispanic), Hispanic, black (non-Hispanic), Asian, Filipino, Pacific Islander, and American Indian. To ensure a sufficiently large sample size, we combined Asian, Filipino, and Pacific Islander into a single category called Asian. We also excluded American Indian from the study sample because of extremely small numbers.

Primary care quality. For this study we used the Pediatric Primary Care Assessment Tool (PCAT) developed by the Johns Hopkins Primary Care Policy Center for the Underserved to evaluate 4 cardinal attributes of primary care quality: first-contact care, longitudinality, comprehensiveness, and coordination Table 1. Scale scores were generated for each attribute based on summed responses to questions, with 4 Likert-type response choices: definitely (score = 4), probably (score = 3), probably not (score = 2), and definitely not (score = 1). “Don’t know” responses were coded as the middle score (2.5) because we assumed that not knowing about an important feature of primary care signified some partial failure to convey the availability of that particular feature. For example, parents’ not knowing whether their child could receive immunizations from the provider signified some partial communication failure on the part of the provider. Both child and adult versions of the instrument have been developed, the reliability and validity of which are reported elsewhere.^29,30

Within each cardinal attribute, the PCAT assesses structural characteristics of the facility or provider that reflect the capacity to achieve quality primary care and processes of care that indicate the achievement of the function in actual practice. Only patients who reported a regular source of care (n = 403) were asked to assess the quality of primary care.

First-contact care. Two subdomains of first-contact care are measured by the PCAT: accessibility of the provider and the degree to which the provider is used as a single point of entry into the medical care system. Accessibility is evaluated with 8 questions about characteristics of the health system that facilitate access (eg, If the facility is open on weekends, would the provider see the child the same day?). The utilization subdomain is scored with an algorithm that assigns a higher score for each type of service (acute illness, regular check-up, and immunizations) that is sought from the parent-identified regular source of care.

Longitudinality. Two subdomains of longitudinality are measured by the PCAT: interpersonal relationship with the provider and extent of affiliation. The relationship subdomain is evaluated with 14 questions concerning the parents’ perception of the “person orientation” of the interactions between provider, parents and child (eg, the degree of interest the provider has in the child as a person rather than as someone with a medical problem). The extent of affiliation subdomain addresses the extent of the child’s relationship with a specific provider. This is scored with an algorithm that assigns a higher score if the provider identified as the regular source of care also knows the child best and is the provider from whom care would be sought for a new problem.

Comprehensiveness. Two subdomains of comprehensiveness are measured by the PCAT: services available and services provided. Six questions address the availability of specific primary care services (eg, immunizations and tests for lead poisoning). Another 5 questions address the services received from the primary care source (eg, discussions of ways to stay healthy such as eating nutritious foods and getting enough sleep).

Coordination. For children who have visited a specialist (n = 135), 7 questions address the degree of interaction and integration between the primary care physician and specialist services (eg, Did the primary care provider know that you made the visit to the specialist?).

Covariates. We selected covariates based on previous studies demonstrating a relation between the variables and aspects of primary care quality such as accessibility and continuity of care. We controlled for socioeconomic status (income, employment, and education), characteristics of the health care system (provider specialty, practice setting, and cost-sharing requirements), and demographics (child’s age, sex, general health status, and insurance coverage). Because of extensive managed care penetration in California, we clarified the range of practice settings by using names of local managed care clinics, public health centers, and group practices as examples.

Analysis

The independent variable was racial and ethnic background, and its analytic categories included Asian, black, Hispanic, and white. Comparisons were made between race or ethnicity and the study covariates and scores for the primary care subdomains. Frequencies of the study covariates were compared across racial and ethnic groups, and the significance of these differences was assessed with chi-squared tests of association. Generalized linear model procedures were used to assess differences in primary care quality across racial and ethnic groups after adjusting for study covariates. Bonferroni t tests were used to test significance and account for multiple comparisons.

Two total primary care scores were generated by summing the mean scores for the primary care subdomains. The first total primary care score (A) included coordination of care, a domain that was answered only by a subset of the population that reported they had visited a specialist since they first saw their regular provider. Therefore, total score A was limited to 1 subset of the population (n = 135). The second total primary care score (B) did not include coordination of care and thus included the full study sample.

Multiple linear regression analyses were conducted to predict primary care quality. Four models were constructed incrementally, with the first including only dummy variables of race and ethnicity (with white “race” as the reference group). Additional models controlled for (1) socioeconomic status covariates, (2) health system characteristics, and (3) socioeconomic status, health system characteristics, and demographics. The models were constructed separately for primary care scores A and B. Regression coefficients and respective P values are reported for race/ethnicity categories and study covariates. The coefficient of determination (R² and adjusted R²) is reported for each model to describe how much of the variance in primary care quality was explained by the study variables.

Table 2 compares the unadjusted socioeconomic status, health system characteristics, and demographic factors of our analytic sample across racial and ethnic groups. As per the sampling strategy, respondents were nearly equally divided among the 4 categories of race and ethnicity. Most respondents (74.3%) had family incomes greater than $36,000/year, although a significantly smaller proportion of black (64.2%) and Hispanic (67.7%) families had incomes above this amount compared with whites (90.2%) and Asians (84.0%; P < .001). Racial and ethnic groups also differed in maternal education and employment, with Asians reporting the highest proportion with a high school education or greater (P < .01) and blacks reporting the highest employment among mothers (P < .001).

With regard to health system factors, Asians and whites were most likely to report seeking care at a doctor’s office (58.8% and 57.0%, respectively) compared to seeking care from a health maintenance organization clinic or other setting (P < .001). Hispanics reported the largest proportion of children receiving care from a health maintenance organization clinic setting (39.4%), and Asians reported the smallest proportion (20.6%; P < .05). White respondents had the highest proportion covered by private health insurance (86.6%) and Hispanic respondents had the lowest (79.1%; P < .05). Hispanics were most likely to be uninsured (13.13%; P < .05).

Asians were most likely to report having any cost sharing such as a deductible or co-payment (83.2%; P < .01). There were no significant differences in child’s age, sex, or health status across racial and ethnic groups.

Table 3 compares adjusted primary care quality scores across racial and ethnic groups. The attribute scales were standardized by summing the responses to each question in the attribute and dividing by the number of questions (range, 1-4). In general, Asian, black, and Hispanic parents reported slightly lower quality of primary care than did whites. Minority parents reported lower scores for 6 of the 7 subdomains, although only some of the findings were significant. Asian respondents reported the lowest (or statistically equivalent to the lowest) primary care quality for 5 of the 7 domains, reflecting differences of approximately 5% to 10%. These scores were significantly lower than those reported by whites for first-contact accessibility (P < .05), first-contact utilization (P < .01), interpersonal relationship (P < .05), and comprehensiveness of services received (P < .001). Moreover, Asians reported significantly lower mean scores than did whites for both total primary care scales. Black respondents reported significantly worse first-contact utilization but slightly greater accessibility than did whites, although the difference was not significant.

Table 4 presents 4 multiple regression models for the 2 versions of total primary care quality after successive adjustment for socioeconomic status, health system characteristics, and demographics. In model 1A (not adjusted for any covariates) Asian and black races (vs whites) were significant negative predictors of primary care quality. In model 1B (ie, coordination of care domain excluded) all minority groups (vs whites) were significant negative predictors of quality (P < .01). In models 2B and 3B, after controlling for socioeconomic status and health system characteristics, respectively, 2 additional covariates positively predicted total primary care quality. These were family income greater than $36,000 (P < .01) and having a pediatrician as opposed to any other type of family practitioner or generalist primary care provider (P < .001). Despite the addition of these covariates to models 2 and 3, minority racial and ethnic groups remained significant negative predictors for both versions of primary care quality (P < .05). Asian race remained a particularly significant predictor of quality (P < .001).

In model 4A, which controlled for all covariates, older child age and, nonintuitively, being uninsured were significant positive predictors of quality (likely due to the small number of uninsured respondents). In model 4B, health status was a significant predictor of quality (P < .05). With the addition of the full complement of covariates, Hispanic and black race and ethnicity became nonsignificant in both models despite small changes in the magnitude of the coefficients and P values (.07 and .06, respectively). The loss of significance in this model is likely attributable to the number of variables that was controlled for given the moderate sample size rather than to any confounding effects of specific covariates. In model 4A, Asian race remained a significant negative predictor of quality (P < .05). In model 4B, Asian race remained a strong negative predictor of quality and having a pediatrician remained a strong positive predictor (P < .001 for both). After adjustment for the natural rise in R² associated with the inclusion of additional covariates, the final models for both total scales (A and B) explained only about 8% and 11%, respectively, of the variation in primary care quality.

Discussion

This community-based study advances the literature by demonstrating that Asian, black, and Hispanic children experience poorer quality of primary care than whites, even after controlling for many differences in socioeconomic status, health system factors, and demographics. This suggests that racial and ethnic differences in quality of care are not simple reflections of ability to pay, health disparities, or other sociodemographics.

The findings in this study that parents of minority children, in particular Asian Americans, report lower quality of primary care is consistent with previous research among adults but has not been demonstrated previously for children.^20-24 This finding is particularly important because of the growing numbers of Asian Americans in the United States and because Asian children, despite their family’s higher education, are more likely than whites and some other ethnic groups to be in fair or poor health, underimmunized, and at risk for contracting preventable illnesses such as hepatitis B.^31-33 These differences in health and health risk may be remedied in part by the receipt of high-quality primary care.³⁴

Of the primary care measures, the greatest difference between Asians and whites was in comprehensiveness of services received. This domain covered the range of services that patients could receive from their regular provider and included items such as preventive counseling and discussions about growth and development. Although language was unlikely to be a determinant of quality in this study (because we excluded those unable to complete the survey in English or Spanish), it does not discount the potential of undetected or unstudied language difficulties to enhance disparities in health care. For example, even though Asian families in our study were able to communicate sufficiently in English, they might have rated the patient-provider relationship lower because of trouble finding a provider who spoke their language. Regardless, the finding suggests the importance of making services more widely available to minority groups, including improvements in communication about existing primary care services.

An interesting secondary finding was that parents who reported a pediatrician as the child’s regular source of care reported higher quality primary care than did parents reporting other generalist providers. In particular, pediatricians appeared to perform better than other providers on 3 of the attributes: utilization of services (P < .02), patient-provider relationship (P < .0001), and services provided (P < .0001; data not shown). The differences may be attributable in part to the greater frequency of visits to pediatricians for well-child care; thus, greater opportunities may exist for delivery of preventive services and the development of the patient-provider relationship. Future research should explore the experience of minority patients receiving care from various provider specialties.

Despite significant findings, the most comprehensive regression models explained only a small proportion of the variation in primary care quality (about 11%). Other factors that may play a role in determining primary care quality, but were not included in this analysis, include health insurance plan restrictions (discussed in an upcoming paper), practice arrangements, racial concordance between the patient and provider, family mobility, and perhaps provider-specific factors such as training or years in practice.

This study has several limitations. First, the cross-sectional design and analysis allowed the demonstration of association and not of causality. Second, this study examined only 4 broad classifications of race and ethnicity that do not capture within-group variations in ethnicity or culture that could be associated with differences in quality of care received. Standard measurements of race and ethnicity also do not fully capture biologic, cultural, socioeconomic, and political aspects of multiculturalism that may interact and produce more complex findings than those reported.³⁵

Third, because of the moderate response rate, the respondents in this study may not be fully representative of the population under study. Although respondents were demographically similar to nonrespondents, participants may have been more likely than nonrespondents to have children in poorer health status or have more negative experiences with the health care system. Although this does not threaten the internal validity of this study (because response rates did not differ substantially across racial groups), such bias could lead to lower overall estimates of primary care quality regardless of racial group.

Fourth, studies that rely on patient reporting to compare quality of care across racial groups often may capture racial and ethnic group variations in perceptions of care or different standards for assessing care. In this study, we used an instrument for assessing quality of care that relies heavily on factual reporting (eg, waiting times and receipt of particular services) rather than on satisfaction or performance ratings, so our study was less subject to these biases.

In conclusion, this study demonstrated significant differences in the quality of primary care for children across racial and ethnic groups. These findings in part suggest that ensuring adequate health insurance coverage may not be sufficient to reduce racial and ethnic disparities in quality of care. Although the cause or mechanism of these disparities in quality is not entirely established, the findings encourage careful additional monitoring of the delivery of primary care, in particular to minority children. At a minimum, health care providers and organizations should make primary care services more accessible to minority families, provide the services in a culturally and linguistically competent manner (to encourage the development of the physician-patient relationship), and communicate more effectively with families about the range of child health services offered.

*The Institute of Medicine defines primary care as “the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.”⁵ The practice of primary care is best characterized as a set of attributes or functions that, only when performed together, constitute the delivery of primary care. Empirical studies have further delineated and operationalized 4 core attributes of primary care: first-contact care with a designated primary care physician; longitudinality, or ongoing care, with a physician or place of care; comprehensiveness of services; and coordination or integration of those services⁶Table 1.

Acknowledgments

The authors thank Barbara Starfield, MD, Lisa Cooper, MD, and Maria Trent, MD, for their thoughtful review of this manuscript. They also thank Jane Lyon for her generous support of and help in coordinating the study in the school district.

This study was undertaken to clarify whether pulmonary hypertension is a useful marker for underlying obstructive sleep apnea in patients with edema. Twenty-eight ambulatory adults with bilateral leg edema and a normal echocardiogram were enrolled. Sixteen subjects had pulmonary hypertension, and 12 subjects had normal pulmonary artery pressures. Spirometry, pulse oximetry on room air, and polysomnography were obtained for each subject. Ten of 16 (63%) pulmonary hypertension subjects and 9 of 12 (75%) nonpulmonary hypertension subjects had obstructive sleep apnea (P = .48). Eleven of 16 (69%) pulmonary hypertension subjects and 11 of 12 (92%) nonpulmonary hypertension subjects were obese (P = .20). If these results are generalizable, obstructive sleep apnea is frequently associated with bilateral leg edema and obesity, regardless of the presence of pulmonary hypertension. Thus, especially in obese patients, bilateral leg edema may be a useful clinical marker for underlying obstructive sleep apnea.

We previously found an association between bilateral leg edema and pulmonary hypertension in primary care patients.¹ After consideration of the differential diagnosis of pulmonary hypertension, obstructive sleep apnea was deemed the most likely explanation for the high frequency of pulmonary hypertension.² Subsequently, we identified an association among leg edema, obesity, pulmonary hypertension, and obstructive sleep apnea in ambulatory patients with normal left ventricular function.³

Our earlier data failed to clarify whether leg edema, obesity, pulmonary hypertension, or a combination thereof is the most useful marker for obstructive sleep apnea. This cross-sectional study was undertaken to determine whether subjects with bilateral leg edema and pulmonary hypertension have a higher frequency of obstructive sleep apnea than edematous subjects with normal pulmonary artery pressures.

Methods

A single physician (R.P.B.) enrolled a convenience sample of subjects from an inner city group family practice in Cleveland OH, from July 1995 to September 1997, and from a 2-physician suburban family practice near Cleveland, OH, from October 1997 to July 2000. Ambulatory patients older than 18 years with bilateral pitting leg edema, no clinically overt lung disease, no echocardiographic evidence of a cardiac abnormality, and an echocardiogram that permitted an estimation of the pulmonary artery pressure were eligible to participate in the study. The methodology for estimating the pulmonary artery pressures has been described previously.^3-5 For this study, pulmonary hypertension was defined as an estimated pulmonary artery systolic pressure > 30 mm Hg, whereas an estimated pulmonary artery systolic pressure 30 mm Hg was considered normal.

Subjects were excluded if their echocardiogram revealed valvular heart disease, congenital heart disease, or left ventricular systolic or diastolic dysfunction; if they used dihydropyridine calcium antagonists; if they had a known pulmonary condition; or if pulmonary function evaluation indicated the presence of obstructive or restrictive lung disease. Individuals with asthma were included as long as the asthma was well controlled. The protocol was approved by the Institutional Review Board at the MetroHealth Medical Center (Cleveland, OH).

The medical history of each subject was reviewed for risk factors recognized as being associated with pulmonary hypertension,³ and subjects answered the Epworth sleepiness scale questions.⁶ The percent predicted forced vital capacity (FVC), the percent predicted forced expiratory volume in 1 second (FEV₁), and the FEV₁ in relation to the FVC were determined by spirometry (Brentwood Spiroscan 2000, Hoks Electronics, Inc, Japan). Oxygen saturations on room air were determined by oximetry (N-20, Nellcor, Inc, Hayward, CA). Polysomnography was performed on all subjects in a sleep laboratory, and the average number of episodes of apneas and hypopneas per hour of sleep (apnea-hypopnea index) was calculated.

No universally accepted criteria exist for diagnosing obstructive sleep apnea.⁷ For this study, obstructive sleep apnea was defined as an apneahypopnea index of ≥ 20 events per hour,8 or a rapid eye movement-specific apnea-hypopnea index of ≥ 20 events per hour. Levels of serum albumin, antinuclear antibody, rheumatoid factor, and thyroid stimulating hormone were obtained on all subjects, as were sedimentation rate and results of liver function tests. Subjects were considered obese if they had a body mass index (weight in kg/height in m²) of more than 30 kg/m².⁹

Mean values between study groups were compared with Student’s t-test, and 2 statistics were used to compare differences between proportions. A final regression analysis was conducted to test whether controlling for potential confounding variables altered the univariate association observed. A hierarchical logistic regression analysis was performed by first regressing obstructive sleep apnea status on potential confounding variables as the first level, and then allowing pulmonary hypertension status to enter the equation as the second level. These analyses compared the extent to which pulmonary hypertension is associated with obstructive sleep apnea status before and after adjusting for confounding variables.

Results

Twenty-eight subjects enrolled in the study, 16 with pulmonary hypertension and 12 without. Findings regarding 15 of the 16 subjects with pulmonary hypertension were reported previously.³ The edema was mild (1+ or 2+ pitting) for most subjects, typically presenting as an incidental examination finding. Of the edematous patients recruited for enrollment, many more than the number who actually participated were ineligible because their echocardiograms did not allow an estimation of the pulmonary artery pressure.

Demographic information on the subjects with and without pulmonary hypertension is shown in Table 1. Subjects with pulmonary hypertension were older (mean age 63.4 ± 13.6 years versus 52.2 ± 9.9 years, P = .02). Most subjects in both groups were obese. There were no differences between the 2 groups in sex, race, education, marital status, body mass indices, or duration of edema.

Ten of 16 (63%) subjects with pulmonary hypertension and 9 of 12 (75%) subjects without pulmonary hypertension had obstructive sleep apnea (P = .48). There were no differences between the 2 groups in apnea-hypopnea indices, spirometry measurements, oxygen saturation, asthma, systemic hypertension, previous use of appetite suppressants, use of prescription medications, or Epworth sleepiness scale scores. Because Epworth sleepiness scale scores of 9 to 10 or less are considered mild,⁶ the low Epworth sleepiness scale scores in both groups indicate that many individuals with obstructive sleep apnea and edema lack symptoms of excessive daytime sleepiness. In the hierarchical logistic regression analysis, the probability associated with the adjusted regression coefficient for pulmonary hypertension status was .71, indicating that even with adjustment for potential confounding variables (age, duration of edema), there was no association between pulmonary hypertension and obstructive sleep apnea.

TABLE 1
Demographic characteristics, pathologic conditions, and laboratory data of subjects with bilateral leg edema

Discussion

We found a high prevalence of obstructive sleep apnea (68%) in patients with bilateral leg edema, most of whom were obese. The proportion of obstructive sleep apnea was high whether or not pulmonary hypertension was present. Our findings suggest that bilateral leg edema, but not pulmonary hypertension, may be a useful marker for underlying obstructive sleep apnea, especially in obese patients. Moreover, if the data are generalizable, many individuals with bilateral leg edema and normal left ventricular systolic function may be misdiagnosed or underdiagnosed as having idiopathic edema, venous insufficiency,¹ or diastolic dysfunction.¹⁰ The finding that subjects with pulmonary hypertension were older than those with normal pulmonary artery pressures suggests that either patient age or the duration of the obstructive sleep apnea may be important variables in the development of pulmonary hypertension in edematous patients with obstructive sleep apnea.

Because of the small sample, a type II error might be the explanation for the lack of difference between the pulmonary hypertension and nonpulmonary hypertension groups. Because of the small sample size and the possibility of selection bias, the results of this study should be interpreted with caution. These findings need to be replicated with a larger sample to confirm the association. In addition, further research is necessary to clarify whether leg edema, obesity, or a combination thereof is the most useful marker for obstructive sleep apnea.

If our patients are typical of those in other practices, we estimate that leg edema associated with obstructive sleep apnea occurs frequently compared with other cardiovascular diseases. In both the inner city and suburban family practices of one of the authors (R.P.B.), leg edema associated with obstructive sleep apnea is the third most common cardiovascular condition, occurring less often than systemic hypertension and coronary artery disease but more frequently than congestive heart failure, cerebrovascular accidents, or cardiac arrhythmias.

Because our experience represents primary care rather than tertiary or specialty care, and because our experience is similar in inner city and suburban settings, we believe that our experience may be generalizable to a variety of practice settings. We now practice according to the clinical dictum that for patients without symptoms or signs of congestive heart failure and without overt lung disease, bilateral leg edema represents obstructive sleep apnea until proven otherwise.

Our data raise the question of a possible causal relationship between obstructive sleep apnea and leg edema. Most of the participants in our study have not used nasal continuous positive airway pressure (CPAP) for long. However, using nightly nasal CPAP, 4 edematous patients experienced reduced leg edema, and 3 have stopped using diuretic medication (Blankfield, unpublished data). This small subset of obstructive sleep apnea patients suggests that obstructive sleep apnea may be a cause of edema.

Making a diagnosis of obstructive sleep apnea does not necessarily mean that treatment is indicated. An abnormal apnea-hypopnea index without excessive daytime sleepiness does not warrant treatment.¹¹ The results of this study have unclear clinical relevance for patients with obstructive sleep apnea and edema who lack symptoms of daytime somnolence because no study has evaluated whether treating obstructive sleep apnea alters morbidity or mortality in these individuals. Accordingly, it may be prudent for clinicians to refer edematous patients for polysomnography only if they have symptoms of excessive daytime sleepiness, desire a remedy for their edema, use diuretic medication, or develop complications of edema formation such as cellulitis, stasis dermatitis, or venous stasis ulcers.

However, if obstructive sleep apnea contributes to or causes pulmonary hypertension or edema, then it may be advisable to treat patients who have these cardiovascular complications, regardless of the presence or absence of symptoms of sleep-disordered breathing. Previous research is inconclusive regarding a causal relationship between obstructive sleep apnea and pulmonary hypertension. Most of the literature favors the premise that obstructive sleep apnea is not a cause of pulmonary hypertension,^12-17 but some studies suggest otherwise.^18,19

If subsequent research demonstrates that obstructive sleep apnea causes either pulmonary hypertension or edema, then clinical trials will be necessary to document whether morbidity and mortality rates improve after appropriate treatment of the obstructive sleep apnea. This information will be essential to determine if treatment is warranted for obstructive sleep apnea patients who have pulmonary hypertension or edema, but who lack symptoms of excessive daytime sleepiness.

Acknowledgments

The authors appreciate data collection assistance by Louise Wiatrak, MA and Simone Powers, data entry assistance by Amy Tapolyai, MBA, and Gregory Zyzanski, and manuscript assistance by Kurt Stange, MD, PhD.

This study was undertaken to clarify whether pulmonary hypertension is a useful marker for underlying obstructive sleep apnea in patients with edema. Twenty-eight ambulatory adults with bilateral leg edema and a normal echocardiogram were enrolled. Sixteen subjects had pulmonary hypertension, and 12 subjects had normal pulmonary artery pressures. Spirometry, pulse oximetry on room air, and polysomnography were obtained for each subject. Ten of 16 (63%) pulmonary hypertension subjects and 9 of 12 (75%) nonpulmonary hypertension subjects had obstructive sleep apnea (P = .48). Eleven of 16 (69%) pulmonary hypertension subjects and 11 of 12 (92%) nonpulmonary hypertension subjects were obese (P = .20). If these results are generalizable, obstructive sleep apnea is frequently associated with bilateral leg edema and obesity, regardless of the presence of pulmonary hypertension. Thus, especially in obese patients, bilateral leg edema may be a useful clinical marker for underlying obstructive sleep apnea.

We previously found an association between bilateral leg edema and pulmonary hypertension in primary care patients.¹ After consideration of the differential diagnosis of pulmonary hypertension, obstructive sleep apnea was deemed the most likely explanation for the high frequency of pulmonary hypertension.² Subsequently, we identified an association among leg edema, obesity, pulmonary hypertension, and obstructive sleep apnea in ambulatory patients with normal left ventricular function.³

Our earlier data failed to clarify whether leg edema, obesity, pulmonary hypertension, or a combination thereof is the most useful marker for obstructive sleep apnea. This cross-sectional study was undertaken to determine whether subjects with bilateral leg edema and pulmonary hypertension have a higher frequency of obstructive sleep apnea than edematous subjects with normal pulmonary artery pressures.

Methods

A single physician (R.P.B.) enrolled a convenience sample of subjects from an inner city group family practice in Cleveland OH, from July 1995 to September 1997, and from a 2-physician suburban family practice near Cleveland, OH, from October 1997 to July 2000. Ambulatory patients older than 18 years with bilateral pitting leg edema, no clinically overt lung disease, no echocardiographic evidence of a cardiac abnormality, and an echocardiogram that permitted an estimation of the pulmonary artery pressure were eligible to participate in the study. The methodology for estimating the pulmonary artery pressures has been described previously.^3-5 For this study, pulmonary hypertension was defined as an estimated pulmonary artery systolic pressure > 30 mm Hg, whereas an estimated pulmonary artery systolic pressure 30 mm Hg was considered normal.

Subjects were excluded if their echocardiogram revealed valvular heart disease, congenital heart disease, or left ventricular systolic or diastolic dysfunction; if they used dihydropyridine calcium antagonists; if they had a known pulmonary condition; or if pulmonary function evaluation indicated the presence of obstructive or restrictive lung disease. Individuals with asthma were included as long as the asthma was well controlled. The protocol was approved by the Institutional Review Board at the MetroHealth Medical Center (Cleveland, OH).

The medical history of each subject was reviewed for risk factors recognized as being associated with pulmonary hypertension,³ and subjects answered the Epworth sleepiness scale questions.⁶ The percent predicted forced vital capacity (FVC), the percent predicted forced expiratory volume in 1 second (FEV₁), and the FEV₁ in relation to the FVC were determined by spirometry (Brentwood Spiroscan 2000, Hoks Electronics, Inc, Japan). Oxygen saturations on room air were determined by oximetry (N-20, Nellcor, Inc, Hayward, CA). Polysomnography was performed on all subjects in a sleep laboratory, and the average number of episodes of apneas and hypopneas per hour of sleep (apnea-hypopnea index) was calculated.

No universally accepted criteria exist for diagnosing obstructive sleep apnea.⁷ For this study, obstructive sleep apnea was defined as an apneahypopnea index of ≥ 20 events per hour,8 or a rapid eye movement-specific apnea-hypopnea index of ≥ 20 events per hour. Levels of serum albumin, antinuclear antibody, rheumatoid factor, and thyroid stimulating hormone were obtained on all subjects, as were sedimentation rate and results of liver function tests. Subjects were considered obese if they had a body mass index (weight in kg/height in m²) of more than 30 kg/m².⁹

Mean values between study groups were compared with Student’s t-test, and 2 statistics were used to compare differences between proportions. A final regression analysis was conducted to test whether controlling for potential confounding variables altered the univariate association observed. A hierarchical logistic regression analysis was performed by first regressing obstructive sleep apnea status on potential confounding variables as the first level, and then allowing pulmonary hypertension status to enter the equation as the second level. These analyses compared the extent to which pulmonary hypertension is associated with obstructive sleep apnea status before and after adjusting for confounding variables.

Results

Twenty-eight subjects enrolled in the study, 16 with pulmonary hypertension and 12 without. Findings regarding 15 of the 16 subjects with pulmonary hypertension were reported previously.³ The edema was mild (1+ or 2+ pitting) for most subjects, typically presenting as an incidental examination finding. Of the edematous patients recruited for enrollment, many more than the number who actually participated were ineligible because their echocardiograms did not allow an estimation of the pulmonary artery pressure.

Demographic information on the subjects with and without pulmonary hypertension is shown in Table 1. Subjects with pulmonary hypertension were older (mean age 63.4 ± 13.6 years versus 52.2 ± 9.9 years, P = .02). Most subjects in both groups were obese. There were no differences between the 2 groups in sex, race, education, marital status, body mass indices, or duration of edema.

Ten of 16 (63%) subjects with pulmonary hypertension and 9 of 12 (75%) subjects without pulmonary hypertension had obstructive sleep apnea (P = .48). There were no differences between the 2 groups in apnea-hypopnea indices, spirometry measurements, oxygen saturation, asthma, systemic hypertension, previous use of appetite suppressants, use of prescription medications, or Epworth sleepiness scale scores. Because Epworth sleepiness scale scores of 9 to 10 or less are considered mild,⁶ the low Epworth sleepiness scale scores in both groups indicate that many individuals with obstructive sleep apnea and edema lack symptoms of excessive daytime sleepiness. In the hierarchical logistic regression analysis, the probability associated with the adjusted regression coefficient for pulmonary hypertension status was .71, indicating that even with adjustment for potential confounding variables (age, duration of edema), there was no association between pulmonary hypertension and obstructive sleep apnea.

TABLE 1
Demographic characteristics, pathologic conditions, and laboratory data of subjects with bilateral leg edema

Discussion

We found a high prevalence of obstructive sleep apnea (68%) in patients with bilateral leg edema, most of whom were obese. The proportion of obstructive sleep apnea was high whether or not pulmonary hypertension was present. Our findings suggest that bilateral leg edema, but not pulmonary hypertension, may be a useful marker for underlying obstructive sleep apnea, especially in obese patients. Moreover, if the data are generalizable, many individuals with bilateral leg edema and normal left ventricular systolic function may be misdiagnosed or underdiagnosed as having idiopathic edema, venous insufficiency,¹ or diastolic dysfunction.¹⁰ The finding that subjects with pulmonary hypertension were older than those with normal pulmonary artery pressures suggests that either patient age or the duration of the obstructive sleep apnea may be important variables in the development of pulmonary hypertension in edematous patients with obstructive sleep apnea.

Because of the small sample, a type II error might be the explanation for the lack of difference between the pulmonary hypertension and nonpulmonary hypertension groups. Because of the small sample size and the possibility of selection bias, the results of this study should be interpreted with caution. These findings need to be replicated with a larger sample to confirm the association. In addition, further research is necessary to clarify whether leg edema, obesity, or a combination thereof is the most useful marker for obstructive sleep apnea.

If our patients are typical of those in other practices, we estimate that leg edema associated with obstructive sleep apnea occurs frequently compared with other cardiovascular diseases. In both the inner city and suburban family practices of one of the authors (R.P.B.), leg edema associated with obstructive sleep apnea is the third most common cardiovascular condition, occurring less often than systemic hypertension and coronary artery disease but more frequently than congestive heart failure, cerebrovascular accidents, or cardiac arrhythmias.

Because our experience represents primary care rather than tertiary or specialty care, and because our experience is similar in inner city and suburban settings, we believe that our experience may be generalizable to a variety of practice settings. We now practice according to the clinical dictum that for patients without symptoms or signs of congestive heart failure and without overt lung disease, bilateral leg edema represents obstructive sleep apnea until proven otherwise.

Our data raise the question of a possible causal relationship between obstructive sleep apnea and leg edema. Most of the participants in our study have not used nasal continuous positive airway pressure (CPAP) for long. However, using nightly nasal CPAP, 4 edematous patients experienced reduced leg edema, and 3 have stopped using diuretic medication (Blankfield, unpublished data). This small subset of obstructive sleep apnea patients suggests that obstructive sleep apnea may be a cause of edema.

Making a diagnosis of obstructive sleep apnea does not necessarily mean that treatment is indicated. An abnormal apnea-hypopnea index without excessive daytime sleepiness does not warrant treatment.¹¹ The results of this study have unclear clinical relevance for patients with obstructive sleep apnea and edema who lack symptoms of daytime somnolence because no study has evaluated whether treating obstructive sleep apnea alters morbidity or mortality in these individuals. Accordingly, it may be prudent for clinicians to refer edematous patients for polysomnography only if they have symptoms of excessive daytime sleepiness, desire a remedy for their edema, use diuretic medication, or develop complications of edema formation such as cellulitis, stasis dermatitis, or venous stasis ulcers.

However, if obstructive sleep apnea contributes to or causes pulmonary hypertension or edema, then it may be advisable to treat patients who have these cardiovascular complications, regardless of the presence or absence of symptoms of sleep-disordered breathing. Previous research is inconclusive regarding a causal relationship between obstructive sleep apnea and pulmonary hypertension. Most of the literature favors the premise that obstructive sleep apnea is not a cause of pulmonary hypertension,^12-17 but some studies suggest otherwise.^18,19

If subsequent research demonstrates that obstructive sleep apnea causes either pulmonary hypertension or edema, then clinical trials will be necessary to document whether morbidity and mortality rates improve after appropriate treatment of the obstructive sleep apnea. This information will be essential to determine if treatment is warranted for obstructive sleep apnea patients who have pulmonary hypertension or edema, but who lack symptoms of excessive daytime sleepiness.

Acknowledgments

The authors appreciate data collection assistance by Louise Wiatrak, MA and Simone Powers, data entry assistance by Amy Tapolyai, MBA, and Gregory Zyzanski, and manuscript assistance by Kurt Stange, MD, PhD.

This study was undertaken to clarify whether pulmonary hypertension is a useful marker for underlying obstructive sleep apnea in patients with edema. Twenty-eight ambulatory adults with bilateral leg edema and a normal echocardiogram were enrolled. Sixteen subjects had pulmonary hypertension, and 12 subjects had normal pulmonary artery pressures. Spirometry, pulse oximetry on room air, and polysomnography were obtained for each subject. Ten of 16 (63%) pulmonary hypertension subjects and 9 of 12 (75%) nonpulmonary hypertension subjects had obstructive sleep apnea (P = .48). Eleven of 16 (69%) pulmonary hypertension subjects and 11 of 12 (92%) nonpulmonary hypertension subjects were obese (P = .20). If these results are generalizable, obstructive sleep apnea is frequently associated with bilateral leg edema and obesity, regardless of the presence of pulmonary hypertension. Thus, especially in obese patients, bilateral leg edema may be a useful clinical marker for underlying obstructive sleep apnea.

We previously found an association between bilateral leg edema and pulmonary hypertension in primary care patients.¹ After consideration of the differential diagnosis of pulmonary hypertension, obstructive sleep apnea was deemed the most likely explanation for the high frequency of pulmonary hypertension.² Subsequently, we identified an association among leg edema, obesity, pulmonary hypertension, and obstructive sleep apnea in ambulatory patients with normal left ventricular function.³

Our earlier data failed to clarify whether leg edema, obesity, pulmonary hypertension, or a combination thereof is the most useful marker for obstructive sleep apnea. This cross-sectional study was undertaken to determine whether subjects with bilateral leg edema and pulmonary hypertension have a higher frequency of obstructive sleep apnea than edematous subjects with normal pulmonary artery pressures.

Methods

A single physician (R.P.B.) enrolled a convenience sample of subjects from an inner city group family practice in Cleveland OH, from July 1995 to September 1997, and from a 2-physician suburban family practice near Cleveland, OH, from October 1997 to July 2000. Ambulatory patients older than 18 years with bilateral pitting leg edema, no clinically overt lung disease, no echocardiographic evidence of a cardiac abnormality, and an echocardiogram that permitted an estimation of the pulmonary artery pressure were eligible to participate in the study. The methodology for estimating the pulmonary artery pressures has been described previously.^3-5 For this study, pulmonary hypertension was defined as an estimated pulmonary artery systolic pressure > 30 mm Hg, whereas an estimated pulmonary artery systolic pressure 30 mm Hg was considered normal.

Subjects were excluded if their echocardiogram revealed valvular heart disease, congenital heart disease, or left ventricular systolic or diastolic dysfunction; if they used dihydropyridine calcium antagonists; if they had a known pulmonary condition; or if pulmonary function evaluation indicated the presence of obstructive or restrictive lung disease. Individuals with asthma were included as long as the asthma was well controlled. The protocol was approved by the Institutional Review Board at the MetroHealth Medical Center (Cleveland, OH).

The medical history of each subject was reviewed for risk factors recognized as being associated with pulmonary hypertension,³ and subjects answered the Epworth sleepiness scale questions.⁶ The percent predicted forced vital capacity (FVC), the percent predicted forced expiratory volume in 1 second (FEV₁), and the FEV₁ in relation to the FVC were determined by spirometry (Brentwood Spiroscan 2000, Hoks Electronics, Inc, Japan). Oxygen saturations on room air were determined by oximetry (N-20, Nellcor, Inc, Hayward, CA). Polysomnography was performed on all subjects in a sleep laboratory, and the average number of episodes of apneas and hypopneas per hour of sleep (apnea-hypopnea index) was calculated.

No universally accepted criteria exist for diagnosing obstructive sleep apnea.⁷ For this study, obstructive sleep apnea was defined as an apneahypopnea index of ≥ 20 events per hour,8 or a rapid eye movement-specific apnea-hypopnea index of ≥ 20 events per hour. Levels of serum albumin, antinuclear antibody, rheumatoid factor, and thyroid stimulating hormone were obtained on all subjects, as were sedimentation rate and results of liver function tests. Subjects were considered obese if they had a body mass index (weight in kg/height in m²) of more than 30 kg/m².⁹

Mean values between study groups were compared with Student’s t-test, and 2 statistics were used to compare differences between proportions. A final regression analysis was conducted to test whether controlling for potential confounding variables altered the univariate association observed. A hierarchical logistic regression analysis was performed by first regressing obstructive sleep apnea status on potential confounding variables as the first level, and then allowing pulmonary hypertension status to enter the equation as the second level. These analyses compared the extent to which pulmonary hypertension is associated with obstructive sleep apnea status before and after adjusting for confounding variables.

Results

Twenty-eight subjects enrolled in the study, 16 with pulmonary hypertension and 12 without. Findings regarding 15 of the 16 subjects with pulmonary hypertension were reported previously.³ The edema was mild (1+ or 2+ pitting) for most subjects, typically presenting as an incidental examination finding. Of the edematous patients recruited for enrollment, many more than the number who actually participated were ineligible because their echocardiograms did not allow an estimation of the pulmonary artery pressure.

Demographic information on the subjects with and without pulmonary hypertension is shown in Table 1. Subjects with pulmonary hypertension were older (mean age 63.4 ± 13.6 years versus 52.2 ± 9.9 years, P = .02). Most subjects in both groups were obese. There were no differences between the 2 groups in sex, race, education, marital status, body mass indices, or duration of edema.

Ten of 16 (63%) subjects with pulmonary hypertension and 9 of 12 (75%) subjects without pulmonary hypertension had obstructive sleep apnea (P = .48). There were no differences between the 2 groups in apnea-hypopnea indices, spirometry measurements, oxygen saturation, asthma, systemic hypertension, previous use of appetite suppressants, use of prescription medications, or Epworth sleepiness scale scores. Because Epworth sleepiness scale scores of 9 to 10 or less are considered mild,⁶ the low Epworth sleepiness scale scores in both groups indicate that many individuals with obstructive sleep apnea and edema lack symptoms of excessive daytime sleepiness. In the hierarchical logistic regression analysis, the probability associated with the adjusted regression coefficient for pulmonary hypertension status was .71, indicating that even with adjustment for potential confounding variables (age, duration of edema), there was no association between pulmonary hypertension and obstructive sleep apnea.

TABLE 1
Demographic characteristics, pathologic conditions, and laboratory data of subjects with bilateral leg edema

Discussion

We found a high prevalence of obstructive sleep apnea (68%) in patients with bilateral leg edema, most of whom were obese. The proportion of obstructive sleep apnea was high whether or not pulmonary hypertension was present. Our findings suggest that bilateral leg edema, but not pulmonary hypertension, may be a useful marker for underlying obstructive sleep apnea, especially in obese patients. Moreover, if the data are generalizable, many individuals with bilateral leg edema and normal left ventricular systolic function may be misdiagnosed or underdiagnosed as having idiopathic edema, venous insufficiency,¹ or diastolic dysfunction.¹⁰ The finding that subjects with pulmonary hypertension were older than those with normal pulmonary artery pressures suggests that either patient age or the duration of the obstructive sleep apnea may be important variables in the development of pulmonary hypertension in edematous patients with obstructive sleep apnea.

Because of the small sample, a type II error might be the explanation for the lack of difference between the pulmonary hypertension and nonpulmonary hypertension groups. Because of the small sample size and the possibility of selection bias, the results of this study should be interpreted with caution. These findings need to be replicated with a larger sample to confirm the association. In addition, further research is necessary to clarify whether leg edema, obesity, or a combination thereof is the most useful marker for obstructive sleep apnea.

If our patients are typical of those in other practices, we estimate that leg edema associated with obstructive sleep apnea occurs frequently compared with other cardiovascular diseases. In both the inner city and suburban family practices of one of the authors (R.P.B.), leg edema associated with obstructive sleep apnea is the third most common cardiovascular condition, occurring less often than systemic hypertension and coronary artery disease but more frequently than congestive heart failure, cerebrovascular accidents, or cardiac arrhythmias.

Because our experience represents primary care rather than tertiary or specialty care, and because our experience is similar in inner city and suburban settings, we believe that our experience may be generalizable to a variety of practice settings. We now practice according to the clinical dictum that for patients without symptoms or signs of congestive heart failure and without overt lung disease, bilateral leg edema represents obstructive sleep apnea until proven otherwise.

Our data raise the question of a possible causal relationship between obstructive sleep apnea and leg edema. Most of the participants in our study have not used nasal continuous positive airway pressure (CPAP) for long. However, using nightly nasal CPAP, 4 edematous patients experienced reduced leg edema, and 3 have stopped using diuretic medication (Blankfield, unpublished data). This small subset of obstructive sleep apnea patients suggests that obstructive sleep apnea may be a cause of edema.

Making a diagnosis of obstructive sleep apnea does not necessarily mean that treatment is indicated. An abnormal apnea-hypopnea index without excessive daytime sleepiness does not warrant treatment.¹¹ The results of this study have unclear clinical relevance for patients with obstructive sleep apnea and edema who lack symptoms of daytime somnolence because no study has evaluated whether treating obstructive sleep apnea alters morbidity or mortality in these individuals. Accordingly, it may be prudent for clinicians to refer edematous patients for polysomnography only if they have symptoms of excessive daytime sleepiness, desire a remedy for their edema, use diuretic medication, or develop complications of edema formation such as cellulitis, stasis dermatitis, or venous stasis ulcers.

However, if obstructive sleep apnea contributes to or causes pulmonary hypertension or edema, then it may be advisable to treat patients who have these cardiovascular complications, regardless of the presence or absence of symptoms of sleep-disordered breathing. Previous research is inconclusive regarding a causal relationship between obstructive sleep apnea and pulmonary hypertension. Most of the literature favors the premise that obstructive sleep apnea is not a cause of pulmonary hypertension,^12-17 but some studies suggest otherwise.^18,19

If subsequent research demonstrates that obstructive sleep apnea causes either pulmonary hypertension or edema, then clinical trials will be necessary to document whether morbidity and mortality rates improve after appropriate treatment of the obstructive sleep apnea. This information will be essential to determine if treatment is warranted for obstructive sleep apnea patients who have pulmonary hypertension or edema, but who lack symptoms of excessive daytime sleepiness.

Acknowledgments

The authors appreciate data collection assistance by Louise Wiatrak, MA and Simone Powers, data entry assistance by Amy Tapolyai, MBA, and Gregory Zyzanski, and manuscript assistance by Kurt Stange, MD, PhD.

Many general practitioners believe their patients benefit from preventive health care and, as a result, many concentrate on identifying and treating risk factors for coronary heart disease (CHD),^1,2 as many studies show that intervention can reduce risk.^3-6 Other studies have suggested that such intervention results in only modest improvements in the risk profile of the general population,^7-12 which raises questions about the efficacy of preventive health care.^13,16 As of the early 1990s, few randomized, controlled, long-term trials have documented the effect of health screening as a primary prevention tool in reducing cardiovascular risk in the general population.^17,18 In earlier large-scale studies on multiple-riskfactor intervention, interventions were not restricted to the intervention groups (controls received similar interventions to some extent); moreover, the studies contained other methodological problems that may have minimized the outcomes between control and intervention groups.^19-21

This study was inspired by a Danish trial²² that focused not only on the prevention of CHD, but on preventing general health problems using lifestyle changes as the primary intervention tool.²³ During the 1990s, results from 2 studies using different, though comparable, randomized designs were published.^7-10 These studies focused more narrowly on the prevention of CHD^8,17,18 and only 1 study had follow-up of more than 1 year. Relevant studies of the impact of intervention, therefore, are still lacking.

This article reports on the impact of general health screenings and health discussions with general practitioners on the cardiovascular risk profile of an unrandomized population. Other aspects of the study have been reported elsewhere.^24-32

Methods

Setting and participants

The study took place in the district of Ebeltoft, Aarhus County, Denmark, a rural area with a total population of approximately 13,000. All 9 general practitioners from the district participated. Before the study began, the general practitioners participated in 4 meetings on prevention of heart and lung disease, dietary advice, and engaging in health discussions with patients.

Of 3464 inhabitants aged 30 to 49 years by January 1, 1991, and registered with a local general practitioner, a random sample of 2000 (57.7%) were invited to participate in the study. An employee of Aarhus County who was not otherwise involved in the study selected participants by birth dates. Registration with a general practitioner gives free access to medical services and is available to all Danish citizens. The 3464 persons from whom the participants were drawn constituted 87% of the entire population in the selected age group.

In September 1991, the 2000 persons received an invitation to participate along with a questionnaire about general demographic information and lifestyle, signed by their general practitioner. All who agreed to participate received an extensive supplementary baseline questionnaire with detailed questions that evaluated the participant’s health, lifestyle, psychosocial status, and psychosocial life events. Participants were informed by their general practitioner about which intervention they would be offered.

Randomization

Participants were randomly assigned to 1 of 3 groups by proportional, stratified randomization based on the general practitioner with whom they were registered, their sex, age, cohabitation status, and body mass index (BMI). All 3 groups received questionnaires. Health screenings were offered to 2 of the groups and follow-up health discussions with the general practitioner were offered to participants in only 1 of the 2 intervention groups. An employee of Aarhus County who was not otherwise involved in the study carried out the randomization.

Health screenings

Participants were given a multiphasic, broad-spectrum screening. This included a calculation of cardiovascular risk score (CRS), giving an estimate of the risk of premature cardiovascular disease for each individual. Figure 1 shows the calculation of CRS based on sex, familial inheritance (number of family members with ischemic heart disease before age 55), tobacco use, blood pressure, serum cholesterol (total), and BMI³³ and the subsequent division into risk groups. Baseline health screenings were performed by 3 laboratory assistants between December 1991 and June 1992 and took place in the town of Ebeltoft, in the central clinic which 5 of the general practitioners shared. A few weeks after the health screening, all those tested received personal written feedback from their general practitioners. Where values fell outside the normal range, the feedback included advice relating primarily to lifestyle changes. All participants who had been advised that they had an elevated or high CRS were encouraged to see their general practitioner, regardless of their randomization group. All tested participants also received pamphlets on leading a healthy lifestyle from the Danish Heart Foundation.

FIGURE 1
Calculation of cardiovascular risk score

Health discussions

A 45-minute consultation with their own general practitioner was offered to participants from the health screening plus discussion group. Prior to the consultation, the participants completed a short questionnaire about suitable topics for discussion. At the end of the consultation, general practitioners invited participants to set a maximum of 3 health-related lifestyle goals for the following year. In cooperation with the participant, general practitioners then recorded these goals in a separate questionnaire.

Follow-up

Follow-up took place 1 and 5 years after the baseline intervention. Participants received follow-up questionnaires and were offered health screenings and health discussions according to their group of randomization. Participants in the health screening plus discussion group were offered annual consultations. The control group was promised a health screening and a health discussion at the end of the study period. Other details of the design are outlined elsewhere.²³

Data analysis and statistics

SPSS version 9.0 for Windows was used to analyze results. Double data entry was used for the laboratory tests. Differences between groups were evaluated by 2^- test for categorical data, by t-test for means, and by nonparametric testing for nonparametric data. Ninety-five percent confidence intervals (95% CI) were applied to relative risk (RR) values. Information was used from the baseline questionnaires to identify baseline risk groups among all those randomized. At the 5-year follow-up, randomized groups were compared according to the intention-to-treat rule (ie, regardless of their compliance with the intervention program).

Results

Participation at baseline and follow-up

Seventy-five percent (1507) of the 2000 persons invited to participate agreed to take part in the study. The percentage was higher among women (80.0%) than among men (71.0%).

Table 1shows the distribution of sociodemographic and cardiovascular risk factors at baseline among the randomized groups. No significant differences between groups were found. General practitioners advised 11.4% (103 persons) of the 905 tested in the intervention groups that they had an elevated or high CRS (≥10) at baseline. Of these, 52 belonged to the health screening group and 51 to the health screening plus discussion group. Prior to the test almost all participants were unaware of any existing cardiovascular disease.

Of the 443 persons in the health screening plus discussion group who accepted the offer of a consultation, 307 (69.3%) (95% CI, 64.8%–73.6%) decided to change their lifestyle in 1 or more respects. The number was significantly higher among those who had been advised of an elevated cardiovascular risk and who accepted the offer of a health discussion: 46 of 51 (90.2%) (95% CI, 78.6%–96.7%). In decreasing frequency, the goals set related to weight (63%) (95% CI, 47.5%–76.8%), diet (50.0%) (95% CI, 34.9%–65.1%), physical activity (50.0%) (95% CI, 34.9%–65.1%), smoking (43.5%) (95% CI 28.9%–58.9%), alcohol use (17.4%) (95% CI 7.8%–31.4%), and work (13.0%) (95% CI, 4.9%–26.3%). Emotional well-being, drug treatment, and other subjects (in each case by 2 different participants) were also discussed.

Figure 2 presents the flowchart of the study, focusing on participation in the health screenings. For the health discussions, the participation rate at baseline (1992) was very high. However, interest declined markedly in the follow-up period. Among baseline participants in the health screening plus discussion group, the percentage who agreed to the follow-up consultations was 97.1% in 1992, 35.7% in 1993, 16.9% in 1994, 15.1% in 1995, 8.6% in 1996, and 7.0% in 1997 (87.9% in 1992, 32.3% in 1993, 15.3% in 1994, 13.7% in 1995, 7.7% in 1996, and 6.5% in 1997 of all those randomized into the health screening plus discussion group). In total, 88.9% of those randomized into the health screening plus discussion group had at least 1 health discussion, 45.2% had at least 2 discussions, and 18.1% had at least 3 discussions.

TABLE 1
Baseline demographics and cardiovascular risk factors

FIGURE 2
Flowchart of participation in The Ebeltoft Health Promotion Study, focusing on participation in the health screenings

Impact on cardiovascular risk

Table 2 shows the mean CRS and other cardiovascular risk factors at the 5-year follow-up. No significant differences were noted in any of the measures between the 2 intervention groups; therefore, data from these 2 groups are presented together. In comparison to the control group, participants in the intervention groups have a significantly lower CRS, BMI, and serum cholesterol level after 5 years. There were no significant differences between the control and intervention groups in terms of blood pressure. Differences between the control and the intervention groups are more pronounced among the baseline risk groups. Smoking and CO concentration were not significantly affected overall or between risk groups.

Table 3 shows a marked difference between the control and the intervention groups in the prevalence of persons with elevated CRS at the 5-year follow-up. The RR is reduced to about half—at the 5-year follow-up the prevalence of those with elevated CRS in the intervention groups is approximately half that in the control group. The absolute risk reduction is 8.6% (number needed to treat = 11.6). The same pattern is evident among baseline risk groups—the RR of having elevated CRS is reduced to about half, but with larger absolute risk reductions.

According to self-reported data at the 5-year follow-up, the positive impact on cardiovascular risk factors was not a result of medication. In the control group, 6.8% were using blood pressure medicine, compared to 4.8% in the intervention groups; 1.0% of the control group and 0.9% of the intervention groups were on heart medication, and 3.9% of the control group and 3.7% of the intervention groups were on diuretic medication.

TABLE 2
Cardiovascular risk score and other cardiovascular risk factors after 5 years of follow-up

TABLE 3
Prevalence proportion and relative risk of having elevated cardiovascular risk score or other cardiovascular risk factors, after 5 years of follow-up

Discussion

This study is the first to present 5-year follow-up results from a randomized controlled trial showing the impact of general health screenings and discussions with general practitioners on the cardiovascular risk profile of a general population. The intervention had a modest impact on mean CRS in the general population, and a marked impact on the prevalence of those who were at cardiovascular risk. The impact was significantly greater for groups at cardiovascular risk; the relative risk reduction was approximately the same in those at risk as those not at risk, but with larger absolute risk reductions. The study does not indicate whether the reduction in CRS factors will result in reduced morbidity or mortality.

At the 5-year follow-up there was no difference between the CRS in the health screening plus discussion group and the screening only group. The discussion alone had no discernible impact. Several factors, however, may obscure the role of the discussions with general practitioners in this study. For ethical reasons, all persons advised of an elevated cardiovascular risk were offered a consultation with their general practitioner, regardless of their intervention group. Although consultations in such cases were probably not as extensive and detailed as those offered as part of the study, they may confound the difference in the degree of intervention between the 2 groups. The Danish Health System ensures that all participants can see their own general practitioner at no cost whenever they wish. Participants who were not offered a health discussion as part of the study may nevertheless have taken advantage of this free system to consult their general practitioner, especially if they were advised to do so. Moreover, the low rate of participation after the primary health discussion weakens the strength of the intervention in the health screening plus discussion group. Although the study thus does not provide evidence that such discussions played an essential role in the intervention, health screenings alone may not achieve the same impact. The psychological impact of the intervention may also be different for those who had personal consultations with their general practitioner after the health screening. The BMI values included a few individuals with unhealthily low BMI (<19). The distribution was not significantly different between the groups, although a tendency for an unhealthily low BMI of a slightly greater number of patients was seen in the intervention groups, highlighting the fact that weight loss is not always a relevant factor. Focusing on lifestyle changes might trigger some individuals to indulge in anorexic attitudes and behavior.

The results indicate that health screenings should be both population-based and individually oriented, and that general practitioners should be involved. The population screening is necessary to identify those at risk, since almost none of those with elevated cardiovascular risk were aware of their condition prior to screening. The fact that the general practitioner personally contacted the participants may have increased the participation rate, which is high in this study. In the written feedback after the screenings, general practitioners adjusted their advice to individual participants according to test results, and where appropriate advised them to come in for a personal consultation.

For several reasons, the impact of the intervention—both health screenings and discussions—may be greater than our findings suggest. We cannot measure the impact of the questionnaires on the control group—a methodological problem which also affected the OXCHECK study.^7,8 In the British Family Heart Study,^9,10 the control group was apparently unaffected, but the design of that study makes it impossible to assess the impact of subsequent intervention on baseline risk groups. Moreover, the fact that all the participants in the present study live in a small community may reduce the differences in degrees of intervention among the groups, although this is partially addressed by placing cohabiting couples in the same intervention group. Contact among patients within the various clinics involved may also have blurred the differences between the intervention groups.

In the present study, the general practitioners were not trained in any specific psychotherapeutic method for conducting the health discussions. The low rate of participation in follow-up consultations suggests a need to find better methods of motivating participants. Training general practitioners to use motivational discussions to inspire behavioral change, for example, might increase the impact of the intervention.^3,4 Counseling to trigger changes in attitude and behavior, particularly when modified to the individual’s readiness to change, might be more effective than a traditional health discussion focusing mainly on various risk factors.

Important findings from this study are that a major part of the population is interested in having health screenings and discussions with their general practitioner, although interest declines rapidly; that individuals with elevated risk of coronary heart disease set relevant goals for themselves for lifestyle changes; and that cardiovascular risk after 5 years of follow-up is reduced. Planned health discussions about the health screening results do not seem to reduce cardiovascular risk.

Acknowledgments

The following general practitioners participated in the study: A. Bøgedal, P. Grønbæk, L. Jørgensen, P.T. Jørgensen, H. Lundberg, J.M. Nielsen, G.S. Pedersen, J.C. Rahbek, and N. Bie. We thank the staff at the general practitioners clinic in Ebeltoft for their extraordinary efforts, including the extensive and brilliant administrative assistance given by A. Hilligsøe and E. Therkildsen. Thanks also to A. Brock, head of the laboratory at Randers Central Hospital for analysis of blood and urine, and Sally Laird for revision of English texts relating to the study.

Many general practitioners believe their patients benefit from preventive health care and, as a result, many concentrate on identifying and treating risk factors for coronary heart disease (CHD),^1,2 as many studies show that intervention can reduce risk.^3-6 Other studies have suggested that such intervention results in only modest improvements in the risk profile of the general population,^7-12 which raises questions about the efficacy of preventive health care.^13,16 As of the early 1990s, few randomized, controlled, long-term trials have documented the effect of health screening as a primary prevention tool in reducing cardiovascular risk in the general population.^17,18 In earlier large-scale studies on multiple-riskfactor intervention, interventions were not restricted to the intervention groups (controls received similar interventions to some extent); moreover, the studies contained other methodological problems that may have minimized the outcomes between control and intervention groups.^19-21

This study was inspired by a Danish trial²² that focused not only on the prevention of CHD, but on preventing general health problems using lifestyle changes as the primary intervention tool.²³ During the 1990s, results from 2 studies using different, though comparable, randomized designs were published.^7-10 These studies focused more narrowly on the prevention of CHD^8,17,18 and only 1 study had follow-up of more than 1 year. Relevant studies of the impact of intervention, therefore, are still lacking.

This article reports on the impact of general health screenings and health discussions with general practitioners on the cardiovascular risk profile of an unrandomized population. Other aspects of the study have been reported elsewhere.^24-32

Methods

Setting and participants

The study took place in the district of Ebeltoft, Aarhus County, Denmark, a rural area with a total population of approximately 13,000. All 9 general practitioners from the district participated. Before the study began, the general practitioners participated in 4 meetings on prevention of heart and lung disease, dietary advice, and engaging in health discussions with patients.

Of 3464 inhabitants aged 30 to 49 years by January 1, 1991, and registered with a local general practitioner, a random sample of 2000 (57.7%) were invited to participate in the study. An employee of Aarhus County who was not otherwise involved in the study selected participants by birth dates. Registration with a general practitioner gives free access to medical services and is available to all Danish citizens. The 3464 persons from whom the participants were drawn constituted 87% of the entire population in the selected age group.

In September 1991, the 2000 persons received an invitation to participate along with a questionnaire about general demographic information and lifestyle, signed by their general practitioner. All who agreed to participate received an extensive supplementary baseline questionnaire with detailed questions that evaluated the participant’s health, lifestyle, psychosocial status, and psychosocial life events. Participants were informed by their general practitioner about which intervention they would be offered.

Randomization

Participants were randomly assigned to 1 of 3 groups by proportional, stratified randomization based on the general practitioner with whom they were registered, their sex, age, cohabitation status, and body mass index (BMI). All 3 groups received questionnaires. Health screenings were offered to 2 of the groups and follow-up health discussions with the general practitioner were offered to participants in only 1 of the 2 intervention groups. An employee of Aarhus County who was not otherwise involved in the study carried out the randomization.

Health screenings

Participants were given a multiphasic, broad-spectrum screening. This included a calculation of cardiovascular risk score (CRS), giving an estimate of the risk of premature cardiovascular disease for each individual. Figure 1 shows the calculation of CRS based on sex, familial inheritance (number of family members with ischemic heart disease before age 55), tobacco use, blood pressure, serum cholesterol (total), and BMI³³ and the subsequent division into risk groups. Baseline health screenings were performed by 3 laboratory assistants between December 1991 and June 1992 and took place in the town of Ebeltoft, in the central clinic which 5 of the general practitioners shared. A few weeks after the health screening, all those tested received personal written feedback from their general practitioners. Where values fell outside the normal range, the feedback included advice relating primarily to lifestyle changes. All participants who had been advised that they had an elevated or high CRS were encouraged to see their general practitioner, regardless of their randomization group. All tested participants also received pamphlets on leading a healthy lifestyle from the Danish Heart Foundation.

FIGURE 1
Calculation of cardiovascular risk score

Health discussions

A 45-minute consultation with their own general practitioner was offered to participants from the health screening plus discussion group. Prior to the consultation, the participants completed a short questionnaire about suitable topics for discussion. At the end of the consultation, general practitioners invited participants to set a maximum of 3 health-related lifestyle goals for the following year. In cooperation with the participant, general practitioners then recorded these goals in a separate questionnaire.

Follow-up

Follow-up took place 1 and 5 years after the baseline intervention. Participants received follow-up questionnaires and were offered health screenings and health discussions according to their group of randomization. Participants in the health screening plus discussion group were offered annual consultations. The control group was promised a health screening and a health discussion at the end of the study period. Other details of the design are outlined elsewhere.²³

Data analysis and statistics

SPSS version 9.0 for Windows was used to analyze results. Double data entry was used for the laboratory tests. Differences between groups were evaluated by 2^- test for categorical data, by t-test for means, and by nonparametric testing for nonparametric data. Ninety-five percent confidence intervals (95% CI) were applied to relative risk (RR) values. Information was used from the baseline questionnaires to identify baseline risk groups among all those randomized. At the 5-year follow-up, randomized groups were compared according to the intention-to-treat rule (ie, regardless of their compliance with the intervention program).

Results

Participation at baseline and follow-up

Seventy-five percent (1507) of the 2000 persons invited to participate agreed to take part in the study. The percentage was higher among women (80.0%) than among men (71.0%).

Table 1shows the distribution of sociodemographic and cardiovascular risk factors at baseline among the randomized groups. No significant differences between groups were found. General practitioners advised 11.4% (103 persons) of the 905 tested in the intervention groups that they had an elevated or high CRS (≥10) at baseline. Of these, 52 belonged to the health screening group and 51 to the health screening plus discussion group. Prior to the test almost all participants were unaware of any existing cardiovascular disease.

Of the 443 persons in the health screening plus discussion group who accepted the offer of a consultation, 307 (69.3%) (95% CI, 64.8%–73.6%) decided to change their lifestyle in 1 or more respects. The number was significantly higher among those who had been advised of an elevated cardiovascular risk and who accepted the offer of a health discussion: 46 of 51 (90.2%) (95% CI, 78.6%–96.7%). In decreasing frequency, the goals set related to weight (63%) (95% CI, 47.5%–76.8%), diet (50.0%) (95% CI, 34.9%–65.1%), physical activity (50.0%) (95% CI, 34.9%–65.1%), smoking (43.5%) (95% CI 28.9%–58.9%), alcohol use (17.4%) (95% CI 7.8%–31.4%), and work (13.0%) (95% CI, 4.9%–26.3%). Emotional well-being, drug treatment, and other subjects (in each case by 2 different participants) were also discussed.

Figure 2 presents the flowchart of the study, focusing on participation in the health screenings. For the health discussions, the participation rate at baseline (1992) was very high. However, interest declined markedly in the follow-up period. Among baseline participants in the health screening plus discussion group, the percentage who agreed to the follow-up consultations was 97.1% in 1992, 35.7% in 1993, 16.9% in 1994, 15.1% in 1995, 8.6% in 1996, and 7.0% in 1997 (87.9% in 1992, 32.3% in 1993, 15.3% in 1994, 13.7% in 1995, 7.7% in 1996, and 6.5% in 1997 of all those randomized into the health screening plus discussion group). In total, 88.9% of those randomized into the health screening plus discussion group had at least 1 health discussion, 45.2% had at least 2 discussions, and 18.1% had at least 3 discussions.

TABLE 1
Baseline demographics and cardiovascular risk factors

FIGURE 2
Flowchart of participation in The Ebeltoft Health Promotion Study, focusing on participation in the health screenings

Impact on cardiovascular risk

Table 2 shows the mean CRS and other cardiovascular risk factors at the 5-year follow-up. No significant differences were noted in any of the measures between the 2 intervention groups; therefore, data from these 2 groups are presented together. In comparison to the control group, participants in the intervention groups have a significantly lower CRS, BMI, and serum cholesterol level after 5 years. There were no significant differences between the control and intervention groups in terms of blood pressure. Differences between the control and the intervention groups are more pronounced among the baseline risk groups. Smoking and CO concentration were not significantly affected overall or between risk groups.

Table 3 shows a marked difference between the control and the intervention groups in the prevalence of persons with elevated CRS at the 5-year follow-up. The RR is reduced to about half—at the 5-year follow-up the prevalence of those with elevated CRS in the intervention groups is approximately half that in the control group. The absolute risk reduction is 8.6% (number needed to treat = 11.6). The same pattern is evident among baseline risk groups—the RR of having elevated CRS is reduced to about half, but with larger absolute risk reductions.

According to self-reported data at the 5-year follow-up, the positive impact on cardiovascular risk factors was not a result of medication. In the control group, 6.8% were using blood pressure medicine, compared to 4.8% in the intervention groups; 1.0% of the control group and 0.9% of the intervention groups were on heart medication, and 3.9% of the control group and 3.7% of the intervention groups were on diuretic medication.

TABLE 2
Cardiovascular risk score and other cardiovascular risk factors after 5 years of follow-up

TABLE 3
Prevalence proportion and relative risk of having elevated cardiovascular risk score or other cardiovascular risk factors, after 5 years of follow-up

Discussion

This study is the first to present 5-year follow-up results from a randomized controlled trial showing the impact of general health screenings and discussions with general practitioners on the cardiovascular risk profile of a general population. The intervention had a modest impact on mean CRS in the general population, and a marked impact on the prevalence of those who were at cardiovascular risk. The impact was significantly greater for groups at cardiovascular risk; the relative risk reduction was approximately the same in those at risk as those not at risk, but with larger absolute risk reductions. The study does not indicate whether the reduction in CRS factors will result in reduced morbidity or mortality.

At the 5-year follow-up there was no difference between the CRS in the health screening plus discussion group and the screening only group. The discussion alone had no discernible impact. Several factors, however, may obscure the role of the discussions with general practitioners in this study. For ethical reasons, all persons advised of an elevated cardiovascular risk were offered a consultation with their general practitioner, regardless of their intervention group. Although consultations in such cases were probably not as extensive and detailed as those offered as part of the study, they may confound the difference in the degree of intervention between the 2 groups. The Danish Health System ensures that all participants can see their own general practitioner at no cost whenever they wish. Participants who were not offered a health discussion as part of the study may nevertheless have taken advantage of this free system to consult their general practitioner, especially if they were advised to do so. Moreover, the low rate of participation after the primary health discussion weakens the strength of the intervention in the health screening plus discussion group. Although the study thus does not provide evidence that such discussions played an essential role in the intervention, health screenings alone may not achieve the same impact. The psychological impact of the intervention may also be different for those who had personal consultations with their general practitioner after the health screening. The BMI values included a few individuals with unhealthily low BMI (<19). The distribution was not significantly different between the groups, although a tendency for an unhealthily low BMI of a slightly greater number of patients was seen in the intervention groups, highlighting the fact that weight loss is not always a relevant factor. Focusing on lifestyle changes might trigger some individuals to indulge in anorexic attitudes and behavior.

The results indicate that health screenings should be both population-based and individually oriented, and that general practitioners should be involved. The population screening is necessary to identify those at risk, since almost none of those with elevated cardiovascular risk were aware of their condition prior to screening. The fact that the general practitioner personally contacted the participants may have increased the participation rate, which is high in this study. In the written feedback after the screenings, general practitioners adjusted their advice to individual participants according to test results, and where appropriate advised them to come in for a personal consultation.

For several reasons, the impact of the intervention—both health screenings and discussions—may be greater than our findings suggest. We cannot measure the impact of the questionnaires on the control group—a methodological problem which also affected the OXCHECK study.^7,8 In the British Family Heart Study,^9,10 the control group was apparently unaffected, but the design of that study makes it impossible to assess the impact of subsequent intervention on baseline risk groups. Moreover, the fact that all the participants in the present study live in a small community may reduce the differences in degrees of intervention among the groups, although this is partially addressed by placing cohabiting couples in the same intervention group. Contact among patients within the various clinics involved may also have blurred the differences between the intervention groups.

In the present study, the general practitioners were not trained in any specific psychotherapeutic method for conducting the health discussions. The low rate of participation in follow-up consultations suggests a need to find better methods of motivating participants. Training general practitioners to use motivational discussions to inspire behavioral change, for example, might increase the impact of the intervention.^3,4 Counseling to trigger changes in attitude and behavior, particularly when modified to the individual’s readiness to change, might be more effective than a traditional health discussion focusing mainly on various risk factors.

Important findings from this study are that a major part of the population is interested in having health screenings and discussions with their general practitioner, although interest declines rapidly; that individuals with elevated risk of coronary heart disease set relevant goals for themselves for lifestyle changes; and that cardiovascular risk after 5 years of follow-up is reduced. Planned health discussions about the health screening results do not seem to reduce cardiovascular risk.

Acknowledgments

The following general practitioners participated in the study: A. Bøgedal, P. Grønbæk, L. Jørgensen, P.T. Jørgensen, H. Lundberg, J.M. Nielsen, G.S. Pedersen, J.C. Rahbek, and N. Bie. We thank the staff at the general practitioners clinic in Ebeltoft for their extraordinary efforts, including the extensive and brilliant administrative assistance given by A. Hilligsøe and E. Therkildsen. Thanks also to A. Brock, head of the laboratory at Randers Central Hospital for analysis of blood and urine, and Sally Laird for revision of English texts relating to the study.

Many general practitioners believe their patients benefit from preventive health care and, as a result, many concentrate on identifying and treating risk factors for coronary heart disease (CHD),^1,2 as many studies show that intervention can reduce risk.^3-6 Other studies have suggested that such intervention results in only modest improvements in the risk profile of the general population,^7-12 which raises questions about the efficacy of preventive health care.^13,16 As of the early 1990s, few randomized, controlled, long-term trials have documented the effect of health screening as a primary prevention tool in reducing cardiovascular risk in the general population.^17,18 In earlier large-scale studies on multiple-riskfactor intervention, interventions were not restricted to the intervention groups (controls received similar interventions to some extent); moreover, the studies contained other methodological problems that may have minimized the outcomes between control and intervention groups.^19-21

This study was inspired by a Danish trial²² that focused not only on the prevention of CHD, but on preventing general health problems using lifestyle changes as the primary intervention tool.²³ During the 1990s, results from 2 studies using different, though comparable, randomized designs were published.^7-10 These studies focused more narrowly on the prevention of CHD^8,17,18 and only 1 study had follow-up of more than 1 year. Relevant studies of the impact of intervention, therefore, are still lacking.

This article reports on the impact of general health screenings and health discussions with general practitioners on the cardiovascular risk profile of an unrandomized population. Other aspects of the study have been reported elsewhere.^24-32

Methods

Setting and participants

The study took place in the district of Ebeltoft, Aarhus County, Denmark, a rural area with a total population of approximately 13,000. All 9 general practitioners from the district participated. Before the study began, the general practitioners participated in 4 meetings on prevention of heart and lung disease, dietary advice, and engaging in health discussions with patients.

Of 3464 inhabitants aged 30 to 49 years by January 1, 1991, and registered with a local general practitioner, a random sample of 2000 (57.7%) were invited to participate in the study. An employee of Aarhus County who was not otherwise involved in the study selected participants by birth dates. Registration with a general practitioner gives free access to medical services and is available to all Danish citizens. The 3464 persons from whom the participants were drawn constituted 87% of the entire population in the selected age group.

In September 1991, the 2000 persons received an invitation to participate along with a questionnaire about general demographic information and lifestyle, signed by their general practitioner. All who agreed to participate received an extensive supplementary baseline questionnaire with detailed questions that evaluated the participant’s health, lifestyle, psychosocial status, and psychosocial life events. Participants were informed by their general practitioner about which intervention they would be offered.

Randomization

Participants were randomly assigned to 1 of 3 groups by proportional, stratified randomization based on the general practitioner with whom they were registered, their sex, age, cohabitation status, and body mass index (BMI). All 3 groups received questionnaires. Health screenings were offered to 2 of the groups and follow-up health discussions with the general practitioner were offered to participants in only 1 of the 2 intervention groups. An employee of Aarhus County who was not otherwise involved in the study carried out the randomization.

Health screenings

Participants were given a multiphasic, broad-spectrum screening. This included a calculation of cardiovascular risk score (CRS), giving an estimate of the risk of premature cardiovascular disease for each individual. Figure 1 shows the calculation of CRS based on sex, familial inheritance (number of family members with ischemic heart disease before age 55), tobacco use, blood pressure, serum cholesterol (total), and BMI³³ and the subsequent division into risk groups. Baseline health screenings were performed by 3 laboratory assistants between December 1991 and June 1992 and took place in the town of Ebeltoft, in the central clinic which 5 of the general practitioners shared. A few weeks after the health screening, all those tested received personal written feedback from their general practitioners. Where values fell outside the normal range, the feedback included advice relating primarily to lifestyle changes. All participants who had been advised that they had an elevated or high CRS were encouraged to see their general practitioner, regardless of their randomization group. All tested participants also received pamphlets on leading a healthy lifestyle from the Danish Heart Foundation.

FIGURE 1
Calculation of cardiovascular risk score

Health discussions

A 45-minute consultation with their own general practitioner was offered to participants from the health screening plus discussion group. Prior to the consultation, the participants completed a short questionnaire about suitable topics for discussion. At the end of the consultation, general practitioners invited participants to set a maximum of 3 health-related lifestyle goals for the following year. In cooperation with the participant, general practitioners then recorded these goals in a separate questionnaire.

Follow-up

Follow-up took place 1 and 5 years after the baseline intervention. Participants received follow-up questionnaires and were offered health screenings and health discussions according to their group of randomization. Participants in the health screening plus discussion group were offered annual consultations. The control group was promised a health screening and a health discussion at the end of the study period. Other details of the design are outlined elsewhere.²³

Data analysis and statistics

SPSS version 9.0 for Windows was used to analyze results. Double data entry was used for the laboratory tests. Differences between groups were evaluated by 2^- test for categorical data, by t-test for means, and by nonparametric testing for nonparametric data. Ninety-five percent confidence intervals (95% CI) were applied to relative risk (RR) values. Information was used from the baseline questionnaires to identify baseline risk groups among all those randomized. At the 5-year follow-up, randomized groups were compared according to the intention-to-treat rule (ie, regardless of their compliance with the intervention program).

Results

Participation at baseline and follow-up

Seventy-five percent (1507) of the 2000 persons invited to participate agreed to take part in the study. The percentage was higher among women (80.0%) than among men (71.0%).

Table 1shows the distribution of sociodemographic and cardiovascular risk factors at baseline among the randomized groups. No significant differences between groups were found. General practitioners advised 11.4% (103 persons) of the 905 tested in the intervention groups that they had an elevated or high CRS (≥10) at baseline. Of these, 52 belonged to the health screening group and 51 to the health screening plus discussion group. Prior to the test almost all participants were unaware of any existing cardiovascular disease.

Of the 443 persons in the health screening plus discussion group who accepted the offer of a consultation, 307 (69.3%) (95% CI, 64.8%–73.6%) decided to change their lifestyle in 1 or more respects. The number was significantly higher among those who had been advised of an elevated cardiovascular risk and who accepted the offer of a health discussion: 46 of 51 (90.2%) (95% CI, 78.6%–96.7%). In decreasing frequency, the goals set related to weight (63%) (95% CI, 47.5%–76.8%), diet (50.0%) (95% CI, 34.9%–65.1%), physical activity (50.0%) (95% CI, 34.9%–65.1%), smoking (43.5%) (95% CI 28.9%–58.9%), alcohol use (17.4%) (95% CI 7.8%–31.4%), and work (13.0%) (95% CI, 4.9%–26.3%). Emotional well-being, drug treatment, and other subjects (in each case by 2 different participants) were also discussed.

Figure 2 presents the flowchart of the study, focusing on participation in the health screenings. For the health discussions, the participation rate at baseline (1992) was very high. However, interest declined markedly in the follow-up period. Among baseline participants in the health screening plus discussion group, the percentage who agreed to the follow-up consultations was 97.1% in 1992, 35.7% in 1993, 16.9% in 1994, 15.1% in 1995, 8.6% in 1996, and 7.0% in 1997 (87.9% in 1992, 32.3% in 1993, 15.3% in 1994, 13.7% in 1995, 7.7% in 1996, and 6.5% in 1997 of all those randomized into the health screening plus discussion group). In total, 88.9% of those randomized into the health screening plus discussion group had at least 1 health discussion, 45.2% had at least 2 discussions, and 18.1% had at least 3 discussions.

TABLE 1
Baseline demographics and cardiovascular risk factors

FIGURE 2
Flowchart of participation in The Ebeltoft Health Promotion Study, focusing on participation in the health screenings

Impact on cardiovascular risk

Table 2 shows the mean CRS and other cardiovascular risk factors at the 5-year follow-up. No significant differences were noted in any of the measures between the 2 intervention groups; therefore, data from these 2 groups are presented together. In comparison to the control group, participants in the intervention groups have a significantly lower CRS, BMI, and serum cholesterol level after 5 years. There were no significant differences between the control and intervention groups in terms of blood pressure. Differences between the control and the intervention groups are more pronounced among the baseline risk groups. Smoking and CO concentration were not significantly affected overall or between risk groups.

Table 3 shows a marked difference between the control and the intervention groups in the prevalence of persons with elevated CRS at the 5-year follow-up. The RR is reduced to about half—at the 5-year follow-up the prevalence of those with elevated CRS in the intervention groups is approximately half that in the control group. The absolute risk reduction is 8.6% (number needed to treat = 11.6). The same pattern is evident among baseline risk groups—the RR of having elevated CRS is reduced to about half, but with larger absolute risk reductions.

According to self-reported data at the 5-year follow-up, the positive impact on cardiovascular risk factors was not a result of medication. In the control group, 6.8% were using blood pressure medicine, compared to 4.8% in the intervention groups; 1.0% of the control group and 0.9% of the intervention groups were on heart medication, and 3.9% of the control group and 3.7% of the intervention groups were on diuretic medication.

TABLE 2
Cardiovascular risk score and other cardiovascular risk factors after 5 years of follow-up

TABLE 3
Prevalence proportion and relative risk of having elevated cardiovascular risk score or other cardiovascular risk factors, after 5 years of follow-up

Discussion

This study is the first to present 5-year follow-up results from a randomized controlled trial showing the impact of general health screenings and discussions with general practitioners on the cardiovascular risk profile of a general population. The intervention had a modest impact on mean CRS in the general population, and a marked impact on the prevalence of those who were at cardiovascular risk. The impact was significantly greater for groups at cardiovascular risk; the relative risk reduction was approximately the same in those at risk as those not at risk, but with larger absolute risk reductions. The study does not indicate whether the reduction in CRS factors will result in reduced morbidity or mortality.

At the 5-year follow-up there was no difference between the CRS in the health screening plus discussion group and the screening only group. The discussion alone had no discernible impact. Several factors, however, may obscure the role of the discussions with general practitioners in this study. For ethical reasons, all persons advised of an elevated cardiovascular risk were offered a consultation with their general practitioner, regardless of their intervention group. Although consultations in such cases were probably not as extensive and detailed as those offered as part of the study, they may confound the difference in the degree of intervention between the 2 groups. The Danish Health System ensures that all participants can see their own general practitioner at no cost whenever they wish. Participants who were not offered a health discussion as part of the study may nevertheless have taken advantage of this free system to consult their general practitioner, especially if they were advised to do so. Moreover, the low rate of participation after the primary health discussion weakens the strength of the intervention in the health screening plus discussion group. Although the study thus does not provide evidence that such discussions played an essential role in the intervention, health screenings alone may not achieve the same impact. The psychological impact of the intervention may also be different for those who had personal consultations with their general practitioner after the health screening. The BMI values included a few individuals with unhealthily low BMI (<19). The distribution was not significantly different between the groups, although a tendency for an unhealthily low BMI of a slightly greater number of patients was seen in the intervention groups, highlighting the fact that weight loss is not always a relevant factor. Focusing on lifestyle changes might trigger some individuals to indulge in anorexic attitudes and behavior.

The results indicate that health screenings should be both population-based and individually oriented, and that general practitioners should be involved. The population screening is necessary to identify those at risk, since almost none of those with elevated cardiovascular risk were aware of their condition prior to screening. The fact that the general practitioner personally contacted the participants may have increased the participation rate, which is high in this study. In the written feedback after the screenings, general practitioners adjusted their advice to individual participants according to test results, and where appropriate advised them to come in for a personal consultation.

For several reasons, the impact of the intervention—both health screenings and discussions—may be greater than our findings suggest. We cannot measure the impact of the questionnaires on the control group—a methodological problem which also affected the OXCHECK study.^7,8 In the British Family Heart Study,^9,10 the control group was apparently unaffected, but the design of that study makes it impossible to assess the impact of subsequent intervention on baseline risk groups. Moreover, the fact that all the participants in the present study live in a small community may reduce the differences in degrees of intervention among the groups, although this is partially addressed by placing cohabiting couples in the same intervention group. Contact among patients within the various clinics involved may also have blurred the differences between the intervention groups.

In the present study, the general practitioners were not trained in any specific psychotherapeutic method for conducting the health discussions. The low rate of participation in follow-up consultations suggests a need to find better methods of motivating participants. Training general practitioners to use motivational discussions to inspire behavioral change, for example, might increase the impact of the intervention.^3,4 Counseling to trigger changes in attitude and behavior, particularly when modified to the individual’s readiness to change, might be more effective than a traditional health discussion focusing mainly on various risk factors.

Important findings from this study are that a major part of the population is interested in having health screenings and discussions with their general practitioner, although interest declines rapidly; that individuals with elevated risk of coronary heart disease set relevant goals for themselves for lifestyle changes; and that cardiovascular risk after 5 years of follow-up is reduced. Planned health discussions about the health screening results do not seem to reduce cardiovascular risk.

Acknowledgments

The following general practitioners participated in the study: A. Bøgedal, P. Grønbæk, L. Jørgensen, P.T. Jørgensen, H. Lundberg, J.M. Nielsen, G.S. Pedersen, J.C. Rahbek, and N. Bie. We thank the staff at the general practitioners clinic in Ebeltoft for their extraordinary efforts, including the extensive and brilliant administrative assistance given by A. Hilligsøe and E. Therkildsen. Thanks also to A. Brock, head of the laboratory at Randers Central Hospital for analysis of blood and urine, and Sally Laird for revision of English texts relating to the study.

As US medical care has evolved, physicians have been expected to recognize and treat mental health problems in primary care,¹ “the hidden mental health network.”^2,3 Primary care clinicians are expected to observe signs of possible mental health problems, incorporate those observations into differential diagnoses, and decide which problems to treat or monitor and which to send for consultation or referral.⁴ These decisions can have important financial and health consequences, especially in dealing with depression.

Depression is common in the community⁵ and among primary care patients, 6% to 9% of whom report symptoms of major depression.^6-8 An additional 10% to 15% of primary care patients show signs of less severe but important depressive problems.^8,9 “Subclinical depression” is marked by symptoms that might indicate physical disease, signs of depression, or both; recognition may affect costs of care.^10-13

Research has begun to define the impact of depression on processes¹⁴ and costs of care.^15-17 For example, elderly patients reporting symptoms of depression have more laboratory tests performed at higher cost.¹⁵ Primary care patients diagnosed with depression had total yearly health care costs almost double those of patients without depression, with increased costs secondary to higher medical utilization and not mental health specialty treatment.¹⁶ There is evidence that depressive symptoms and the diagnosis of depression may predict increases in costs of care.¹⁷

Costs of care might be influenced by the model used by primary care physicians to identify depression.¹⁸ For example, a biomedical model might use more laboratory testing to reach a diagnosis of depression by exclusion, whereas a psychosocial model would use fewer laboratory tests while the physician pursues psychosocial issues. To identify optimal strategies for practice, it is important to determine how symptoms of depression and physician diagnosis of depression might interrelate and affect medical care costs.

We explored the following hypotheses: (1) that there are significant differences in each type of charge determined by the presence or absence of symptoms and diagnosis of depression; (2) that depressive symptoms and physician diagnosis of depression predict the occurrence of charges for specialty care, emergency services, laboratory services, and hospitalization; and (3) that depressive symptoms and physician diagnosis of depression predict the magnitude of medical charges for primary care, specialty care, emergency services, laboratory services, hospitalization, and total charges.

Methods

Study design

Five hundred eight adult nonpregnant new patients were assigned randomly to primary care providers in either family practice or general internal medicine clinics in a teaching hospital. Children younger than 18 years and pregnant women were excluded because they are not followed in general internal medicine. At enrollment and follow-up, self-reported depression was determined with the abbreviated Beck Depression Inventory (BDI)¹⁹ and health status was measured with the Medical Outcomes Studies Short Form-36 (MOS SF-36).²⁰ To avoid altering clinician practice, physicians were not provided with either score. Physicians included 105 senior residents (second and third year) in family practice and general internal medicine.

Measures

Beck Depression Inventory. The BDI is a reliable and valid instrument used to measure depressive symptoms.^19,21 The abbreviated version includes 13 items weighted and summed to produce a total score.¹⁹ A score between 9 and 15 indicates moderate depression, and a score of at least 16 indicates severe depression. The BDI is used widely for screening and to assess treatment efficacy.²² In this study, a BDI score between 0 and 8 was considered “low” or normal, and a score of at least 9 was considered “high” or indicative of symptoms of depression.

At study entrance or exit, 130 patients were identified with significant symptoms of depression (BDI > 8) by meeting criteria for moderate or severe depression¹⁹ and thus identifying roughly the top quartile of BDI scores among participants. This proportion approximates that of primary care patients estimated to experience significant depression.^6,7

Medical Outcomes Studies Short Form-36. Health status was measured with the MOS SF-36,²⁰ a 36-item self-report questionnaire. Reliability has been verified for difficult populations.²³ Summary measures can describe a physical component score and a mental component score.^24,25 The physical component score was used in this study to measure physical health status.

Medical chart review. Two physicians (K.D.B. and J.A.R.) reviewed the charts to identify notations of depression on problem lists and in visit notes to signify physician diagnosis of depression.

Charges. Charges were used as a proxy for costs. Electronic data for all health system charges were monitored from the initial visit through 1 full year of care. Six categories were monitored: primary care, specialty care, laboratory testing, emergency department, hospitalization, and total charges. Pharmacy charges were excluded because some patients purchased prescriptions outside the hospital system.

Statistical procedures

Mean log values for each area of medical charges were determined and contrasted with the Duncan multiple range test²⁶ to explore the first hypothesis that charges are associated with symptoms and diagnoses of depression. Next, a double hurdle model was used to test the hypotheses that depressive symptoms and physician diagnosis of depression predict the occurrence and magnitude of charges for a variety of services.^27,28 In a double hurdle model, the first “hurdle,” or step, involves exploring whether there are variables that can significantly predict the occurrence of an event (such as a medical charge). The second step involves exploring whether there are variables that can predict the magnitude of the event (eg, a medical charge).

Log-transformation of charges was performed to eliminate undue influence from outliers. No logistic regression models were developed for the occurrence of primary care charges or total charges (the first hurdle) because all study patients had charges in both categories. Results are presented by hypothesis.

Results

Seventy-seven of 508 study patients (15.1%) were identified as depressed by their primary care providers in chart notes. BDI scores showed considerable spread (range, 0–31) and were significantly associated with the diagnosis of depression (P < .001). Whereas 140 patients reported BDI scores of at least 9, only 36 of these patients were diagnosed as depressed by their physicians. Similarly, 41 patients were diagnosed as depressed despite reporting low (normal) BDI scores. Patients were assigned to 1 of 4 groups: those diagnosed as depressed and having high (abnormal) BDI scores (n = 36); those diagnosed as depressed despite low BDI scores (n = 41); those not diagnosed as depressed despite high BDI scores (n = 94); and those not diagnosed as depressed and not having high BDI scores (n = 337).

Hypothesis 1: overall impact of symptoms and diagnosis on charges

Groups diagnosed with depression had significantly higher log primary care charges than did those not diagnosed (Table 1). Both groups diagnosed with depression showed the highest primary care and total medical charges. Patients diagnosed with depression and reporting high BDI scores had higher specialty charges than those not depressed. Highest laboratory costs were found for those diagnosed as depressed despite low BDI scores and those with elevated BDI scores who were not diagnosed as depressed. There were no significant differences among groups for log charges for emergency care and hospital charges.

TABLE 1
Log charges of care by diagnosis and symptoms of depression

Hypotheses 2 and 3: factors predicting occurrence and magnitude of charges

Cost models are presented as regressions in Table 2. The left side of the table presents logistic regressions exploring which variables predict whether or not a patient accrues charges in all areas except primary care and total charges. Because all patients had at least 1 primary care visit charge and, hence, a total charge, it was not possible to develop a model to predict the occurrence of those charges.

Physical health status (measured by the physical component score of the MOS SF-36) predicted the occurrence of all charges measured with the exception of laboratory tests. Advanced patient age predicted increased likelihood of charges in each area; female sex showed a trend toward predicting occurrence of emergency care charges; and education showed a trend toward predicting occurrence of laboratory charges. BDI scores (measure of symptoms of depression) and physician diagnosis of depression failed to contribute significantly to the prediction of specialty care, emergency care, laboratory testing, or hospital charges. However, there was a trend for depressive symptoms to predict the occurrence of laboratory charges.

The right side of Table 2 presents regression models that predicted the magnitude of the different categories of charges. Physical health status was a significant predictor of the magnitude of all types of charges except emergency care. Patient age contributed to prediction of size of all types of charges except emergency visits and laboratory tests. Female sex was a significant predictor of magnitude of charges in primary care, laboratory tests, and total medical charges. The diagnosis of depression was a significant predictor of magnitude of primary care (P = .0029) and total medical (P = .0158) charges. Neither depressive symptoms nor the diagnosis of depression contributed significantly to the prediction of magnitude of charges for specialty care, emergency care, laboratory testing, or hospital use, although there was a trend for depressive symptoms to predict the magnitude of laboratory costs. Although an interaction term was entered into both kinds of regression equations, there was no evidence of a significant contribution from the interaction of symptoms of depression and diagnosis of depression in any of the predictor models developed.

TABLE 2
Regression analyses predicting charges

Discussion

Medical charges were related to symptoms of depression and physician diagnosis of depression in this study. Although the patient sample was small, it was representative of the primary care population in displaying a wide range of depressive symptoms as measured by the BDI.^6,7 In this study, physician diagnosis of depression was related to self-reported depression ratings: those diagnosed as depressed had significantly higher BDI scores than did those not diagnosed as depressed. However, the relationship between self-reported symptoms and diagnosis was not perfect: 72% of patients with high BDI scores were not recognized as depressed, as often occurs in primary care.^6,7 In fact, more patients diagnosed with depression had low BDI scores (< 9, n = 41) than high BDI scores (> 8, n = 36). Clearly, other factors enter the process by which primary care physicians reach the diagnosis of depression.

Symptoms of depression and the diagnosis of depression probably influence the process of care in different ways. Differences in process of care likely would be reflected in different relationships to medical charges. Physician diagnosis of depression was associated with higher primary care and total costs and contributed to models predicting magnitude of primary care and total charges. However, neither symptoms of depression nor diagnosis of depression predicted which patients were more likely to incur charges for specialty care, emergency care, laboratory tests, or hospitalization. There was a trend only for the symptoms of depression to predict who would incur laboratory charges. These findings suggest that the relationship between depression and primary care charges and total charges is clear but less apparent when looking at less frequently occurring charges.

Other demographic factors showed fairly robust associations with the occurrence of charges. Patient age predicted who would get specialty care, emergency care, laboratory costs, and hospitalization, and there was a trend for female sex to predict occurrence of emergency department charges. Health status proved to be a significant predictor of the magnitude of all charges except those for emergency care. These powerful influences must be considered to accurately assess the impact of depression on charges.

Age also predicted the total amount of charges for primary and all medical care for the year and showed a trend toward prediction of magnitude of specialty charges. Female sex was a significant predictor of magnitude of primary care charges, laboratory charges, and total charges, and less education was a significant predictor of magnitude of emergency department and hospital charges. Some of these demographic predictors are readily explained. For example, as patients age, the number and costs of medical problems often increase. More education may enhance socioeconomic status and self-care, each of which may buffer against the need for emergency care and hospitalization. The reasons that charges are often higher for women are probably more complex. Higher utilization of primary and specialty care for women was associated with lower self-report-ed health status, less education, and lower socioeconomic status in our previous study.²⁹

These results also suggest that physician diagnosis of depression in the absence of elevated BDI scores may flag a different kind of patient presentation. Diagnosis of depression without elevated BDI scores could result from effective treatment controlling the symptoms of previously diagnosed depression, but this does not adequately explain the occurrence. Perhaps other aspects of physician–patient interaction trigger a depression diagnosis without symptoms. This group ranked highest for log-transformed charges for 5 of the 6 areas explored: only for specialty care did those with high BDI scores and diagnosis of depression rank higher in total cost. This strong association with charges implies that these patients represent diagnostic dilemmas, thereby generating more primary care visits and laboratory tests. They may be diagnosed as depressed despite their low BDI scores simply because no organic explanation can be readily identified.

BDI scores showed a trend toward predicting higher laboratory charges in our models. This finding supports the importance of depressive symptoms in influencing the process of primary care, especially laboratory testing.^15,30 Perhaps the diagnosis of depression actually slowed the ordering of laboratory tests.¹⁸ Because our data did not allow a separation of charges for laboratory tests before and after the diagnosis of depression, we did not test this possibility.

The size of this sample (N = 508) and the length of time patients were followed (1 year) might not have provided adequate power to fully test the contributions of symptoms and diagnosis of depression to the 6 sets of charges. This was likely true for hospitalization charges because hospitalization was an infrequent event in this study. Previous, larger studies found indications of increased hospitalization charges for those diagnosed as depressed¹⁷ and those with symptoms of depression.^15,30 Alternatively, the recent emphasis on decreasing hospitalizations to reduce medical costs may mean that hospitalization for depressive symptoms rather than for physical illness is less likely to occur.³¹ In addition, these observations were made by resident physicians and not by community clinicians. It is not clear whether these results would generalize to another setting, although they are consistent with community observations in previous research.

These data do suggest an intriguing interplay of the impact of physician diagnosis of depression and presence of symptoms of depression in a number of indicators of charges and utilization in primary care. Even though each element was associated with increased utilization and charges, their differential impact is unclear. Both may prove important for efforts to enhance recognition of depression; recognition of a mental health problem appeared to shift the process of care in this and previous studies.^14,32 To date, there are no data indicating that the diagnosis of depression reduces utilization or costs of primary care delivery. What is known is that physicians working in primary care are more apt to accurately diagnose those with more severe symptoms of depression than those with more transient or less severe symptoms.^16,33 Although introducing a screening device such as the BDI or the PRIME-MD⁹ likely would increase the number of patients diagnosed with depression, it is unclear what impact that would have on the process, costs, and outcomes of care. Simpler interventions such as training in communication skills such as empathy³⁴ might provide the primary care physician with all the tools needed for identification of emotional distress and mental health problems^14,30 and appropriate treatment or referral.

As US medical care has evolved, physicians have been expected to recognize and treat mental health problems in primary care,¹ “the hidden mental health network.”^2,3 Primary care clinicians are expected to observe signs of possible mental health problems, incorporate those observations into differential diagnoses, and decide which problems to treat or monitor and which to send for consultation or referral.⁴ These decisions can have important financial and health consequences, especially in dealing with depression.

Depression is common in the community⁵ and among primary care patients, 6% to 9% of whom report symptoms of major depression.^6-8 An additional 10% to 15% of primary care patients show signs of less severe but important depressive problems.^8,9 “Subclinical depression” is marked by symptoms that might indicate physical disease, signs of depression, or both; recognition may affect costs of care.^10-13

Research has begun to define the impact of depression on processes¹⁴ and costs of care.^15-17 For example, elderly patients reporting symptoms of depression have more laboratory tests performed at higher cost.¹⁵ Primary care patients diagnosed with depression had total yearly health care costs almost double those of patients without depression, with increased costs secondary to higher medical utilization and not mental health specialty treatment.¹⁶ There is evidence that depressive symptoms and the diagnosis of depression may predict increases in costs of care.¹⁷

Costs of care might be influenced by the model used by primary care physicians to identify depression.¹⁸ For example, a biomedical model might use more laboratory testing to reach a diagnosis of depression by exclusion, whereas a psychosocial model would use fewer laboratory tests while the physician pursues psychosocial issues. To identify optimal strategies for practice, it is important to determine how symptoms of depression and physician diagnosis of depression might interrelate and affect medical care costs.

We explored the following hypotheses: (1) that there are significant differences in each type of charge determined by the presence or absence of symptoms and diagnosis of depression; (2) that depressive symptoms and physician diagnosis of depression predict the occurrence of charges for specialty care, emergency services, laboratory services, and hospitalization; and (3) that depressive symptoms and physician diagnosis of depression predict the magnitude of medical charges for primary care, specialty care, emergency services, laboratory services, hospitalization, and total charges.

Methods

Study design

Five hundred eight adult nonpregnant new patients were assigned randomly to primary care providers in either family practice or general internal medicine clinics in a teaching hospital. Children younger than 18 years and pregnant women were excluded because they are not followed in general internal medicine. At enrollment and follow-up, self-reported depression was determined with the abbreviated Beck Depression Inventory (BDI)¹⁹ and health status was measured with the Medical Outcomes Studies Short Form-36 (MOS SF-36).²⁰ To avoid altering clinician practice, physicians were not provided with either score. Physicians included 105 senior residents (second and third year) in family practice and general internal medicine.

Measures

Beck Depression Inventory. The BDI is a reliable and valid instrument used to measure depressive symptoms.^19,21 The abbreviated version includes 13 items weighted and summed to produce a total score.¹⁹ A score between 9 and 15 indicates moderate depression, and a score of at least 16 indicates severe depression. The BDI is used widely for screening and to assess treatment efficacy.²² In this study, a BDI score between 0 and 8 was considered “low” or normal, and a score of at least 9 was considered “high” or indicative of symptoms of depression.

At study entrance or exit, 130 patients were identified with significant symptoms of depression (BDI > 8) by meeting criteria for moderate or severe depression¹⁹ and thus identifying roughly the top quartile of BDI scores among participants. This proportion approximates that of primary care patients estimated to experience significant depression.^6,7

Medical Outcomes Studies Short Form-36. Health status was measured with the MOS SF-36,²⁰ a 36-item self-report questionnaire. Reliability has been verified for difficult populations.²³ Summary measures can describe a physical component score and a mental component score.^24,25 The physical component score was used in this study to measure physical health status.

Medical chart review. Two physicians (K.D.B. and J.A.R.) reviewed the charts to identify notations of depression on problem lists and in visit notes to signify physician diagnosis of depression.

Charges. Charges were used as a proxy for costs. Electronic data for all health system charges were monitored from the initial visit through 1 full year of care. Six categories were monitored: primary care, specialty care, laboratory testing, emergency department, hospitalization, and total charges. Pharmacy charges were excluded because some patients purchased prescriptions outside the hospital system.

Statistical procedures

Mean log values for each area of medical charges were determined and contrasted with the Duncan multiple range test²⁶ to explore the first hypothesis that charges are associated with symptoms and diagnoses of depression. Next, a double hurdle model was used to test the hypotheses that depressive symptoms and physician diagnosis of depression predict the occurrence and magnitude of charges for a variety of services.^27,28 In a double hurdle model, the first “hurdle,” or step, involves exploring whether there are variables that can significantly predict the occurrence of an event (such as a medical charge). The second step involves exploring whether there are variables that can predict the magnitude of the event (eg, a medical charge).

Log-transformation of charges was performed to eliminate undue influence from outliers. No logistic regression models were developed for the occurrence of primary care charges or total charges (the first hurdle) because all study patients had charges in both categories. Results are presented by hypothesis.

Results

Seventy-seven of 508 study patients (15.1%) were identified as depressed by their primary care providers in chart notes. BDI scores showed considerable spread (range, 0–31) and were significantly associated with the diagnosis of depression (P < .001). Whereas 140 patients reported BDI scores of at least 9, only 36 of these patients were diagnosed as depressed by their physicians. Similarly, 41 patients were diagnosed as depressed despite reporting low (normal) BDI scores. Patients were assigned to 1 of 4 groups: those diagnosed as depressed and having high (abnormal) BDI scores (n = 36); those diagnosed as depressed despite low BDI scores (n = 41); those not diagnosed as depressed despite high BDI scores (n = 94); and those not diagnosed as depressed and not having high BDI scores (n = 337).

Hypothesis 1: overall impact of symptoms and diagnosis on charges

Groups diagnosed with depression had significantly higher log primary care charges than did those not diagnosed (Table 1). Both groups diagnosed with depression showed the highest primary care and total medical charges. Patients diagnosed with depression and reporting high BDI scores had higher specialty charges than those not depressed. Highest laboratory costs were found for those diagnosed as depressed despite low BDI scores and those with elevated BDI scores who were not diagnosed as depressed. There were no significant differences among groups for log charges for emergency care and hospital charges.

TABLE 1
Log charges of care by diagnosis and symptoms of depression

Hypotheses 2 and 3: factors predicting occurrence and magnitude of charges

Cost models are presented as regressions in Table 2. The left side of the table presents logistic regressions exploring which variables predict whether or not a patient accrues charges in all areas except primary care and total charges. Because all patients had at least 1 primary care visit charge and, hence, a total charge, it was not possible to develop a model to predict the occurrence of those charges.

Physical health status (measured by the physical component score of the MOS SF-36) predicted the occurrence of all charges measured with the exception of laboratory tests. Advanced patient age predicted increased likelihood of charges in each area; female sex showed a trend toward predicting occurrence of emergency care charges; and education showed a trend toward predicting occurrence of laboratory charges. BDI scores (measure of symptoms of depression) and physician diagnosis of depression failed to contribute significantly to the prediction of specialty care, emergency care, laboratory testing, or hospital charges. However, there was a trend for depressive symptoms to predict the occurrence of laboratory charges.

The right side of Table 2 presents regression models that predicted the magnitude of the different categories of charges. Physical health status was a significant predictor of the magnitude of all types of charges except emergency care. Patient age contributed to prediction of size of all types of charges except emergency visits and laboratory tests. Female sex was a significant predictor of magnitude of charges in primary care, laboratory tests, and total medical charges. The diagnosis of depression was a significant predictor of magnitude of primary care (P = .0029) and total medical (P = .0158) charges. Neither depressive symptoms nor the diagnosis of depression contributed significantly to the prediction of magnitude of charges for specialty care, emergency care, laboratory testing, or hospital use, although there was a trend for depressive symptoms to predict the magnitude of laboratory costs. Although an interaction term was entered into both kinds of regression equations, there was no evidence of a significant contribution from the interaction of symptoms of depression and diagnosis of depression in any of the predictor models developed.

TABLE 2
Regression analyses predicting charges

Discussion

Medical charges were related to symptoms of depression and physician diagnosis of depression in this study. Although the patient sample was small, it was representative of the primary care population in displaying a wide range of depressive symptoms as measured by the BDI.^6,7 In this study, physician diagnosis of depression was related to self-reported depression ratings: those diagnosed as depressed had significantly higher BDI scores than did those not diagnosed as depressed. However, the relationship between self-reported symptoms and diagnosis was not perfect: 72% of patients with high BDI scores were not recognized as depressed, as often occurs in primary care.^6,7 In fact, more patients diagnosed with depression had low BDI scores (< 9, n = 41) than high BDI scores (> 8, n = 36). Clearly, other factors enter the process by which primary care physicians reach the diagnosis of depression.

Symptoms of depression and the diagnosis of depression probably influence the process of care in different ways. Differences in process of care likely would be reflected in different relationships to medical charges. Physician diagnosis of depression was associated with higher primary care and total costs and contributed to models predicting magnitude of primary care and total charges. However, neither symptoms of depression nor diagnosis of depression predicted which patients were more likely to incur charges for specialty care, emergency care, laboratory tests, or hospitalization. There was a trend only for the symptoms of depression to predict who would incur laboratory charges. These findings suggest that the relationship between depression and primary care charges and total charges is clear but less apparent when looking at less frequently occurring charges.

Other demographic factors showed fairly robust associations with the occurrence of charges. Patient age predicted who would get specialty care, emergency care, laboratory costs, and hospitalization, and there was a trend for female sex to predict occurrence of emergency department charges. Health status proved to be a significant predictor of the magnitude of all charges except those for emergency care. These powerful influences must be considered to accurately assess the impact of depression on charges.

Age also predicted the total amount of charges for primary and all medical care for the year and showed a trend toward prediction of magnitude of specialty charges. Female sex was a significant predictor of magnitude of primary care charges, laboratory charges, and total charges, and less education was a significant predictor of magnitude of emergency department and hospital charges. Some of these demographic predictors are readily explained. For example, as patients age, the number and costs of medical problems often increase. More education may enhance socioeconomic status and self-care, each of which may buffer against the need for emergency care and hospitalization. The reasons that charges are often higher for women are probably more complex. Higher utilization of primary and specialty care for women was associated with lower self-report-ed health status, less education, and lower socioeconomic status in our previous study.²⁹

These results also suggest that physician diagnosis of depression in the absence of elevated BDI scores may flag a different kind of patient presentation. Diagnosis of depression without elevated BDI scores could result from effective treatment controlling the symptoms of previously diagnosed depression, but this does not adequately explain the occurrence. Perhaps other aspects of physician–patient interaction trigger a depression diagnosis without symptoms. This group ranked highest for log-transformed charges for 5 of the 6 areas explored: only for specialty care did those with high BDI scores and diagnosis of depression rank higher in total cost. This strong association with charges implies that these patients represent diagnostic dilemmas, thereby generating more primary care visits and laboratory tests. They may be diagnosed as depressed despite their low BDI scores simply because no organic explanation can be readily identified.

BDI scores showed a trend toward predicting higher laboratory charges in our models. This finding supports the importance of depressive symptoms in influencing the process of primary care, especially laboratory testing.^15,30 Perhaps the diagnosis of depression actually slowed the ordering of laboratory tests.¹⁸ Because our data did not allow a separation of charges for laboratory tests before and after the diagnosis of depression, we did not test this possibility.

The size of this sample (N = 508) and the length of time patients were followed (1 year) might not have provided adequate power to fully test the contributions of symptoms and diagnosis of depression to the 6 sets of charges. This was likely true for hospitalization charges because hospitalization was an infrequent event in this study. Previous, larger studies found indications of increased hospitalization charges for those diagnosed as depressed¹⁷ and those with symptoms of depression.^15,30 Alternatively, the recent emphasis on decreasing hospitalizations to reduce medical costs may mean that hospitalization for depressive symptoms rather than for physical illness is less likely to occur.³¹ In addition, these observations were made by resident physicians and not by community clinicians. It is not clear whether these results would generalize to another setting, although they are consistent with community observations in previous research.

These data do suggest an intriguing interplay of the impact of physician diagnosis of depression and presence of symptoms of depression in a number of indicators of charges and utilization in primary care. Even though each element was associated with increased utilization and charges, their differential impact is unclear. Both may prove important for efforts to enhance recognition of depression; recognition of a mental health problem appeared to shift the process of care in this and previous studies.^14,32 To date, there are no data indicating that the diagnosis of depression reduces utilization or costs of primary care delivery. What is known is that physicians working in primary care are more apt to accurately diagnose those with more severe symptoms of depression than those with more transient or less severe symptoms.^16,33 Although introducing a screening device such as the BDI or the PRIME-MD⁹ likely would increase the number of patients diagnosed with depression, it is unclear what impact that would have on the process, costs, and outcomes of care. Simpler interventions such as training in communication skills such as empathy³⁴ might provide the primary care physician with all the tools needed for identification of emotional distress and mental health problems^14,30 and appropriate treatment or referral.

As US medical care has evolved, physicians have been expected to recognize and treat mental health problems in primary care,¹ “the hidden mental health network.”^2,3 Primary care clinicians are expected to observe signs of possible mental health problems, incorporate those observations into differential diagnoses, and decide which problems to treat or monitor and which to send for consultation or referral.⁴ These decisions can have important financial and health consequences, especially in dealing with depression.

Depression is common in the community⁵ and among primary care patients, 6% to 9% of whom report symptoms of major depression.^6-8 An additional 10% to 15% of primary care patients show signs of less severe but important depressive problems.^8,9 “Subclinical depression” is marked by symptoms that might indicate physical disease, signs of depression, or both; recognition may affect costs of care.^10-13

Research has begun to define the impact of depression on processes¹⁴ and costs of care.^15-17 For example, elderly patients reporting symptoms of depression have more laboratory tests performed at higher cost.¹⁵ Primary care patients diagnosed with depression had total yearly health care costs almost double those of patients without depression, with increased costs secondary to higher medical utilization and not mental health specialty treatment.¹⁶ There is evidence that depressive symptoms and the diagnosis of depression may predict increases in costs of care.¹⁷

Costs of care might be influenced by the model used by primary care physicians to identify depression.¹⁸ For example, a biomedical model might use more laboratory testing to reach a diagnosis of depression by exclusion, whereas a psychosocial model would use fewer laboratory tests while the physician pursues psychosocial issues. To identify optimal strategies for practice, it is important to determine how symptoms of depression and physician diagnosis of depression might interrelate and affect medical care costs.

We explored the following hypotheses: (1) that there are significant differences in each type of charge determined by the presence or absence of symptoms and diagnosis of depression; (2) that depressive symptoms and physician diagnosis of depression predict the occurrence of charges for specialty care, emergency services, laboratory services, and hospitalization; and (3) that depressive symptoms and physician diagnosis of depression predict the magnitude of medical charges for primary care, specialty care, emergency services, laboratory services, hospitalization, and total charges.

Methods

Study design

Five hundred eight adult nonpregnant new patients were assigned randomly to primary care providers in either family practice or general internal medicine clinics in a teaching hospital. Children younger than 18 years and pregnant women were excluded because they are not followed in general internal medicine. At enrollment and follow-up, self-reported depression was determined with the abbreviated Beck Depression Inventory (BDI)¹⁹ and health status was measured with the Medical Outcomes Studies Short Form-36 (MOS SF-36).²⁰ To avoid altering clinician practice, physicians were not provided with either score. Physicians included 105 senior residents (second and third year) in family practice and general internal medicine.

Measures

Beck Depression Inventory. The BDI is a reliable and valid instrument used to measure depressive symptoms.^19,21 The abbreviated version includes 13 items weighted and summed to produce a total score.¹⁹ A score between 9 and 15 indicates moderate depression, and a score of at least 16 indicates severe depression. The BDI is used widely for screening and to assess treatment efficacy.²² In this study, a BDI score between 0 and 8 was considered “low” or normal, and a score of at least 9 was considered “high” or indicative of symptoms of depression.

At study entrance or exit, 130 patients were identified with significant symptoms of depression (BDI > 8) by meeting criteria for moderate or severe depression¹⁹ and thus identifying roughly the top quartile of BDI scores among participants. This proportion approximates that of primary care patients estimated to experience significant depression.^6,7

Medical Outcomes Studies Short Form-36. Health status was measured with the MOS SF-36,²⁰ a 36-item self-report questionnaire. Reliability has been verified for difficult populations.²³ Summary measures can describe a physical component score and a mental component score.^24,25 The physical component score was used in this study to measure physical health status.

Medical chart review. Two physicians (K.D.B. and J.A.R.) reviewed the charts to identify notations of depression on problem lists and in visit notes to signify physician diagnosis of depression.

Charges. Charges were used as a proxy for costs. Electronic data for all health system charges were monitored from the initial visit through 1 full year of care. Six categories were monitored: primary care, specialty care, laboratory testing, emergency department, hospitalization, and total charges. Pharmacy charges were excluded because some patients purchased prescriptions outside the hospital system.

Statistical procedures

Mean log values for each area of medical charges were determined and contrasted with the Duncan multiple range test²⁶ to explore the first hypothesis that charges are associated with symptoms and diagnoses of depression. Next, a double hurdle model was used to test the hypotheses that depressive symptoms and physician diagnosis of depression predict the occurrence and magnitude of charges for a variety of services.^27,28 In a double hurdle model, the first “hurdle,” or step, involves exploring whether there are variables that can significantly predict the occurrence of an event (such as a medical charge). The second step involves exploring whether there are variables that can predict the magnitude of the event (eg, a medical charge).

Log-transformation of charges was performed to eliminate undue influence from outliers. No logistic regression models were developed for the occurrence of primary care charges or total charges (the first hurdle) because all study patients had charges in both categories. Results are presented by hypothesis.

Results

Seventy-seven of 508 study patients (15.1%) were identified as depressed by their primary care providers in chart notes. BDI scores showed considerable spread (range, 0–31) and were significantly associated with the diagnosis of depression (P < .001). Whereas 140 patients reported BDI scores of at least 9, only 36 of these patients were diagnosed as depressed by their physicians. Similarly, 41 patients were diagnosed as depressed despite reporting low (normal) BDI scores. Patients were assigned to 1 of 4 groups: those diagnosed as depressed and having high (abnormal) BDI scores (n = 36); those diagnosed as depressed despite low BDI scores (n = 41); those not diagnosed as depressed despite high BDI scores (n = 94); and those not diagnosed as depressed and not having high BDI scores (n = 337).

Hypothesis 1: overall impact of symptoms and diagnosis on charges

Groups diagnosed with depression had significantly higher log primary care charges than did those not diagnosed (Table 1). Both groups diagnosed with depression showed the highest primary care and total medical charges. Patients diagnosed with depression and reporting high BDI scores had higher specialty charges than those not depressed. Highest laboratory costs were found for those diagnosed as depressed despite low BDI scores and those with elevated BDI scores who were not diagnosed as depressed. There were no significant differences among groups for log charges for emergency care and hospital charges.

TABLE 1
Log charges of care by diagnosis and symptoms of depression

Hypotheses 2 and 3: factors predicting occurrence and magnitude of charges

Cost models are presented as regressions in Table 2. The left side of the table presents logistic regressions exploring which variables predict whether or not a patient accrues charges in all areas except primary care and total charges. Because all patients had at least 1 primary care visit charge and, hence, a total charge, it was not possible to develop a model to predict the occurrence of those charges.

Physical health status (measured by the physical component score of the MOS SF-36) predicted the occurrence of all charges measured with the exception of laboratory tests. Advanced patient age predicted increased likelihood of charges in each area; female sex showed a trend toward predicting occurrence of emergency care charges; and education showed a trend toward predicting occurrence of laboratory charges. BDI scores (measure of symptoms of depression) and physician diagnosis of depression failed to contribute significantly to the prediction of specialty care, emergency care, laboratory testing, or hospital charges. However, there was a trend for depressive symptoms to predict the occurrence of laboratory charges.

The right side of Table 2 presents regression models that predicted the magnitude of the different categories of charges. Physical health status was a significant predictor of the magnitude of all types of charges except emergency care. Patient age contributed to prediction of size of all types of charges except emergency visits and laboratory tests. Female sex was a significant predictor of magnitude of charges in primary care, laboratory tests, and total medical charges. The diagnosis of depression was a significant predictor of magnitude of primary care (P = .0029) and total medical (P = .0158) charges. Neither depressive symptoms nor the diagnosis of depression contributed significantly to the prediction of magnitude of charges for specialty care, emergency care, laboratory testing, or hospital use, although there was a trend for depressive symptoms to predict the magnitude of laboratory costs. Although an interaction term was entered into both kinds of regression equations, there was no evidence of a significant contribution from the interaction of symptoms of depression and diagnosis of depression in any of the predictor models developed.

TABLE 2
Regression analyses predicting charges

Discussion

Medical charges were related to symptoms of depression and physician diagnosis of depression in this study. Although the patient sample was small, it was representative of the primary care population in displaying a wide range of depressive symptoms as measured by the BDI.^6,7 In this study, physician diagnosis of depression was related to self-reported depression ratings: those diagnosed as depressed had significantly higher BDI scores than did those not diagnosed as depressed. However, the relationship between self-reported symptoms and diagnosis was not perfect: 72% of patients with high BDI scores were not recognized as depressed, as often occurs in primary care.^6,7 In fact, more patients diagnosed with depression had low BDI scores (< 9, n = 41) than high BDI scores (> 8, n = 36). Clearly, other factors enter the process by which primary care physicians reach the diagnosis of depression.

Symptoms of depression and the diagnosis of depression probably influence the process of care in different ways. Differences in process of care likely would be reflected in different relationships to medical charges. Physician diagnosis of depression was associated with higher primary care and total costs and contributed to models predicting magnitude of primary care and total charges. However, neither symptoms of depression nor diagnosis of depression predicted which patients were more likely to incur charges for specialty care, emergency care, laboratory tests, or hospitalization. There was a trend only for the symptoms of depression to predict who would incur laboratory charges. These findings suggest that the relationship between depression and primary care charges and total charges is clear but less apparent when looking at less frequently occurring charges.

Other demographic factors showed fairly robust associations with the occurrence of charges. Patient age predicted who would get specialty care, emergency care, laboratory costs, and hospitalization, and there was a trend for female sex to predict occurrence of emergency department charges. Health status proved to be a significant predictor of the magnitude of all charges except those for emergency care. These powerful influences must be considered to accurately assess the impact of depression on charges.

Age also predicted the total amount of charges for primary and all medical care for the year and showed a trend toward prediction of magnitude of specialty charges. Female sex was a significant predictor of magnitude of primary care charges, laboratory charges, and total charges, and less education was a significant predictor of magnitude of emergency department and hospital charges. Some of these demographic predictors are readily explained. For example, as patients age, the number and costs of medical problems often increase. More education may enhance socioeconomic status and self-care, each of which may buffer against the need for emergency care and hospitalization. The reasons that charges are often higher for women are probably more complex. Higher utilization of primary and specialty care for women was associated with lower self-report-ed health status, less education, and lower socioeconomic status in our previous study.²⁹

These results also suggest that physician diagnosis of depression in the absence of elevated BDI scores may flag a different kind of patient presentation. Diagnosis of depression without elevated BDI scores could result from effective treatment controlling the symptoms of previously diagnosed depression, but this does not adequately explain the occurrence. Perhaps other aspects of physician–patient interaction trigger a depression diagnosis without symptoms. This group ranked highest for log-transformed charges for 5 of the 6 areas explored: only for specialty care did those with high BDI scores and diagnosis of depression rank higher in total cost. This strong association with charges implies that these patients represent diagnostic dilemmas, thereby generating more primary care visits and laboratory tests. They may be diagnosed as depressed despite their low BDI scores simply because no organic explanation can be readily identified.

BDI scores showed a trend toward predicting higher laboratory charges in our models. This finding supports the importance of depressive symptoms in influencing the process of primary care, especially laboratory testing.^15,30 Perhaps the diagnosis of depression actually slowed the ordering of laboratory tests.¹⁸ Because our data did not allow a separation of charges for laboratory tests before and after the diagnosis of depression, we did not test this possibility.

The size of this sample (N = 508) and the length of time patients were followed (1 year) might not have provided adequate power to fully test the contributions of symptoms and diagnosis of depression to the 6 sets of charges. This was likely true for hospitalization charges because hospitalization was an infrequent event in this study. Previous, larger studies found indications of increased hospitalization charges for those diagnosed as depressed¹⁷ and those with symptoms of depression.^15,30 Alternatively, the recent emphasis on decreasing hospitalizations to reduce medical costs may mean that hospitalization for depressive symptoms rather than for physical illness is less likely to occur.³¹ In addition, these observations were made by resident physicians and not by community clinicians. It is not clear whether these results would generalize to another setting, although they are consistent with community observations in previous research.

These data do suggest an intriguing interplay of the impact of physician diagnosis of depression and presence of symptoms of depression in a number of indicators of charges and utilization in primary care. Even though each element was associated with increased utilization and charges, their differential impact is unclear. Both may prove important for efforts to enhance recognition of depression; recognition of a mental health problem appeared to shift the process of care in this and previous studies.^14,32 To date, there are no data indicating that the diagnosis of depression reduces utilization or costs of primary care delivery. What is known is that physicians working in primary care are more apt to accurately diagnose those with more severe symptoms of depression than those with more transient or less severe symptoms.^16,33 Although introducing a screening device such as the BDI or the PRIME-MD⁹ likely would increase the number of patients diagnosed with depression, it is unclear what impact that would have on the process, costs, and outcomes of care. Simpler interventions such as training in communication skills such as empathy³⁴ might provide the primary care physician with all the tools needed for identification of emotional distress and mental health problems^14,30 and appropriate treatment or referral.

Current medical therapies for osteoarthritis include conventional nonsteroidal anti-inflammatory drugs (NSAIDs), acetaminophen, glucosamine sulfate, and intra-articular injections of corticosteroids and hyaluronic acid. However, long-term use of corticosteroid injections can exacerbate damage to the affected joints.1,2 Conventional NSAIDs are associated with upper gastrointestinal tract ulceration and inhibition of platelet function.3

Cyclooxygenase-2 (COX-2)–specific inhibitors have demonstrated equivalent efficacy to conventional NSAIDs in treating pain and inflammation associated with osteoarthritis and rheumatoid arthritis. Further, COX-2–specific inhibitors significantly reduce the incidence of gastrointestinal ulceration and bleeding side effects caused by conventional NSAIDs.^4,5 Valdecoxib (Bextra; Pharmacia Corporation and Pfizer Corporation) is a novel COX-2–specific inhibitor that is approximately 28,000-fold more selective against COX-2 than against COX-1. As a potent COX-2–specific inhibitor, valdecoxib is expected to provide efficacy equivalent to conventional NSAIDs for treatment of arthritis and spare the COX-1–related side effects. This randomized, placebo-controlled, double-blind, 12-week study was designed to test this hypothesis by comparing the efficacy and upper gastrointestinal tract safety of valdecoxib with that of naproxen, a leading conventional NSAID comparator.

Methods

Study population

Ambulatory adults who had been diagnosed with moderate to severe osteoarthritis of the knee according to the modified criteria of the American College of Rheumatology^6,7 were eligible to participate in the trial. Patients were recruited from primary care and rheumatology specialty settings. Patients who had baseline scores of at least 40 mm on the Patient’s Assessment of Arthritis Pain–Visual Analog Scale (PAAP-VAS) and baseline categorical scores of poor to very poor on the Patient’s (PaGAA) and Physician’s (PhGAA) Global Assessments of Arthritis were included.^8,9 Any patient suffering from inflammatory arthritis, gout, pseudogout, Paget disease, or any chronic pain syndrome that might interfere with assessment of the Index Knee was excluded from the trial. Patients diagnosed with osteoarthritis of the hip ipsilateral to the Index Knee, severe anserine bursitis, acute joint trauma, or complete loss of articular cartilage on the Index Knee also were excluded. Patients were not eligible if they had active gastrointestinal disease, gastrointestinal tract ulceration 30 days before the trial, a significant bleeding disorder, or a history of gastric or duodenal surgery. Patients with an esophageal, gastric, pyloric channel, or duodenal ulcer or a score of at least 10 for esophageal, gastric, or duodenal erosions at the pretreatment endoscopy examination also were excluded.

FIGURE 1
Patient’s global assessment of arthritis

Study design

This multicenter, randomized, double-blind, placebo-controlled study compared the efficacy and upper gastrointestinal tract safety of valdecoxib at dosages of 5, 10, and 20 mg once daily with placebo and naproxen at a dosage of 500 mg twice daily in relieving moderate to severe osteoarthritis of the knee. The trial was conducted in 85 centers in the United States and Canada, in accordance with the principles of good clinical practice and the Declaration of Helsinki. Eligible patients were randomized to treatment groups and self-administered oral study medication. Patients were randomized to study treatment in the order in which they were enrolled into the study by using a treatment sequence that was determined by a Searle-prepared computer-generated randomization schedule. Patients received their allocated study medications in bottles labeled A and B according to the randomization schedule. Personnel at the study centers carried out the assessments and remained blinded throughout the study. Eligible patients were enrolled and discontinued regular pain medication. Patients discontinued their normal medications at the following specified times before the baseline endoscopy: NSAIDs (including full-dose aspirin at a dosage of ≥325 mg/day) at 48 hours, corticosteroid injections at 4 weeks, and intra-articular injections of corticosteroid or hyaluronic acid preparations at 3 and 6 months, respectively. The use of antiulcer drugs, including H2 blockers, proton pump inhibitors, misoprostol, and sucralfate, was discontinued at least 24 hours before the baseline endoscopy.

Efficacy assessments

The following arthritis assessments were made at baseline and at 2, 6, and 12 weeks or at early termination after study drug administration. PaGAA or PhGAA was measured on a 5-point categorical scale, where 1 = very good, 2 = good, 3 = fair, 4 = poor, and 5 = very poor. The PAAP-VAS was measured on a scale of 0 to 100 mm, where 0 = no pain and 100 = most severe pain. The Western Ontario and McMaster’s Universities (WOMAC) Osteoarthritis indices including Pain, Stiffness, Physical Function, and Composite were measured as described previously.¹⁰

Upper gastrointestinal assessments

Upper gastrointestinal tract endoscopy was performed within 7 days before the first study dose and at the 12-week assessment or at early termination if the patient withdrew. An endoscopy could be performed at any time if the patient experienced symptoms suggestive of an ulcer. The endoscopists performing baseline and 12-week (early termination) assessments remained blinded throughout the study.

General safety assessments

Clinical laboratory tests were performed at screening, baseline, weeks 2, 6, and 12, or at early termination, and a complete physical examination was performed at screening and final visits. The incidence of adverse events occurring in each treatment arm was monitored throughout the study. Adverse events occurring within 7 days and serious adverse events occurring within 30 days of the last study dosage of medication were included in the safety analyses.

Statistical analyses

A sample size of 200 patients per treatment group was deemed sufficient to detect a difference in ulcer rates of 5% for valdecoxib vs 16% for naproxen, with 80% power and type 1 error at .017 (adjusted for 3 primary comparisons against placebo). Homogeneity of treatment groups at baseline with respect to age, height, weight, duration of osteoarthritis, PAAP-VAS, and WOMAC Osteoarthritis Index scores was assessed with 2-way analysis of variance, with treatment group and center as factors. All other demographics and baseline characteristics were compared with the Cochran-Mantel-Haenszel (CMH) test, stratified by center.

All efficacy assessments were performed on the modified intent-to-treat (ITT) cohort by using the last observation carried forward approach. The ITT cohort comprised all patients who were randomized and had taken at least 1 dose of study medication. Analyses of mean change from baseline for PaGAA, PhGAA, PAAP-VAS, and WOMAC Osteoarthritis indices were performed by using analysis of covariance, with treatment and center as factors and the corresponding baseline score as the covariate. Pairwise comparisons of valdecoxib at dosages of 10 and 20 mg once daily vs placebo were interpreted with the Hochberg procedure.¹¹ Primary pairwise comparisons were amended in the statistical analysis plan before data unblinding to compare placebo with 10 and 20 mg valdecoxib, but not with the 5-mg dose. For all other comparisons, including 5 mg valdecoxib and naproxen vs placebo, differences were considered significant if the pairwise P values were less than .05. The incidence of withdrawal due to treatment failure was analyzed by the Fisher exact test, and the time to withdrawal in each treatment group was analyzed by log-rank test and plotted with the Kaplan-Meier product limit.^12,13

Upper gastrointestinal tract endoscopic analyses were performed on the upper gastrointestinal tract ITT population. Randomized patients were included in this cohort if they received at least 1 dose of study medication and had undergone pretreatment and posttreatment endoscopies. Overall and pairwise comparisons of gastroduodenal, gastric, and duodenal ulcers and erosions were assessed with the CMH test stratified by center. The incidence of adverse events was compared between treatment groups with the Fisher exact test. Changes in vital signs were compared between treatment groups with an analysis of covariance using pairwise treatment comparisons, with treatment group as a factor and baseline value as a covariate.

Results

Patient baseline characteristics

Of the 1019 eligible randomized patients, 1 patient randomized to 10 mg/day valdecoxib, 1 to 20 mg/day valdecoxib, and 1 to 500 mg naproxen twice daily did not take the study medication and were excluded from efficacy and safety analyses. The remaining 1016 randomized patients received study medication and were included in the ITT cohort on which analyses of all efficacy end points were based. A total of 269 patients withdrew before the end of the study due to treatment failure, preexisting protocol violations, noncompliance, or adverse signs and symptoms, or were lost to follow-up: 74 patients in the placebo group, 39 in the 5-mg valdecoxib group, 56 in the 10-mg valdecoxib group, 44 in the 20-mg valdecoxib group, and 56 in the naproxen group. The upper gastrointestinal tract ITT cohort comprised 908 patients who were included in the upper gastrointestinal tract safety analyses. More than 90% of patients included in the study evaluated their osteoarthritis as poor to very poor as assessed by baseline PaGAA scores. Treatment groups were homogeneous with respect to demographics, vital signs, medical history, and all baseline arthritis assessments (Table 1).

TABLE 1
Patient baseline characteristics

Efficacy

The least square mean change in the PaGAA was significantly improved at most assessments in response to valdecoxib (10 and 20 mg/day) and 500 mg naproxen twice daily compared with placebo (Table 2). However, the improvement in response to valdecoxib 5 mg qd did not reach statistical significance (Table 2). Significant improvements in the PhGAA were observed in response to valdecoxib and naproxen at all assessments (Table 2).

The dosages of 20 mg/day valdecoxib and 500 mg naproxen twice daily were associated with a reduction in pain, as assessed by the PAAP-VAS scores. Pain reduction associated with 5 and 10 mg/day valdecoxib was significantly better than that with placebo at all assessments except for week 12 (Table 2).

Valdecoxib and naproxen treatments improved the WOMAC Pain, Stiffness, Physical Function, and Composite indices compared with placebo at 2, 6, and 12 weeks. Valdecoxib 20 mg/day and naproxen 500 mg twice daily produced statistically significant changes in all WOMAC Osteoarthritis scores throughout the 12-week study period compared with placebo (P < .05). WOMAC Pain scores for 10 mg valdecoxib were significantly different from those for placebo at 2 weeks (P < .001) but not at 6 or 12 weeks. No significant differences were noted between any of the valdecoxib treatment doses and naproxen in terms of improvement in WOMAC indices.

The incidences of withdrawal due to treatment failure were 20% (95% confidence interval [CI], 15.3–26.8) in the placebo group; 8% (95% CI, 4.8–12.8), 12% (95% CI, 7.8–17.1), and 10% (95% CI, 6.3–15.2) in the 5-, 10-, and 20-mg/day valdecoxib groups; and 6% (95% CI, 3.6–10.9) in the 500-mg naproxen group (P < .05; Table 3). Patients in the placebo group withdrew at a significantly faster rate than those in the 4 active treatment groups (P < .05), but there were no significant differences in withdrawal rates across the 4 active treatment groups.

TABLE 2
Baseline arthritis assessments and mean changes from baseline scores