Warren J. McIsaac, MD, MSc

In the past decade, bacterial resistance to commonly used antibiotics has risen dramatically.^1,2 While a number of factors have contributed to this problem, overuse of antibiotics by physicians has been implicated.^3-6 An association has been demonstrated between the volume of antibiotic prescriptions and bacterial resistance at both a national^4,5 and a local level.³ Where prescribing by physicians has been reduced, rates of antibiotic resistance have subsequently been observed to decline.^5,6 As a result, physicians have been urged to reduce their use of antibiotics.^7,8 Respiratory infections are the most common reason for the prescribing of antibiotics.⁹ Upper respiratory tract infections (URTIs) and pharyngitis account for 19% to 28% of all antibiotic prescriptions written by family physicians.^9-11

While the use of antibiotics for URTI with sore throat is frequently debated,^12-14 experts continue to recommend such treatment for group A streptococcus (GAS) infections to prevent rheumatic fever.^15,16 However, only 10% to 20% of patients with a sore throat who visit a family physician have a GAS infection,^17-19 whereas antibiotics are prescribed for 50% of URTIs¹⁰ and 90% of cases of tonsillitis.²⁰ Uncertainty as to whether or not a bacterial infection is present and clinical error in estimating the likelihood of a GAS infection are associated with the unnecessary prescription of antibiotics.^21,22 To address clinical uncertainty, a number of prediction rules and clinical scores have been proposed.^23-30 However, physicians taught simply to generate more accurate estimates of the likelihood of a strep infection in this manner do not necessarily lower their use of antibiotics.³¹

We have previously shown that linking score estimates for the likelihood of a GAS infection to explicit management recommendations to take a throat swab or prescribe an antibiotic has the potential to lower antibiotic use significantly.^19,32 In an observational study involving 621 children and adults, this approach would have reduced unnecessary antibiotic prescriptions by 63%.³² We also found a trend toward reduced antibiotic use when physicians were provided with an explicit reminder about the score approach.³³ As a result, we hypothesized that this might also help physicians to learn to adopt the sore throat score approach. Reminders have been found to improve the delivery of preventive health services.^34,35 The objective of this study was to determine whether repeated clinical prompts to community-based family physicians about the score approach could reduce unnecessary antibiotic prescriptions and lower overall antibiotic use for patients with a sore throat.

Methods

In the fall of 1998, a sample of family physicians in the province of Ontario were invited to participate in a trial to reduce antibiotic use in patients with a sore throat. Physicians who had previously participated in practice-based research projects for the College of Family Physicians of Canada and a random sample from the College’s general membership listing were contacted. Those who mailed back a reply card indicating that they wished to participate were randomized to either an intervention or a control group. The study was approved by the University of Toronto Ethics Review Committee.

Both groups of physicians received, by mail, an article describing the clinical score management approach¹⁹; a laminated pocket card summarizing the method; clinical encounter and patient consent forms; and a 1-page survey of practice characteristics. Each physician was asked to enroll 8 patients aged 3 years or older whom they believed to have a new URTI with a sore throat. No attempt was made to further define an eligible presentation to encourage physicians to enroll cases representative of their usual practice. Patients were ineligible if they had taken antibiotics during the previous week, were immunocompromised, or could not understand English. Parents were asked to provide consent for children younger than 16 years of age.

A brief standardized assessment form was completed by the physician for each patient and a throat swab was obtained. The throat swab was submitted to the physician’s local laboratory. A copy of the culture result was forwarded to the study center. Treatment decisions and the management of subsequent culture results were the responsibility of the treating physician.

In the intervention group, physicians were provided with a sticker to apply to the encounter form that listed the score management approach. The sticker contained boxes to be checked by the physicians to calculate the score total and determine appropriate management. Physicians not wishing to use the sticker were prompted on the form to write the score total in a space provided. As a result, physicians in the intervention group received repeated prompts that reminded them to use the score approach each time they completed a clinical encounter form. The control group completed a similar form but without either the sticker or the chart prompts.

The details of the clinical score approach have been previously published.^19,25 Briefly, 4 clinical findings (fever > 38°C, absence of cough, tender anterior cervical adenopathy, tonsillar swelling or exudate) and age < 15 years are each assigned 1 point and totaled. One point is subtracted for age 45 years or more. Explicit recommendations for management are linked to score totals. If the score total is 1 or less, no throat swab or antibiotic is indicated. A throat swab is recommended for a score of 2 or 3 and an antibiotic only if the culture is positive. Either initiating treatment with an antibiotic or taking a throat swab is appropriate for a score of 4 or more.

The main outcome for the study was the prescription of unnecessary antibiotics, defined as a prescription for antibiotic medication given to a patient whose subsequent throat culture was negative for group A streptococcus. The secondary outcome was overall antibiotic use. The sample size was calculated to detect a 30% decrease in unnecessary antibiotic use (2-sided = 0.05, 1- = 0.90), assuming a 40% baseline prescription rate⁹ and a 70% negative culture rate.¹⁷¹⁹ Because groups of patients were treated by the same physician, the sample size was adjusted to take the clustered sampling design into account.³⁶ The intraclass correlation coefficient for prescribing estimated from an earlier study was 0.07.¹⁹ Assuming an average of 5 patients assessed per physician, the sample size was estimated to be 85 physicians and 425 patients in each group.

The clinical characteristics of patients in the intervention and control groups were compared with a chi-square test for categorical variables and a t-test for continuous variables. Associations between prescription rates and the practice and demographic characteristics of the physicians were assessed and adjusted for the clustered sampling with Stata Statistical Software (Release 6, Stata Corp., College Station, Tex.). While clustering improves the efficiency of sampling by requiring participation by fewer physicians, confidence intervals that do not account for the design effect are too narrow. Multiple logistic regression was used to adjust for differences in patient and physician characteristics, taking into account the patient clusters by physician in estimating the effect of the intervention.

Results

One hundred sixty-four physicians agreed to participate and were randomized. Of these, only 97 (59.1%) completed the study and provided patient data (Figure). An equal proportion of physicians in the intervention group (40.2%) and control group (41.5%, P = .87) failed to complete the study. No significant differences were identified between the sex or age of physicians who participated and those who did not participate. Of the participating physicians, 86 (84.3%) returned surveys describing their practice settings.

Patients assessed included 692 children and adults. Of these, 71 (10.3%) were excluded because of a diagnosis of bronchitis (35), sinusitis (16), otitis media (11), or pneumonia (4) or because the patient was less than 3 years old (5). The score approach did not apply to the 4 conditions of exclusion because they involve organisms other than GAS. The remaining 621 patients in the control and intervention groups were similar in demographic and clinical characteristics as well as regarding the prevalence of GAS as documented by throat culture (Table 1). However, a diagnosis of tonsillitis, strep throat, or pharyngitis was more likely to occur in the intervention group (38.6%) than in the control group (28.9%, P = .01). These diagnoses were associated with a higher rate of antibiotic prescribing (54.8%) than were situations in which physicians recorded a URTI or other diagnosis (14.2%, P < .001).

Differences were noted in the characteristics of the treating physicians in each group when considered by patient encounter. Although there were no differences in the age or sex of individual physicians in each group, the participating physicians did not contribute equal numbers of patient encounters. The average number of patients assessed per physician was 3, ranging from a low of 1 patient contributed by some physicians to a high of 8 for others. More patient encounters in the intervention group were contributed by male physicians who had been in practice longer, who worked in smaller communities, and who reported larger practice volumes (Table 1). Physicians from small communities were more likely to diagnose strep throat, tonsillitis, or pharyngitis than were those in larger communities (45.1% vs 28.1%, respectively, P < .001), as were those with higher patient volumes (46.5% vs 30.2%, P = .003).

Certain physician practice characteristics were associated with a patient’s being more likely to receive a prescription for an unnecessary antibiotic (Table 2). For example, physicians were more likely to prescribe unnecessary antibiotics if they saw more than 150 patients per week than if they saw fewer and if they had been in practice for 20 or more years than if they had practiced for a shorter time. In addition, higher overall antibiotic use was associated with higher patient volume and with practicing in a smaller community.

There were no differences between the intervention and control groups in either unnecessary antibiotic prescriptions (20.4% vs 16.1%, respectively, P = .17) or overall antibiotic use (28.1% vs 27.9%, P = .96) (Table 1). However, while the culture reports that were needed to classify prescriptions as unnecessary were available for most (600) patients (96.6%), significantly more culture reports were missing in the control group (5.4%) than in the intervention group (1.2%, P = .007). Antibiotics were prescribed in 59% of the 17 cases with missing culture reports in the control group but for none of the 4 cases with missing culture reports in the intervention group.

Because intervention patients were more likely than controls to have been treated by physicians with higher prescribing characteristics, adjustments were made for the differing physician characteristics and diagnostic practices and for the clustering of patients by physician, using multiple logistic regression (Table 3). After adjustment, the intervention was associated with a nonsignificant reduction in unnecessary antibiotic prescriptions (odds ratio [OR] = 0.76, 95% confidence interval [CI] = 0.42, 1.40) and in overall antibiotic use (OR = 0.57, 95% CI = 0.27, 1.17).

TABLE 1
COMPARISON OF PATIENTS IN CONTROL AND INTERVENTION GROUPS

TABLE 2
ASSOCIATION BETWEEN INDIVIDUAL PHYSICIAN FACTORS* AND ANTIBIOTIC PRESCRIBING, ADJUSTING FOR THE CLUSTERING OF PATIENTS BY PHYSICIAN

TABLE 3
EFFECT OF REPEATED CHART PROMPTS ON PRESCRIBING RATES, ADJUSTING FOR PHYSICIAN FACTORS AND CLUSTERING* OF PATIENTS BY PHYSICIAN (N = 453†)

Figure
FAMILY PHYSICIANS WHO WERE CONTACTED AND WHO COMPLETED THE STUDY

Discussion

The use of repeated chart prompt reminders to family physicians to use a clinical scoring approach in the management of children and adults presenting with URTI and a sore throat did not affect unnecessary antibiotic prescriptions or overall antibiotic use. Problems encountered in conducting this community-based trial may have contributed to the negative result.

Sixty-seven (41%) physicians agreed to be randomized but failed to complete the study. These losses after randomization and the differing sizes of the patient clusters per physician led to differences in the characteristics of the treating physician between the 2 groups. Characteristics associated with higher antibiotic prescribing rates were more common in the intervention group. As a result, despite the randomized design, the 2 patient groups were not initially similar in terms of their likelihood to receive a prescription for an antibiotic. To compensate for these differences, we controlled for the different physician characteristics in the analysis. However, the large number of physician dropouts also resulted in a failure to achieve the planned sample size. As a result, the study had insufficient power to detect the effect size that had been hypothesized.

We had planned the sample size to detect a 30% decrease in unnecessary antibiotic use. The adjusted analysis produced a point estimate of a 23% decrease in unnecessary antibiotic use and a 43% decrease in overall antibiotic use. These point estimates are the same whether or not the clustering is taken into effect; however, the more appropriate clustered analysis increases the estimate for the sample variance, resulting in wider confidence intervals. Examination of the lower 95% confidence interval reveals that the study lacked sufficient power to rule out as much as a 58% reduction in unnecessary antibiotic use. Therefore, while the study failed to find a statistically significant effect from the intervention, it also did not have the power to rule out a clinically important reduction in unnecessary antibiotic use.

We gave information about the clinical scoring approach to physicians in the control group. Doing so may have reduced the study’s ability to detect an effect of the intervention. We did not include a group that had been not exposed to information because we believed that mailed information was the equivalent of “standard” care in terms of changing physician behavior. Mailed information is a common method of informing physicians about new clinical information but has a limited ability to influence clinical behavior.³⁴ However, the rate of antibiotic prescribing in the control group was indeed somewhat lower than is generally reported in the literature.⁹ This finding may be compatible with volunteer bias or the Hawthorne effect. More likely, perhaps, asking the control group to complete encounter forms for multiple patients may have inadvertently reminded them about the score. As a result, the control group may have been contaminated from repeated clinical prompts.

Some problems encountered in this study have been noted by other investigators conducting community-based research in primary care.³⁷ The difficulty of retaining community-based physicians resulted in significant losses after randomization. This situation occurred even though qualifying to be randomized required physicians to mail back a reply card indicating that they wished to participate, suggesting that they were motivated to some degree.³⁷ In addition, they received a modest cash honorarium. Some physicians returned the package stating that circumstances had changed and they would be unable to participate. Many who initially agreed to participate failed to reply despite 3 mailed reminders. The level of dropouts did not become apparent until late in the study. In retrospect, it might have been advisable to phone physicians directly soon after randomization in order to detect problems early. Other physicians could then have been randomly selected from the general membership listing to replace those who had dropped out.

This study found that repeated reminders to physicians to use a clinical score in the management of their patients with a sore throat did not reduce unnecessary antibiotic use. The problems encountered in this community-based intervention trial may have contributed to the negative result. Studies of prescribing behavior may need to stratify physicians before randomization by characteristics, such as patient volume and experience, that are related to prescribing behavior. Including a group that received no information is probably necessary to allow the greatest chance of detecting an effect. Particular attention and resources need to be available to ensure the retention, and replacement if needed, of community-based family physicians participating in research studies.

Acknowledgments

This study was supported by a grant from the Medical Research Council of Canada, Grant No. MA-15088. Dr McIsaac’s work is supported by the Mt. Sinai Hospital and the Family Healthcare Research Unit of the Department of Family and Community Medicine, University of Toronto, Toronto, Canada. This study was conducted in conjunction with the National Research System of the College of Family Physicians of Canada. The cooperation of the Ontario Association of Medical Laboratories is gratefully acknowledged.

In the past decade, bacterial resistance to commonly used antibiotics has risen dramatically.^1,2 While a number of factors have contributed to this problem, overuse of antibiotics by physicians has been implicated.^3-6 An association has been demonstrated between the volume of antibiotic prescriptions and bacterial resistance at both a national^4,5 and a local level.³ Where prescribing by physicians has been reduced, rates of antibiotic resistance have subsequently been observed to decline.^5,6 As a result, physicians have been urged to reduce their use of antibiotics.^7,8 Respiratory infections are the most common reason for the prescribing of antibiotics.⁹ Upper respiratory tract infections (URTIs) and pharyngitis account for 19% to 28% of all antibiotic prescriptions written by family physicians.^9-11

While the use of antibiotics for URTI with sore throat is frequently debated,^12-14 experts continue to recommend such treatment for group A streptococcus (GAS) infections to prevent rheumatic fever.^15,16 However, only 10% to 20% of patients with a sore throat who visit a family physician have a GAS infection,^17-19 whereas antibiotics are prescribed for 50% of URTIs¹⁰ and 90% of cases of tonsillitis.²⁰ Uncertainty as to whether or not a bacterial infection is present and clinical error in estimating the likelihood of a GAS infection are associated with the unnecessary prescription of antibiotics.^21,22 To address clinical uncertainty, a number of prediction rules and clinical scores have been proposed.^23-30 However, physicians taught simply to generate more accurate estimates of the likelihood of a strep infection in this manner do not necessarily lower their use of antibiotics.³¹

We have previously shown that linking score estimates for the likelihood of a GAS infection to explicit management recommendations to take a throat swab or prescribe an antibiotic has the potential to lower antibiotic use significantly.^19,32 In an observational study involving 621 children and adults, this approach would have reduced unnecessary antibiotic prescriptions by 63%.³² We also found a trend toward reduced antibiotic use when physicians were provided with an explicit reminder about the score approach.³³ As a result, we hypothesized that this might also help physicians to learn to adopt the sore throat score approach. Reminders have been found to improve the delivery of preventive health services.^34,35 The objective of this study was to determine whether repeated clinical prompts to community-based family physicians about the score approach could reduce unnecessary antibiotic prescriptions and lower overall antibiotic use for patients with a sore throat.

Methods

In the fall of 1998, a sample of family physicians in the province of Ontario were invited to participate in a trial to reduce antibiotic use in patients with a sore throat. Physicians who had previously participated in practice-based research projects for the College of Family Physicians of Canada and a random sample from the College’s general membership listing were contacted. Those who mailed back a reply card indicating that they wished to participate were randomized to either an intervention or a control group. The study was approved by the University of Toronto Ethics Review Committee.

Both groups of physicians received, by mail, an article describing the clinical score management approach¹⁹; a laminated pocket card summarizing the method; clinical encounter and patient consent forms; and a 1-page survey of practice characteristics. Each physician was asked to enroll 8 patients aged 3 years or older whom they believed to have a new URTI with a sore throat. No attempt was made to further define an eligible presentation to encourage physicians to enroll cases representative of their usual practice. Patients were ineligible if they had taken antibiotics during the previous week, were immunocompromised, or could not understand English. Parents were asked to provide consent for children younger than 16 years of age.

A brief standardized assessment form was completed by the physician for each patient and a throat swab was obtained. The throat swab was submitted to the physician’s local laboratory. A copy of the culture result was forwarded to the study center. Treatment decisions and the management of subsequent culture results were the responsibility of the treating physician.

In the intervention group, physicians were provided with a sticker to apply to the encounter form that listed the score management approach. The sticker contained boxes to be checked by the physicians to calculate the score total and determine appropriate management. Physicians not wishing to use the sticker were prompted on the form to write the score total in a space provided. As a result, physicians in the intervention group received repeated prompts that reminded them to use the score approach each time they completed a clinical encounter form. The control group completed a similar form but without either the sticker or the chart prompts.

The details of the clinical score approach have been previously published.^19,25 Briefly, 4 clinical findings (fever > 38°C, absence of cough, tender anterior cervical adenopathy, tonsillar swelling or exudate) and age < 15 years are each assigned 1 point and totaled. One point is subtracted for age 45 years or more. Explicit recommendations for management are linked to score totals. If the score total is 1 or less, no throat swab or antibiotic is indicated. A throat swab is recommended for a score of 2 or 3 and an antibiotic only if the culture is positive. Either initiating treatment with an antibiotic or taking a throat swab is appropriate for a score of 4 or more.

The main outcome for the study was the prescription of unnecessary antibiotics, defined as a prescription for antibiotic medication given to a patient whose subsequent throat culture was negative for group A streptococcus. The secondary outcome was overall antibiotic use. The sample size was calculated to detect a 30% decrease in unnecessary antibiotic use (2-sided = 0.05, 1- = 0.90), assuming a 40% baseline prescription rate⁹ and a 70% negative culture rate.¹⁷¹⁹ Because groups of patients were treated by the same physician, the sample size was adjusted to take the clustered sampling design into account.³⁶ The intraclass correlation coefficient for prescribing estimated from an earlier study was 0.07.¹⁹ Assuming an average of 5 patients assessed per physician, the sample size was estimated to be 85 physicians and 425 patients in each group.

The clinical characteristics of patients in the intervention and control groups were compared with a chi-square test for categorical variables and a t-test for continuous variables. Associations between prescription rates and the practice and demographic characteristics of the physicians were assessed and adjusted for the clustered sampling with Stata Statistical Software (Release 6, Stata Corp., College Station, Tex.). While clustering improves the efficiency of sampling by requiring participation by fewer physicians, confidence intervals that do not account for the design effect are too narrow. Multiple logistic regression was used to adjust for differences in patient and physician characteristics, taking into account the patient clusters by physician in estimating the effect of the intervention.

Results

One hundred sixty-four physicians agreed to participate and were randomized. Of these, only 97 (59.1%) completed the study and provided patient data (Figure). An equal proportion of physicians in the intervention group (40.2%) and control group (41.5%, P = .87) failed to complete the study. No significant differences were identified between the sex or age of physicians who participated and those who did not participate. Of the participating physicians, 86 (84.3%) returned surveys describing their practice settings.

Patients assessed included 692 children and adults. Of these, 71 (10.3%) were excluded because of a diagnosis of bronchitis (35), sinusitis (16), otitis media (11), or pneumonia (4) or because the patient was less than 3 years old (5). The score approach did not apply to the 4 conditions of exclusion because they involve organisms other than GAS. The remaining 621 patients in the control and intervention groups were similar in demographic and clinical characteristics as well as regarding the prevalence of GAS as documented by throat culture (Table 1). However, a diagnosis of tonsillitis, strep throat, or pharyngitis was more likely to occur in the intervention group (38.6%) than in the control group (28.9%, P = .01). These diagnoses were associated with a higher rate of antibiotic prescribing (54.8%) than were situations in which physicians recorded a URTI or other diagnosis (14.2%, P < .001).

Differences were noted in the characteristics of the treating physicians in each group when considered by patient encounter. Although there were no differences in the age or sex of individual physicians in each group, the participating physicians did not contribute equal numbers of patient encounters. The average number of patients assessed per physician was 3, ranging from a low of 1 patient contributed by some physicians to a high of 8 for others. More patient encounters in the intervention group were contributed by male physicians who had been in practice longer, who worked in smaller communities, and who reported larger practice volumes (Table 1). Physicians from small communities were more likely to diagnose strep throat, tonsillitis, or pharyngitis than were those in larger communities (45.1% vs 28.1%, respectively, P < .001), as were those with higher patient volumes (46.5% vs 30.2%, P = .003).

Certain physician practice characteristics were associated with a patient’s being more likely to receive a prescription for an unnecessary antibiotic (Table 2). For example, physicians were more likely to prescribe unnecessary antibiotics if they saw more than 150 patients per week than if they saw fewer and if they had been in practice for 20 or more years than if they had practiced for a shorter time. In addition, higher overall antibiotic use was associated with higher patient volume and with practicing in a smaller community.

There were no differences between the intervention and control groups in either unnecessary antibiotic prescriptions (20.4% vs 16.1%, respectively, P = .17) or overall antibiotic use (28.1% vs 27.9%, P = .96) (Table 1). However, while the culture reports that were needed to classify prescriptions as unnecessary were available for most (600) patients (96.6%), significantly more culture reports were missing in the control group (5.4%) than in the intervention group (1.2%, P = .007). Antibiotics were prescribed in 59% of the 17 cases with missing culture reports in the control group but for none of the 4 cases with missing culture reports in the intervention group.

Because intervention patients were more likely than controls to have been treated by physicians with higher prescribing characteristics, adjustments were made for the differing physician characteristics and diagnostic practices and for the clustering of patients by physician, using multiple logistic regression (Table 3). After adjustment, the intervention was associated with a nonsignificant reduction in unnecessary antibiotic prescriptions (odds ratio [OR] = 0.76, 95% confidence interval [CI] = 0.42, 1.40) and in overall antibiotic use (OR = 0.57, 95% CI = 0.27, 1.17).

TABLE 1
COMPARISON OF PATIENTS IN CONTROL AND INTERVENTION GROUPS

TABLE 2
ASSOCIATION BETWEEN INDIVIDUAL PHYSICIAN FACTORS* AND ANTIBIOTIC PRESCRIBING, ADJUSTING FOR THE CLUSTERING OF PATIENTS BY PHYSICIAN

TABLE 3
EFFECT OF REPEATED CHART PROMPTS ON PRESCRIBING RATES, ADJUSTING FOR PHYSICIAN FACTORS AND CLUSTERING* OF PATIENTS BY PHYSICIAN (N = 453†)

Figure
FAMILY PHYSICIANS WHO WERE CONTACTED AND WHO COMPLETED THE STUDY

Discussion

The use of repeated chart prompt reminders to family physicians to use a clinical scoring approach in the management of children and adults presenting with URTI and a sore throat did not affect unnecessary antibiotic prescriptions or overall antibiotic use. Problems encountered in conducting this community-based trial may have contributed to the negative result.

Sixty-seven (41%) physicians agreed to be randomized but failed to complete the study. These losses after randomization and the differing sizes of the patient clusters per physician led to differences in the characteristics of the treating physician between the 2 groups. Characteristics associated with higher antibiotic prescribing rates were more common in the intervention group. As a result, despite the randomized design, the 2 patient groups were not initially similar in terms of their likelihood to receive a prescription for an antibiotic. To compensate for these differences, we controlled for the different physician characteristics in the analysis. However, the large number of physician dropouts also resulted in a failure to achieve the planned sample size. As a result, the study had insufficient power to detect the effect size that had been hypothesized.

We had planned the sample size to detect a 30% decrease in unnecessary antibiotic use. The adjusted analysis produced a point estimate of a 23% decrease in unnecessary antibiotic use and a 43% decrease in overall antibiotic use. These point estimates are the same whether or not the clustering is taken into effect; however, the more appropriate clustered analysis increases the estimate for the sample variance, resulting in wider confidence intervals. Examination of the lower 95% confidence interval reveals that the study lacked sufficient power to rule out as much as a 58% reduction in unnecessary antibiotic use. Therefore, while the study failed to find a statistically significant effect from the intervention, it also did not have the power to rule out a clinically important reduction in unnecessary antibiotic use.

We gave information about the clinical scoring approach to physicians in the control group. Doing so may have reduced the study’s ability to detect an effect of the intervention. We did not include a group that had been not exposed to information because we believed that mailed information was the equivalent of “standard” care in terms of changing physician behavior. Mailed information is a common method of informing physicians about new clinical information but has a limited ability to influence clinical behavior.³⁴ However, the rate of antibiotic prescribing in the control group was indeed somewhat lower than is generally reported in the literature.⁹ This finding may be compatible with volunteer bias or the Hawthorne effect. More likely, perhaps, asking the control group to complete encounter forms for multiple patients may have inadvertently reminded them about the score. As a result, the control group may have been contaminated from repeated clinical prompts.

Some problems encountered in this study have been noted by other investigators conducting community-based research in primary care.³⁷ The difficulty of retaining community-based physicians resulted in significant losses after randomization. This situation occurred even though qualifying to be randomized required physicians to mail back a reply card indicating that they wished to participate, suggesting that they were motivated to some degree.³⁷ In addition, they received a modest cash honorarium. Some physicians returned the package stating that circumstances had changed and they would be unable to participate. Many who initially agreed to participate failed to reply despite 3 mailed reminders. The level of dropouts did not become apparent until late in the study. In retrospect, it might have been advisable to phone physicians directly soon after randomization in order to detect problems early. Other physicians could then have been randomly selected from the general membership listing to replace those who had dropped out.

This study found that repeated reminders to physicians to use a clinical score in the management of their patients with a sore throat did not reduce unnecessary antibiotic use. The problems encountered in this community-based intervention trial may have contributed to the negative result. Studies of prescribing behavior may need to stratify physicians before randomization by characteristics, such as patient volume and experience, that are related to prescribing behavior. Including a group that received no information is probably necessary to allow the greatest chance of detecting an effect. Particular attention and resources need to be available to ensure the retention, and replacement if needed, of community-based family physicians participating in research studies.

Acknowledgments

This study was supported by a grant from the Medical Research Council of Canada, Grant No. MA-15088. Dr McIsaac’s work is supported by the Mt. Sinai Hospital and the Family Healthcare Research Unit of the Department of Family and Community Medicine, University of Toronto, Toronto, Canada. This study was conducted in conjunction with the National Research System of the College of Family Physicians of Canada. The cooperation of the Ontario Association of Medical Laboratories is gratefully acknowledged.

In the past decade, bacterial resistance to commonly used antibiotics has risen dramatically.^1,2 While a number of factors have contributed to this problem, overuse of antibiotics by physicians has been implicated.^3-6 An association has been demonstrated between the volume of antibiotic prescriptions and bacterial resistance at both a national^4,5 and a local level.³ Where prescribing by physicians has been reduced, rates of antibiotic resistance have subsequently been observed to decline.^5,6 As a result, physicians have been urged to reduce their use of antibiotics.^7,8 Respiratory infections are the most common reason for the prescribing of antibiotics.⁹ Upper respiratory tract infections (URTIs) and pharyngitis account for 19% to 28% of all antibiotic prescriptions written by family physicians.^9-11

While the use of antibiotics for URTI with sore throat is frequently debated,^12-14 experts continue to recommend such treatment for group A streptococcus (GAS) infections to prevent rheumatic fever.^15,16 However, only 10% to 20% of patients with a sore throat who visit a family physician have a GAS infection,^17-19 whereas antibiotics are prescribed for 50% of URTIs¹⁰ and 90% of cases of tonsillitis.²⁰ Uncertainty as to whether or not a bacterial infection is present and clinical error in estimating the likelihood of a GAS infection are associated with the unnecessary prescription of antibiotics.^21,22 To address clinical uncertainty, a number of prediction rules and clinical scores have been proposed.^23-30 However, physicians taught simply to generate more accurate estimates of the likelihood of a strep infection in this manner do not necessarily lower their use of antibiotics.³¹

We have previously shown that linking score estimates for the likelihood of a GAS infection to explicit management recommendations to take a throat swab or prescribe an antibiotic has the potential to lower antibiotic use significantly.^19,32 In an observational study involving 621 children and adults, this approach would have reduced unnecessary antibiotic prescriptions by 63%.³² We also found a trend toward reduced antibiotic use when physicians were provided with an explicit reminder about the score approach.³³ As a result, we hypothesized that this might also help physicians to learn to adopt the sore throat score approach. Reminders have been found to improve the delivery of preventive health services.^34,35 The objective of this study was to determine whether repeated clinical prompts to community-based family physicians about the score approach could reduce unnecessary antibiotic prescriptions and lower overall antibiotic use for patients with a sore throat.

Methods

In the fall of 1998, a sample of family physicians in the province of Ontario were invited to participate in a trial to reduce antibiotic use in patients with a sore throat. Physicians who had previously participated in practice-based research projects for the College of Family Physicians of Canada and a random sample from the College’s general membership listing were contacted. Those who mailed back a reply card indicating that they wished to participate were randomized to either an intervention or a control group. The study was approved by the University of Toronto Ethics Review Committee.

Both groups of physicians received, by mail, an article describing the clinical score management approach¹⁹; a laminated pocket card summarizing the method; clinical encounter and patient consent forms; and a 1-page survey of practice characteristics. Each physician was asked to enroll 8 patients aged 3 years or older whom they believed to have a new URTI with a sore throat. No attempt was made to further define an eligible presentation to encourage physicians to enroll cases representative of their usual practice. Patients were ineligible if they had taken antibiotics during the previous week, were immunocompromised, or could not understand English. Parents were asked to provide consent for children younger than 16 years of age.

A brief standardized assessment form was completed by the physician for each patient and a throat swab was obtained. The throat swab was submitted to the physician’s local laboratory. A copy of the culture result was forwarded to the study center. Treatment decisions and the management of subsequent culture results were the responsibility of the treating physician.

In the intervention group, physicians were provided with a sticker to apply to the encounter form that listed the score management approach. The sticker contained boxes to be checked by the physicians to calculate the score total and determine appropriate management. Physicians not wishing to use the sticker were prompted on the form to write the score total in a space provided. As a result, physicians in the intervention group received repeated prompts that reminded them to use the score approach each time they completed a clinical encounter form. The control group completed a similar form but without either the sticker or the chart prompts.

The details of the clinical score approach have been previously published.^19,25 Briefly, 4 clinical findings (fever > 38°C, absence of cough, tender anterior cervical adenopathy, tonsillar swelling or exudate) and age < 15 years are each assigned 1 point and totaled. One point is subtracted for age 45 years or more. Explicit recommendations for management are linked to score totals. If the score total is 1 or less, no throat swab or antibiotic is indicated. A throat swab is recommended for a score of 2 or 3 and an antibiotic only if the culture is positive. Either initiating treatment with an antibiotic or taking a throat swab is appropriate for a score of 4 or more.

The main outcome for the study was the prescription of unnecessary antibiotics, defined as a prescription for antibiotic medication given to a patient whose subsequent throat culture was negative for group A streptococcus. The secondary outcome was overall antibiotic use. The sample size was calculated to detect a 30% decrease in unnecessary antibiotic use (2-sided = 0.05, 1- = 0.90), assuming a 40% baseline prescription rate⁹ and a 70% negative culture rate.¹⁷¹⁹ Because groups of patients were treated by the same physician, the sample size was adjusted to take the clustered sampling design into account.³⁶ The intraclass correlation coefficient for prescribing estimated from an earlier study was 0.07.¹⁹ Assuming an average of 5 patients assessed per physician, the sample size was estimated to be 85 physicians and 425 patients in each group.

The clinical characteristics of patients in the intervention and control groups were compared with a chi-square test for categorical variables and a t-test for continuous variables. Associations between prescription rates and the practice and demographic characteristics of the physicians were assessed and adjusted for the clustered sampling with Stata Statistical Software (Release 6, Stata Corp., College Station, Tex.). While clustering improves the efficiency of sampling by requiring participation by fewer physicians, confidence intervals that do not account for the design effect are too narrow. Multiple logistic regression was used to adjust for differences in patient and physician characteristics, taking into account the patient clusters by physician in estimating the effect of the intervention.

Results

One hundred sixty-four physicians agreed to participate and were randomized. Of these, only 97 (59.1%) completed the study and provided patient data (Figure). An equal proportion of physicians in the intervention group (40.2%) and control group (41.5%, P = .87) failed to complete the study. No significant differences were identified between the sex or age of physicians who participated and those who did not participate. Of the participating physicians, 86 (84.3%) returned surveys describing their practice settings.

Patients assessed included 692 children and adults. Of these, 71 (10.3%) were excluded because of a diagnosis of bronchitis (35), sinusitis (16), otitis media (11), or pneumonia (4) or because the patient was less than 3 years old (5). The score approach did not apply to the 4 conditions of exclusion because they involve organisms other than GAS. The remaining 621 patients in the control and intervention groups were similar in demographic and clinical characteristics as well as regarding the prevalence of GAS as documented by throat culture (Table 1). However, a diagnosis of tonsillitis, strep throat, or pharyngitis was more likely to occur in the intervention group (38.6%) than in the control group (28.9%, P = .01). These diagnoses were associated with a higher rate of antibiotic prescribing (54.8%) than were situations in which physicians recorded a URTI or other diagnosis (14.2%, P < .001).

Differences were noted in the characteristics of the treating physicians in each group when considered by patient encounter. Although there were no differences in the age or sex of individual physicians in each group, the participating physicians did not contribute equal numbers of patient encounters. The average number of patients assessed per physician was 3, ranging from a low of 1 patient contributed by some physicians to a high of 8 for others. More patient encounters in the intervention group were contributed by male physicians who had been in practice longer, who worked in smaller communities, and who reported larger practice volumes (Table 1). Physicians from small communities were more likely to diagnose strep throat, tonsillitis, or pharyngitis than were those in larger communities (45.1% vs 28.1%, respectively, P < .001), as were those with higher patient volumes (46.5% vs 30.2%, P = .003).

Certain physician practice characteristics were associated with a patient’s being more likely to receive a prescription for an unnecessary antibiotic (Table 2). For example, physicians were more likely to prescribe unnecessary antibiotics if they saw more than 150 patients per week than if they saw fewer and if they had been in practice for 20 or more years than if they had practiced for a shorter time. In addition, higher overall antibiotic use was associated with higher patient volume and with practicing in a smaller community.

There were no differences between the intervention and control groups in either unnecessary antibiotic prescriptions (20.4% vs 16.1%, respectively, P = .17) or overall antibiotic use (28.1% vs 27.9%, P = .96) (Table 1). However, while the culture reports that were needed to classify prescriptions as unnecessary were available for most (600) patients (96.6%), significantly more culture reports were missing in the control group (5.4%) than in the intervention group (1.2%, P = .007). Antibiotics were prescribed in 59% of the 17 cases with missing culture reports in the control group but for none of the 4 cases with missing culture reports in the intervention group.

Because intervention patients were more likely than controls to have been treated by physicians with higher prescribing characteristics, adjustments were made for the differing physician characteristics and diagnostic practices and for the clustering of patients by physician, using multiple logistic regression (Table 3). After adjustment, the intervention was associated with a nonsignificant reduction in unnecessary antibiotic prescriptions (odds ratio [OR] = 0.76, 95% confidence interval [CI] = 0.42, 1.40) and in overall antibiotic use (OR = 0.57, 95% CI = 0.27, 1.17).

TABLE 1
COMPARISON OF PATIENTS IN CONTROL AND INTERVENTION GROUPS

TABLE 2
ASSOCIATION BETWEEN INDIVIDUAL PHYSICIAN FACTORS* AND ANTIBIOTIC PRESCRIBING, ADJUSTING FOR THE CLUSTERING OF PATIENTS BY PHYSICIAN

TABLE 3
EFFECT OF REPEATED CHART PROMPTS ON PRESCRIBING RATES, ADJUSTING FOR PHYSICIAN FACTORS AND CLUSTERING* OF PATIENTS BY PHYSICIAN (N = 453†)

Figure
FAMILY PHYSICIANS WHO WERE CONTACTED AND WHO COMPLETED THE STUDY

Discussion

The use of repeated chart prompt reminders to family physicians to use a clinical scoring approach in the management of children and adults presenting with URTI and a sore throat did not affect unnecessary antibiotic prescriptions or overall antibiotic use. Problems encountered in conducting this community-based trial may have contributed to the negative result.

Sixty-seven (41%) physicians agreed to be randomized but failed to complete the study. These losses after randomization and the differing sizes of the patient clusters per physician led to differences in the characteristics of the treating physician between the 2 groups. Characteristics associated with higher antibiotic prescribing rates were more common in the intervention group. As a result, despite the randomized design, the 2 patient groups were not initially similar in terms of their likelihood to receive a prescription for an antibiotic. To compensate for these differences, we controlled for the different physician characteristics in the analysis. However, the large number of physician dropouts also resulted in a failure to achieve the planned sample size. As a result, the study had insufficient power to detect the effect size that had been hypothesized.

We had planned the sample size to detect a 30% decrease in unnecessary antibiotic use. The adjusted analysis produced a point estimate of a 23% decrease in unnecessary antibiotic use and a 43% decrease in overall antibiotic use. These point estimates are the same whether or not the clustering is taken into effect; however, the more appropriate clustered analysis increases the estimate for the sample variance, resulting in wider confidence intervals. Examination of the lower 95% confidence interval reveals that the study lacked sufficient power to rule out as much as a 58% reduction in unnecessary antibiotic use. Therefore, while the study failed to find a statistically significant effect from the intervention, it also did not have the power to rule out a clinically important reduction in unnecessary antibiotic use.

We gave information about the clinical scoring approach to physicians in the control group. Doing so may have reduced the study’s ability to detect an effect of the intervention. We did not include a group that had been not exposed to information because we believed that mailed information was the equivalent of “standard” care in terms of changing physician behavior. Mailed information is a common method of informing physicians about new clinical information but has a limited ability to influence clinical behavior.³⁴ However, the rate of antibiotic prescribing in the control group was indeed somewhat lower than is generally reported in the literature.⁹ This finding may be compatible with volunteer bias or the Hawthorne effect. More likely, perhaps, asking the control group to complete encounter forms for multiple patients may have inadvertently reminded them about the score. As a result, the control group may have been contaminated from repeated clinical prompts.

Some problems encountered in this study have been noted by other investigators conducting community-based research in primary care.³⁷ The difficulty of retaining community-based physicians resulted in significant losses after randomization. This situation occurred even though qualifying to be randomized required physicians to mail back a reply card indicating that they wished to participate, suggesting that they were motivated to some degree.³⁷ In addition, they received a modest cash honorarium. Some physicians returned the package stating that circumstances had changed and they would be unable to participate. Many who initially agreed to participate failed to reply despite 3 mailed reminders. The level of dropouts did not become apparent until late in the study. In retrospect, it might have been advisable to phone physicians directly soon after randomization in order to detect problems early. Other physicians could then have been randomly selected from the general membership listing to replace those who had dropped out.

This study found that repeated reminders to physicians to use a clinical score in the management of their patients with a sore throat did not reduce unnecessary antibiotic use. The problems encountered in this community-based intervention trial may have contributed to the negative result. Studies of prescribing behavior may need to stratify physicians before randomization by characteristics, such as patient volume and experience, that are related to prescribing behavior. Including a group that received no information is probably necessary to allow the greatest chance of detecting an effect. Particular attention and resources need to be available to ensure the retention, and replacement if needed, of community-based family physicians participating in research studies.

Acknowledgments

This study was supported by a grant from the Medical Research Council of Canada, Grant No. MA-15088. Dr McIsaac’s work is supported by the Mt. Sinai Hospital and the Family Healthcare Research Unit of the Department of Family and Community Medicine, University of Toronto, Toronto, Canada. This study was conducted in conjunction with the National Research System of the College of Family Physicians of Canada. The cooperation of the Ontario Association of Medical Laboratories is gratefully acknowledged.