Original Research

Although many medications are available for patients with asthma, inhaled corticosteroids are generally the preferred treatment.^1-4 Multiple studies have demonstrated that inhaled corticosteroid therapy improves patient outcomes.¹ Inhaled corticosteroids have been shown to decrease costs⁵ and use of medical care resources.^6-8

More recently, leukotriene modifiers have been introduced for asthma treatment. This class of medication has bronchodilator and anti-inflammatory effects.⁹ Although multiple studies have indicated improved outcomes and decreased costs associated with leukotriene modifier therapy incertain patient populations,^10,11 its role in asthmamanagement is uncertain.⁹

Several studies have compared the clinical outcomes of these therapies^12-15 and their impact on medical care resource use and costs.^16,17 However, these studies were not powered specifically to detect significant differences in resource use or costs.

We performed a meta-analysis to (1) compare the rate of hospitalization among patients with asthma treated with inhaled corticosteroids vs those treated with leukotriene modifiers and (2) evaluate other resource use rates and costs for these patients.

Methods

The meta-analysis consisted of a literature search, the development of inclusion criteria, form development, and literature review.

Literature search

We searched the MEDLINE, EMBASE, Cochrane Collaboration Study Registry, and GlaxoSmithKline databases and consulted experts in this field. The GlaxoSmithKline database consists of studies sponsored by GlaxoSmithKline that met companywide minimum quality thresholds and were published in full or abstract form.

We also contacted the manufacturers of leukotriene modifiers available in the United States, AstraZeneca and Merck, to request published and unpublished information on studies comparing leukotriene modifiers with inhaled corticosteroids. To provide results corresponding to current treatment patterns, only studies from 1991 to 2001 were included. Published and unpublished materials were included.^18,19

Inclusion criteria

Studies were included in the meta-analysis if they met the following criteria.

Population. Patients with diagnosed asthma. Only studies that did not restrict analyses to severe asthma patients or children were included.

Study design. Prospective and retrospective comparative studies of patients receiving inhaled corticosteroid or leukotriene modifier monotherapy (no other controller therapy) in the same study. Studies were required to have defined inclusion and exclusion criteria, defined number of patients in each study arm, defined treatment protocol (ie, medications and doses used), and separate results for each medication.

Only studies presenting primary research (hence excluding review articles and metaanalyses) were included. Only studies presenting data for at least 6 months on all participants were included.

Outcomes. Hospitalization visit rates and costs, emergency department visit rates and costs, pharmacy costs, total asthma-related costs, and total medical care costs. Because resource use patterns and medical care cost information differs substantially between countries, we only included US studies.

Study process

Each identified article was evaluated by 2 independent reviewers (KMS and MG); any differences were discussed with a project leader to reach a consensus. Documents selected for inclusion were then reviewed by the 2 reviewers, and differences in data abstraction were resolved before inclusion.

Analysis

We used the Q statistic²⁰ to assess heterogeneity and, when appropriate, combined data from included studies with the use of a random effects model. Random effects methodology was used to assess the impact of inhaled corticosteroid vs leukotriene modifier therapy on the overall asthma population, not just the subpopulation of patients participating in included studies.²¹

Two sets of meta-analyses were performed with the abstracted data. First, the specified outcome measures were compared for patients taking inhaled corticosteroid vs leukotriene modifier therapy. Second, the impact (before vs after) of each treatment initiation was compared for each outcome.

Assessment of statistically significant differences between meta-analysis results was performed with the Student t test, with an Α of .05. Charges were used in the included studies as proxies for costs. These costs were inflated to 2000 values by using the medical care component of the consumer price index before inclusion.

Results

We identified 49 documents and reviewed them for inclusion in the meta-analysis; 6 (12.2%) met the inclusion criteria (Table 1). Five were retrospective cohort studies; only 1 study was identified as a prospective trial comparing inhaled corticosteroid and leukotriene modifier therapies and including results on resource use or medical care costs.^16,17,22-25 All 6 studies were performed with support from GlaxoSmithKline.

Forty-three documents were excluded due to 1 or more of the following criteria: lack of primary results (9 documents, 21%); did not contain resource use rate or cost outcomes (22 documents, 51%); did not provide at least 6 months’ worth of data on resource use or cost outcomes (5 documents, 12%); did not meet the defined inclusion and exclusion criteria (primarily studies not including both inhaled corticosteroid and leukotriene modifiers or those restricted to patient clinical subgroups; 10 documents, 34%); or did not define the number of patients included in the study (1 document, 3%).

Because few studies presented data on the specified outcomes, we were unable to assess asthma-specific costs for subcategories of resource use. Too few studies included data on hospitalization costs (either asthma-specific or overall) to include in the analysis. Therefore, meta-analysis was performed on overall (ie, all causes) emergency department, pharmacy, and total medical care costs.

TABLE 1
Characteristics of studies included in the meta-analysis

Primary analysis

The primary objective of this study was to evaluate the impact of inhaled corticosteroid and leukotriene modifier treatment on the mean annual hospitalization rate. Four of the 6 included studies contained information on hospitalization rate for each treatment. Results from the primary analysis are presented in Table 2.

Patients taking inhaled corticosteroids had a significantly lower annual rate of hospitalization than did patients taking leukotriene modifiers (2.23% vs 4.30%, respectively; P<.005). The absolute risk reduction was 2.07% (number needed to treat=48 for 1 year).

The difference in annual hospitalization visit rates for each study in the primary analysis is presented in the Figure, where negative values reflect lower hospitalization rates among patients taking inhaled corticosteroids than among those taking leukotriene modifiers. Two studies^16,23 had statistically significant differences in hospitalization rates, whereas the differences in the other 2 studies were not statistically significant (P<.05). Combining studies with the use of a random effects model (the default methodology for this analysis) or a fixed effects model produced similar results. The Q statistic indicated no significant heterogeneity (P=.43).

TABLE 2
Meta-analysis results for inhaled corticosteroid vs leukotriene modifier therapy*

FIGURE
Difference in hospitalization rates (mean, confidence interval)

Secondary outcomes

Results of secondary analyses for 5 study outcomes (annual visits to the emergency department due to asthma, total emergency department costs, total drug costs, total asthma-related costs, and overall total cost) are presented in Table 2.

Mean annual rates of visits to the emergency department and total annual drug costs were significantly higher for patients taking leukotriene modifiers than for those taking inhaled corticosteroids (P<.005 for each). Patients taking leukotriene modifiers had lower annual costs for visits to the emergency department than did those taking inhaled corticosteroids, although this difference was not statistically significant. The higher rate and lower cost of emergency department visits for patients taking leukotriene modifiers suggest that medical resources were used less at each visit as compared with those for patients taking inhaled corticosteroids.

Total asthma-related costs for patients taking inhaled corticosteroids were significantly lower than those for patients taking leukotriene modifiers (P<.05). Patients taking inhaled corticosteroids also incurred decreased annual total (allcause) medical care costs. Although this difference was qualitatively large (approximately $1900, or a decrease of over 17%), it did not reach statistical significance.

Pre- vs post-initiation of therapy

In addition to comparing the impact of the 2 therapies on resource use and costs, we were interested in the impact of initiating each therapy on the study outcomes. We therefore assessed resource use rates and costs before and after treatment initiation. These patients may have been receiving asthma therapies (or no asthma therapy) other than inhaled corticosteroids or leukotriene modifiers. The number of studies presenting data on costs was too small to allow comparison.

Results of this within-group analysis are presented in Table 3. For patients taking inhaled corticosteroids, hospitalization rates and emergency department visit rates decreased significantly after treatment initiation compared with the preinitiation values (P<.005 and P<.05, respectively). The decreases for patients taking leukotriene modifiers upon treatment initiation were smaller and not statistically significant.

Both groups showed similar small decreases in annual emergency department costs, neither of which was significant. The increases in annual total drug costs and annual total medical care costs after treatment initiation were significant for both groups of patients (all at P<.005). However, both increases were greater for patients taking leukotriene modifiers; the increase in drug costs was statistically significant (P<.001).

TABLE 3
Resource utilization and costs before and after therapy initiation for inhaled corticosteroids vs leukotriene modifiers*

Discussion

In this study, we used meta-analysis to combine data across studies and determine more robust estimates of the impact of inhaled corticosteroid vs leukotriene modifier therapy on medical resource use rates and costs. The primary analysis indicated that annual hospitalization rates among patients taking inhaled corticosteroids are significantly lower that those taking leukotriene modifiers. Other resource use rates and costs evaluated in this study also generally showed decreased values for patients taking inhaled corticosteroids.

Although meta-analysis generally has been used for clinical outcome measures, it is a highly appropriate method for resource use and cost outcomes. In general, studies of the impact of a particular treatment are powered to assess safety and efficacy or effectiveness; there is often insufficient power to detect differences in resource use or costs in any one study. Due to substantial variation in treatment patterns, the variance associated with resource use rates (and associated costs) may be substantially higher than that for clinical measures; such a wide variance adds to the difficulties in assessing differences for nonclinical outcomes.

Limitations

This study has a number of limitations. Only a few studies met inclusion criteria for the meta-analysis; the analysis should be replicated as additional studies become available. Data were abstracted from the included studies without modification (except for inflating costs to 2000 values when necessary). As in all meta-analyses, any problems present in the original data are present in the combined data; limitations of the original data are not addressed by this method.

Among the studies evaluated for the metaanalysis were a number published only as abstracts. Inclusion of unpublished literature in meta-analyses is controversial; however, several sources^18,19 now recommend inclusion of published and unpublished studies. Egger and Smith²⁶ found that studies with significant results are more likely to be published than are studies with nonsignificant results, leading to publication bias. Studies with significant results also may be more likely to be published in indexed journals, leading to “database bias.” As such, inclusion of unpublished studies is important to produce unbiased results.

Five of the 6 studies that met the inclusion criteria were observational, retrospective cohort analyses. Whereas many meta-analyses focus solely on prospective, randomized clinical trials, several have included retrospective data.^27,28 Retrospective analyses and observational data have a number of limitations, in particular the lack of randomization that can lead to differences in characteristics of specified treatment groups. Further, as discussed by Egger et al,²⁹ metaanalyses based on observational studies may involve bias and confounding.

However, observational data and retrospective analyses also have the advantage of reflecting “real world” treatment patterns and broader patient groups that increase the generalizability of the data, whereas clinical trials may include protocol-driven utilization and selected patient groups. Clinical trials also may occur in specialized health care settings, whereas observational (cohort) data may be more applicable to clinical practice. Due to these factors, meta-analysis of observational data has become common.²⁹

The pre-initiation vs post-initiation analysis indicated that values for each treatment group provide information on the similarities between treatment groups before initiation of controller therapy. Even though the treatment groups were not randomized to each therapy and we have no means to ensure compatibility between groups, having similar rates of resource use between groups provides some evidence regarding similarity. Nonetheless, given the limitations of the retrospective data and meta-analysis in general, it will be important to validate the results of this meta-analysis in the future with naturalistic prospective studies.

Despite these limitations, this study provides important information on the impact of asthma therapies on resource use and costs. Specifically, the resource use and cost outcomes assessed in this study were lower for inhaled corticosteroid patients than for leukotriene modifier patients. This study also illustrates the usefulness of metaanalysis in evaluating resource use and costs. By selecting and combining outcomes across studies in a standardized, rigorous, and transparent manner, the effects of different therapies can be evaluated with greater precision.

Acknowledgments

We thank John O’Donnell and Layne Gothard for their assistance with this manuscript.

Corresponding author
Michael T. Halpern, MD, PhD, Principal Scientist, Exponent, Inc., 1800 Diagonal Road, Alexandria, VA 22314. E-mail: [email protected]

Although many medications are available for patients with asthma, inhaled corticosteroids are generally the preferred treatment.^1-4 Multiple studies have demonstrated that inhaled corticosteroid therapy improves patient outcomes.¹ Inhaled corticosteroids have been shown to decrease costs⁵ and use of medical care resources.^6-8

More recently, leukotriene modifiers have been introduced for asthma treatment. This class of medication has bronchodilator and anti-inflammatory effects.⁹ Although multiple studies have indicated improved outcomes and decreased costs associated with leukotriene modifier therapy incertain patient populations,^10,11 its role in asthmamanagement is uncertain.⁹

Several studies have compared the clinical outcomes of these therapies^12-15 and their impact on medical care resource use and costs.^16,17 However, these studies were not powered specifically to detect significant differences in resource use or costs.

We performed a meta-analysis to (1) compare the rate of hospitalization among patients with asthma treated with inhaled corticosteroids vs those treated with leukotriene modifiers and (2) evaluate other resource use rates and costs for these patients.

Methods

The meta-analysis consisted of a literature search, the development of inclusion criteria, form development, and literature review.

Literature search

We searched the MEDLINE, EMBASE, Cochrane Collaboration Study Registry, and GlaxoSmithKline databases and consulted experts in this field. The GlaxoSmithKline database consists of studies sponsored by GlaxoSmithKline that met companywide minimum quality thresholds and were published in full or abstract form.

We also contacted the manufacturers of leukotriene modifiers available in the United States, AstraZeneca and Merck, to request published and unpublished information on studies comparing leukotriene modifiers with inhaled corticosteroids. To provide results corresponding to current treatment patterns, only studies from 1991 to 2001 were included. Published and unpublished materials were included.^18,19

Inclusion criteria

Studies were included in the meta-analysis if they met the following criteria.

Population. Patients with diagnosed asthma. Only studies that did not restrict analyses to severe asthma patients or children were included.

Study design. Prospective and retrospective comparative studies of patients receiving inhaled corticosteroid or leukotriene modifier monotherapy (no other controller therapy) in the same study. Studies were required to have defined inclusion and exclusion criteria, defined number of patients in each study arm, defined treatment protocol (ie, medications and doses used), and separate results for each medication.

Only studies presenting primary research (hence excluding review articles and metaanalyses) were included. Only studies presenting data for at least 6 months on all participants were included.

Outcomes. Hospitalization visit rates and costs, emergency department visit rates and costs, pharmacy costs, total asthma-related costs, and total medical care costs. Because resource use patterns and medical care cost information differs substantially between countries, we only included US studies.

Study process

Each identified article was evaluated by 2 independent reviewers (KMS and MG); any differences were discussed with a project leader to reach a consensus. Documents selected for inclusion were then reviewed by the 2 reviewers, and differences in data abstraction were resolved before inclusion.

Analysis

We used the Q statistic²⁰ to assess heterogeneity and, when appropriate, combined data from included studies with the use of a random effects model. Random effects methodology was used to assess the impact of inhaled corticosteroid vs leukotriene modifier therapy on the overall asthma population, not just the subpopulation of patients participating in included studies.²¹

Two sets of meta-analyses were performed with the abstracted data. First, the specified outcome measures were compared for patients taking inhaled corticosteroid vs leukotriene modifier therapy. Second, the impact (before vs after) of each treatment initiation was compared for each outcome.

Assessment of statistically significant differences between meta-analysis results was performed with the Student t test, with an Α of .05. Charges were used in the included studies as proxies for costs. These costs were inflated to 2000 values by using the medical care component of the consumer price index before inclusion.

Results

We identified 49 documents and reviewed them for inclusion in the meta-analysis; 6 (12.2%) met the inclusion criteria (Table 1). Five were retrospective cohort studies; only 1 study was identified as a prospective trial comparing inhaled corticosteroid and leukotriene modifier therapies and including results on resource use or medical care costs.^16,17,22-25 All 6 studies were performed with support from GlaxoSmithKline.

Forty-three documents were excluded due to 1 or more of the following criteria: lack of primary results (9 documents, 21%); did not contain resource use rate or cost outcomes (22 documents, 51%); did not provide at least 6 months’ worth of data on resource use or cost outcomes (5 documents, 12%); did not meet the defined inclusion and exclusion criteria (primarily studies not including both inhaled corticosteroid and leukotriene modifiers or those restricted to patient clinical subgroups; 10 documents, 34%); or did not define the number of patients included in the study (1 document, 3%).

Because few studies presented data on the specified outcomes, we were unable to assess asthma-specific costs for subcategories of resource use. Too few studies included data on hospitalization costs (either asthma-specific or overall) to include in the analysis. Therefore, meta-analysis was performed on overall (ie, all causes) emergency department, pharmacy, and total medical care costs.

TABLE 1
Characteristics of studies included in the meta-analysis

Primary analysis

The primary objective of this study was to evaluate the impact of inhaled corticosteroid and leukotriene modifier treatment on the mean annual hospitalization rate. Four of the 6 included studies contained information on hospitalization rate for each treatment. Results from the primary analysis are presented in Table 2.

Patients taking inhaled corticosteroids had a significantly lower annual rate of hospitalization than did patients taking leukotriene modifiers (2.23% vs 4.30%, respectively; P<.005). The absolute risk reduction was 2.07% (number needed to treat=48 for 1 year).

The difference in annual hospitalization visit rates for each study in the primary analysis is presented in the Figure, where negative values reflect lower hospitalization rates among patients taking inhaled corticosteroids than among those taking leukotriene modifiers. Two studies^16,23 had statistically significant differences in hospitalization rates, whereas the differences in the other 2 studies were not statistically significant (P<.05). Combining studies with the use of a random effects model (the default methodology for this analysis) or a fixed effects model produced similar results. The Q statistic indicated no significant heterogeneity (P=.43).

TABLE 2
Meta-analysis results for inhaled corticosteroid vs leukotriene modifier therapy*

FIGURE
Difference in hospitalization rates (mean, confidence interval)

Secondary outcomes

Results of secondary analyses for 5 study outcomes (annual visits to the emergency department due to asthma, total emergency department costs, total drug costs, total asthma-related costs, and overall total cost) are presented in Table 2.

Mean annual rates of visits to the emergency department and total annual drug costs were significantly higher for patients taking leukotriene modifiers than for those taking inhaled corticosteroids (P<.005 for each). Patients taking leukotriene modifiers had lower annual costs for visits to the emergency department than did those taking inhaled corticosteroids, although this difference was not statistically significant. The higher rate and lower cost of emergency department visits for patients taking leukotriene modifiers suggest that medical resources were used less at each visit as compared with those for patients taking inhaled corticosteroids.

Total asthma-related costs for patients taking inhaled corticosteroids were significantly lower than those for patients taking leukotriene modifiers (P<.05). Patients taking inhaled corticosteroids also incurred decreased annual total (allcause) medical care costs. Although this difference was qualitatively large (approximately $1900, or a decrease of over 17%), it did not reach statistical significance.

Pre- vs post-initiation of therapy

In addition to comparing the impact of the 2 therapies on resource use and costs, we were interested in the impact of initiating each therapy on the study outcomes. We therefore assessed resource use rates and costs before and after treatment initiation. These patients may have been receiving asthma therapies (or no asthma therapy) other than inhaled corticosteroids or leukotriene modifiers. The number of studies presenting data on costs was too small to allow comparison.

Results of this within-group analysis are presented in Table 3. For patients taking inhaled corticosteroids, hospitalization rates and emergency department visit rates decreased significantly after treatment initiation compared with the preinitiation values (P<.005 and P<.05, respectively). The decreases for patients taking leukotriene modifiers upon treatment initiation were smaller and not statistically significant.

Both groups showed similar small decreases in annual emergency department costs, neither of which was significant. The increases in annual total drug costs and annual total medical care costs after treatment initiation were significant for both groups of patients (all at P<.005). However, both increases were greater for patients taking leukotriene modifiers; the increase in drug costs was statistically significant (P<.001).

TABLE 3
Resource utilization and costs before and after therapy initiation for inhaled corticosteroids vs leukotriene modifiers*

Discussion

In this study, we used meta-analysis to combine data across studies and determine more robust estimates of the impact of inhaled corticosteroid vs leukotriene modifier therapy on medical resource use rates and costs. The primary analysis indicated that annual hospitalization rates among patients taking inhaled corticosteroids are significantly lower that those taking leukotriene modifiers. Other resource use rates and costs evaluated in this study also generally showed decreased values for patients taking inhaled corticosteroids.

Although meta-analysis generally has been used for clinical outcome measures, it is a highly appropriate method for resource use and cost outcomes. In general, studies of the impact of a particular treatment are powered to assess safety and efficacy or effectiveness; there is often insufficient power to detect differences in resource use or costs in any one study. Due to substantial variation in treatment patterns, the variance associated with resource use rates (and associated costs) may be substantially higher than that for clinical measures; such a wide variance adds to the difficulties in assessing differences for nonclinical outcomes.

Limitations

This study has a number of limitations. Only a few studies met inclusion criteria for the meta-analysis; the analysis should be replicated as additional studies become available. Data were abstracted from the included studies without modification (except for inflating costs to 2000 values when necessary). As in all meta-analyses, any problems present in the original data are present in the combined data; limitations of the original data are not addressed by this method.

Among the studies evaluated for the metaanalysis were a number published only as abstracts. Inclusion of unpublished literature in meta-analyses is controversial; however, several sources^18,19 now recommend inclusion of published and unpublished studies. Egger and Smith²⁶ found that studies with significant results are more likely to be published than are studies with nonsignificant results, leading to publication bias. Studies with significant results also may be more likely to be published in indexed journals, leading to “database bias.” As such, inclusion of unpublished studies is important to produce unbiased results.

Five of the 6 studies that met the inclusion criteria were observational, retrospective cohort analyses. Whereas many meta-analyses focus solely on prospective, randomized clinical trials, several have included retrospective data.^27,28 Retrospective analyses and observational data have a number of limitations, in particular the lack of randomization that can lead to differences in characteristics of specified treatment groups. Further, as discussed by Egger et al,²⁹ metaanalyses based on observational studies may involve bias and confounding.

However, observational data and retrospective analyses also have the advantage of reflecting “real world” treatment patterns and broader patient groups that increase the generalizability of the data, whereas clinical trials may include protocol-driven utilization and selected patient groups. Clinical trials also may occur in specialized health care settings, whereas observational (cohort) data may be more applicable to clinical practice. Due to these factors, meta-analysis of observational data has become common.²⁹

The pre-initiation vs post-initiation analysis indicated that values for each treatment group provide information on the similarities between treatment groups before initiation of controller therapy. Even though the treatment groups were not randomized to each therapy and we have no means to ensure compatibility between groups, having similar rates of resource use between groups provides some evidence regarding similarity. Nonetheless, given the limitations of the retrospective data and meta-analysis in general, it will be important to validate the results of this meta-analysis in the future with naturalistic prospective studies.

Despite these limitations, this study provides important information on the impact of asthma therapies on resource use and costs. Specifically, the resource use and cost outcomes assessed in this study were lower for inhaled corticosteroid patients than for leukotriene modifier patients. This study also illustrates the usefulness of metaanalysis in evaluating resource use and costs. By selecting and combining outcomes across studies in a standardized, rigorous, and transparent manner, the effects of different therapies can be evaluated with greater precision.

Acknowledgments

We thank John O’Donnell and Layne Gothard for their assistance with this manuscript.

Corresponding author
Michael T. Halpern, MD, PhD, Principal Scientist, Exponent, Inc., 1800 Diagonal Road, Alexandria, VA 22314. E-mail: [email protected]

Although many medications are available for patients with asthma, inhaled corticosteroids are generally the preferred treatment.^1-4 Multiple studies have demonstrated that inhaled corticosteroid therapy improves patient outcomes.¹ Inhaled corticosteroids have been shown to decrease costs⁵ and use of medical care resources.^6-8

More recently, leukotriene modifiers have been introduced for asthma treatment. This class of medication has bronchodilator and anti-inflammatory effects.⁹ Although multiple studies have indicated improved outcomes and decreased costs associated with leukotriene modifier therapy incertain patient populations,^10,11 its role in asthmamanagement is uncertain.⁹

Several studies have compared the clinical outcomes of these therapies^12-15 and their impact on medical care resource use and costs.^16,17 However, these studies were not powered specifically to detect significant differences in resource use or costs.

We performed a meta-analysis to (1) compare the rate of hospitalization among patients with asthma treated with inhaled corticosteroids vs those treated with leukotriene modifiers and (2) evaluate other resource use rates and costs for these patients.

Methods

The meta-analysis consisted of a literature search, the development of inclusion criteria, form development, and literature review.

Literature search

We searched the MEDLINE, EMBASE, Cochrane Collaboration Study Registry, and GlaxoSmithKline databases and consulted experts in this field. The GlaxoSmithKline database consists of studies sponsored by GlaxoSmithKline that met companywide minimum quality thresholds and were published in full or abstract form.

We also contacted the manufacturers of leukotriene modifiers available in the United States, AstraZeneca and Merck, to request published and unpublished information on studies comparing leukotriene modifiers with inhaled corticosteroids. To provide results corresponding to current treatment patterns, only studies from 1991 to 2001 were included. Published and unpublished materials were included.^18,19

Inclusion criteria

Studies were included in the meta-analysis if they met the following criteria.

Population. Patients with diagnosed asthma. Only studies that did not restrict analyses to severe asthma patients or children were included.

Study design. Prospective and retrospective comparative studies of patients receiving inhaled corticosteroid or leukotriene modifier monotherapy (no other controller therapy) in the same study. Studies were required to have defined inclusion and exclusion criteria, defined number of patients in each study arm, defined treatment protocol (ie, medications and doses used), and separate results for each medication.

Only studies presenting primary research (hence excluding review articles and metaanalyses) were included. Only studies presenting data for at least 6 months on all participants were included.

Outcomes. Hospitalization visit rates and costs, emergency department visit rates and costs, pharmacy costs, total asthma-related costs, and total medical care costs. Because resource use patterns and medical care cost information differs substantially between countries, we only included US studies.

Study process

Each identified article was evaluated by 2 independent reviewers (KMS and MG); any differences were discussed with a project leader to reach a consensus. Documents selected for inclusion were then reviewed by the 2 reviewers, and differences in data abstraction were resolved before inclusion.

Analysis

We used the Q statistic²⁰ to assess heterogeneity and, when appropriate, combined data from included studies with the use of a random effects model. Random effects methodology was used to assess the impact of inhaled corticosteroid vs leukotriene modifier therapy on the overall asthma population, not just the subpopulation of patients participating in included studies.²¹

Two sets of meta-analyses were performed with the abstracted data. First, the specified outcome measures were compared for patients taking inhaled corticosteroid vs leukotriene modifier therapy. Second, the impact (before vs after) of each treatment initiation was compared for each outcome.

Assessment of statistically significant differences between meta-analysis results was performed with the Student t test, with an Α of .05. Charges were used in the included studies as proxies for costs. These costs were inflated to 2000 values by using the medical care component of the consumer price index before inclusion.

Results

We identified 49 documents and reviewed them for inclusion in the meta-analysis; 6 (12.2%) met the inclusion criteria (Table 1). Five were retrospective cohort studies; only 1 study was identified as a prospective trial comparing inhaled corticosteroid and leukotriene modifier therapies and including results on resource use or medical care costs.^16,17,22-25 All 6 studies were performed with support from GlaxoSmithKline.

Forty-three documents were excluded due to 1 or more of the following criteria: lack of primary results (9 documents, 21%); did not contain resource use rate or cost outcomes (22 documents, 51%); did not provide at least 6 months’ worth of data on resource use or cost outcomes (5 documents, 12%); did not meet the defined inclusion and exclusion criteria (primarily studies not including both inhaled corticosteroid and leukotriene modifiers or those restricted to patient clinical subgroups; 10 documents, 34%); or did not define the number of patients included in the study (1 document, 3%).

Because few studies presented data on the specified outcomes, we were unable to assess asthma-specific costs for subcategories of resource use. Too few studies included data on hospitalization costs (either asthma-specific or overall) to include in the analysis. Therefore, meta-analysis was performed on overall (ie, all causes) emergency department, pharmacy, and total medical care costs.

TABLE 1
Characteristics of studies included in the meta-analysis

Primary analysis

The primary objective of this study was to evaluate the impact of inhaled corticosteroid and leukotriene modifier treatment on the mean annual hospitalization rate. Four of the 6 included studies contained information on hospitalization rate for each treatment. Results from the primary analysis are presented in Table 2.

Patients taking inhaled corticosteroids had a significantly lower annual rate of hospitalization than did patients taking leukotriene modifiers (2.23% vs 4.30%, respectively; P<.005). The absolute risk reduction was 2.07% (number needed to treat=48 for 1 year).

The difference in annual hospitalization visit rates for each study in the primary analysis is presented in the Figure, where negative values reflect lower hospitalization rates among patients taking inhaled corticosteroids than among those taking leukotriene modifiers. Two studies^16,23 had statistically significant differences in hospitalization rates, whereas the differences in the other 2 studies were not statistically significant (P<.05). Combining studies with the use of a random effects model (the default methodology for this analysis) or a fixed effects model produced similar results. The Q statistic indicated no significant heterogeneity (P=.43).

TABLE 2
Meta-analysis results for inhaled corticosteroid vs leukotriene modifier therapy*

FIGURE
Difference in hospitalization rates (mean, confidence interval)

Secondary outcomes

Results of secondary analyses for 5 study outcomes (annual visits to the emergency department due to asthma, total emergency department costs, total drug costs, total asthma-related costs, and overall total cost) are presented in Table 2.

Mean annual rates of visits to the emergency department and total annual drug costs were significantly higher for patients taking leukotriene modifiers than for those taking inhaled corticosteroids (P<.005 for each). Patients taking leukotriene modifiers had lower annual costs for visits to the emergency department than did those taking inhaled corticosteroids, although this difference was not statistically significant. The higher rate and lower cost of emergency department visits for patients taking leukotriene modifiers suggest that medical resources were used less at each visit as compared with those for patients taking inhaled corticosteroids.

Total asthma-related costs for patients taking inhaled corticosteroids were significantly lower than those for patients taking leukotriene modifiers (P<.05). Patients taking inhaled corticosteroids also incurred decreased annual total (allcause) medical care costs. Although this difference was qualitatively large (approximately $1900, or a decrease of over 17%), it did not reach statistical significance.

Pre- vs post-initiation of therapy

In addition to comparing the impact of the 2 therapies on resource use and costs, we were interested in the impact of initiating each therapy on the study outcomes. We therefore assessed resource use rates and costs before and after treatment initiation. These patients may have been receiving asthma therapies (or no asthma therapy) other than inhaled corticosteroids or leukotriene modifiers. The number of studies presenting data on costs was too small to allow comparison.

Results of this within-group analysis are presented in Table 3. For patients taking inhaled corticosteroids, hospitalization rates and emergency department visit rates decreased significantly after treatment initiation compared with the preinitiation values (P<.005 and P<.05, respectively). The decreases for patients taking leukotriene modifiers upon treatment initiation were smaller and not statistically significant.

Both groups showed similar small decreases in annual emergency department costs, neither of which was significant. The increases in annual total drug costs and annual total medical care costs after treatment initiation were significant for both groups of patients (all at P<.005). However, both increases were greater for patients taking leukotriene modifiers; the increase in drug costs was statistically significant (P<.001).

TABLE 3
Resource utilization and costs before and after therapy initiation for inhaled corticosteroids vs leukotriene modifiers*

Discussion

In this study, we used meta-analysis to combine data across studies and determine more robust estimates of the impact of inhaled corticosteroid vs leukotriene modifier therapy on medical resource use rates and costs. The primary analysis indicated that annual hospitalization rates among patients taking inhaled corticosteroids are significantly lower that those taking leukotriene modifiers. Other resource use rates and costs evaluated in this study also generally showed decreased values for patients taking inhaled corticosteroids.

Although meta-analysis generally has been used for clinical outcome measures, it is a highly appropriate method for resource use and cost outcomes. In general, studies of the impact of a particular treatment are powered to assess safety and efficacy or effectiveness; there is often insufficient power to detect differences in resource use or costs in any one study. Due to substantial variation in treatment patterns, the variance associated with resource use rates (and associated costs) may be substantially higher than that for clinical measures; such a wide variance adds to the difficulties in assessing differences for nonclinical outcomes.

Limitations

This study has a number of limitations. Only a few studies met inclusion criteria for the meta-analysis; the analysis should be replicated as additional studies become available. Data were abstracted from the included studies without modification (except for inflating costs to 2000 values when necessary). As in all meta-analyses, any problems present in the original data are present in the combined data; limitations of the original data are not addressed by this method.

Among the studies evaluated for the metaanalysis were a number published only as abstracts. Inclusion of unpublished literature in meta-analyses is controversial; however, several sources^18,19 now recommend inclusion of published and unpublished studies. Egger and Smith²⁶ found that studies with significant results are more likely to be published than are studies with nonsignificant results, leading to publication bias. Studies with significant results also may be more likely to be published in indexed journals, leading to “database bias.” As such, inclusion of unpublished studies is important to produce unbiased results.

Five of the 6 studies that met the inclusion criteria were observational, retrospective cohort analyses. Whereas many meta-analyses focus solely on prospective, randomized clinical trials, several have included retrospective data.^27,28 Retrospective analyses and observational data have a number of limitations, in particular the lack of randomization that can lead to differences in characteristics of specified treatment groups. Further, as discussed by Egger et al,²⁹ metaanalyses based on observational studies may involve bias and confounding.

However, observational data and retrospective analyses also have the advantage of reflecting “real world” treatment patterns and broader patient groups that increase the generalizability of the data, whereas clinical trials may include protocol-driven utilization and selected patient groups. Clinical trials also may occur in specialized health care settings, whereas observational (cohort) data may be more applicable to clinical practice. Due to these factors, meta-analysis of observational data has become common.²⁹

The pre-initiation vs post-initiation analysis indicated that values for each treatment group provide information on the similarities between treatment groups before initiation of controller therapy. Even though the treatment groups were not randomized to each therapy and we have no means to ensure compatibility between groups, having similar rates of resource use between groups provides some evidence regarding similarity. Nonetheless, given the limitations of the retrospective data and meta-analysis in general, it will be important to validate the results of this meta-analysis in the future with naturalistic prospective studies.

Despite these limitations, this study provides important information on the impact of asthma therapies on resource use and costs. Specifically, the resource use and cost outcomes assessed in this study were lower for inhaled corticosteroid patients than for leukotriene modifier patients. This study also illustrates the usefulness of metaanalysis in evaluating resource use and costs. By selecting and combining outcomes across studies in a standardized, rigorous, and transparent manner, the effects of different therapies can be evaluated with greater precision.

Acknowledgments

We thank John O’Donnell and Layne Gothard for their assistance with this manuscript.

Corresponding author
Michael T. Halpern, MD, PhD, Principal Scientist, Exponent, Inc., 1800 Diagonal Road, Alexandria, VA 22314. E-mail: [email protected]

Telemedicine enables doctors in rural areas or areas with poor medical service to consult with experts at distant locations. Telecolposcopy and cervicography both enable remote diagnoses of the cervix. The 2 methods differ in equipment, operations, image format, timeliness of consultation, and probably cost. However, these diagnostic approaches have not been compared previously. The purpose of this study was to compare the accuracy of telecolposcopy and cervicography with on-site colposcopy in the remote evaluation of women with potential cervical neoplasia.

Telecolposcopy and cervicography

Telecolposcopy involves a distant expert colposcopist’s evaluation of women with potential lower genital tract neoplasia.¹ Existing telemedicine network and computer systems provide an audiovisual interface between local colposcopists and expert colposcopists at other locations.² For health systems already using computer or video networks, telecolposcopic consultation can be implemented with only small additional charges per examination.² Telecolposcopy services may improve health care access for women in medically underserved areas.¹

Cervicography is distant evaluation of 2 photographs taken of the cervix following 5% acetic acid application.³ A special 35-mm camera is used to take these images. The end product, developed at a central processing center, resembles a low-magnification colposcopic photograph. Certified evaluators interpret these images, classifying them as negative, atypical, or positive. Cervicography is used primarily as an adjunct test to the Papanicolaou (Pap) smear.⁴ It has also been evaluated as an intermediate triage test for evaluating women with mildly abnormal Pap smear results.^5-8

Methods

Women aged 18 years or older who came to 1 of 2 rural clinic sites for a colposcopic examination were enrolled in the trial after signing an institutional review board–approved informed consent document. We included women with a recent abnormal Pap smear report or a lower genital tract finding that required further evaluation by colposcopy. The exclusion criteria were pregnancy, severe cervicitis, heavy menses, refusal to participate, or technical problems with the telecolposcopy or cervicography equipment.

Both clinics were part of the Medical College of Georgia Telemedicine Network. This system uses sophisticated telecommunications equipment to provide distant consultation services to clinicians practicing in rural areas of the state.¹ Small change-coupled device cameras were attached to the colposcopes at the 2 clinics.

For network telecolposcopy, images were transmitted using the network’s existing hardware and high-speed telecommunication lines. For computer telecolposcopy, personal computers (DIMS, DenVu, Tucson, Ariz) were also used to capture and transmit images to a computer at the Telemedicine Center. These digitized images were transmitted by modem via telephone lines.² Cerviscopes (35-mm cameras) supplied by the manufacturer (NTL Worldwide, Fenton, Mo) were used to acquire cervigrams (photographs).

Pertinent clinicians received appropriate training to take cervigrams. Certified evaluators interpreted the images according to company protocol and returned a standardized report to the investigators at a later date.

Study design

The study design has been described in detail previously.^1,2 Briefly, subjects were initially examined by 1 of 3 on-site, university-based expert colposcopists, who took 2 cervigrams of each patient, and then conducted a colposcopic examination independently.

A local clinician then completed another colposcopic examination, including histologic sampling, if indicated. This examination was observed simultaneously by another expert at a telemedicine center. Prior to obtaining histologic samples or using dilute Lugol’s iodine solution, the local clinician captured 2 cervical images (low and high magnification) using the computer telemedicine system. These images were then transmitted to the expert at the telemedicine center for independent interpretation.

A third expert colposcopist interpreted the video and computer images at a later time. However, these third interpretations were not considered in this report. Colposcopists were blinded to each other’s clinical diagnoses. However, all colposcopists were informed of the subject’s referral cervical cytology results and other pertinent history.

Data analysis

Each subject had 2 observations using each of the 3 colposcopy methods (on-site, network, and computer-based), and a single observation using cervicography. On-site colposcopy, consisting of the observations of the on-site expert and local colposcopist, was considered for reference purposes. Agreement with histologic results was calculated for each method, across all histologic diagnoses together and separately by diagnosis.

Sensitivity and specificity estimates were calculated using 2 definitions of disease: (1) normal versus any other histologic diagnosis, and (2) normal or cervical intraepithelial neoplasia 1 (CIN 1) versus any more severe diagnosis. The primary analysis model was complete block analysis of variance, with subjects included as blocks in the analysis to account for the multiple observations on the same subjects. Nonparametric comparisons of proportions of agreement with histology, sensitivity, and specificity among the methods were made using permutation tests. Post-hoc comparisons were made using a Tukey test; 95% confidence intervals (CIs) were calculated for all point estimates. Adjustment for dependence among multiple observations per subject was made by basing these tests and CIs on least-squares means.

The available sample sizes for all analyses were adequate to ensure approximate normality of the estimated means. Power to detect, at Α=.05, a difference in agreement of 15% between cervigram and the other evaluation methods, was estimated using Monte Carlo simulations. Data were simulated using the observed levels of agreement for on-site, network, and computer telecolposcopy, and specifying a difference of 15% between cervicography agreement and the maximum of the other methods’ agreement. Power estimates were based on analysis of 1000 simulations. SAS release 8.02 was used for all calculations (SAS, Inc, Cary, NC).

Results

A total of 264 subjects were enrolled in the trial, but the total number of subjects considered differed depending on the various analyses of interest. The demographic data of this study cohort have been published previously.¹

Briefly, the subjects’ mean age was 31.7 years and mean parity was 2.1. Subjects presented with a wide range of prior cervical cytology results: 20.4% normal, 29.2% atypical squamous cells of undetermined significance, 40.4% low-grade squamous intraepithelial lesion, 7.3% high-grade squamous intraepithelial lesion, and 2.7% atypical glandular cells of undetermined significance. Histology results included all levels of CIN (52.9% CIN 1 and 13.4% CIN 2 or 3), and endocervical histologic sampling results were reported as both positive and negative for neoplasia.

The agreement between telecolposcopic/cervicography impressions and histology were estimated (Table 1). Data for on-site colposcopy was also considered for reference purposes.

When all histologic diagnoses were considered, there was no statistically significant difference in the rates of agreement for colposcopy, the 2 types of telecolposcopy, and cervicography. This was also true if only cases of CIN 1 were examined.

However, a statistically significant difference was noted between agreement rates for computer-based telecolposcopy (63.95%) and on-site colposcopy (47.7%, P=.03, Tukey test) for normal histology. A statistically significant difference was also found between agreement rates for on-site colposcopy (50.0%) and cervicography (19.1%, P=.04, Tukey test) for women with biopsy-proven CIN 2 or 3. If all histologic diagnoses were considered, the study provided 85% power to detect a difference in agreement of 15% among the evaluation methods.

We also estimated the sensitivity and specificity of the four diagnostic methods to detect cervical neoplasia (Table 2). A statistically significant difference was found in observed sensitivity between on-site colposcopy (47.7%) and cervicography (18.2%, P=.04, Tukey test) when a positive threshold of at least CIN 2 was considered. The difference was not significant, however, if the lower positive test threshold of at least CIN 1 was considered.

A statistically significant difference in specificity was noted between computer-based telecolposcopy (64.0%) and on-site colposcopy (47.7%, P=.03, Tukey test) at a positive threshold of at least CIN 1. The study provided a power of 71% and 60% to detect differences of 15% in sensitivity and specificity, respectively, using the CIN 1 threshold. Using CIN 2 as the positive threshold, the power to detect this 15% difference was 24% and 81% for sensitivity and specificity, respectively.

TABLE 1
Colposcopic, telecolposcopic, and cervicographic agreement with histology

TABLE 2
Sensitivity and specificity of tests to detect cervical neoplasia

Discussion

Until recently, cervicography had been the only type of remote diagnostic system available for the evaluation of women with potential lower genital tract neoplasia. With the advent of telemedicine during the past decade, expert-level health care has now become more readily available to patients previously isolated from this important resource.

The future of telecolposcopy

Because of its nature, telecolposcopy may also be well suited to assist in the evaluation and management of women with lower genital tract neoplasia. Computer-based telecolposcopy has the potential to support clinical sites located wherever standard telephone service exists. Cellular telephone systems now broaden access to nearglobal availability. Soon, assuming sufficient funding is obtained, the provision of expertenhanced colposcopy may become a reality for all women. However, universal availability may be irrelevant if computer-based telecolposcopy performs at a substandard level.

Telecolposcopy vs cervicography

We have demonstrated that telecolposcopy was at least as effective as cervicography for detecting cervical cancer precursors. Although the difference was not statistically significant, both network and computer-based telecolposcopy systems detected a higher percentage of women with CIN 2 or 3 than cervicography.

Our results also included on-site colposcopy. As anticipated, on-site colposcopy had the greatest sensitivity for disease detection at either positive test thresholds (at least CIN 1 and CIN 2). Ability to manipulate the cervix, stereoscopic viewing, longitudinal observation after 5% acetic acid application, and better resolution of the cervical epithelium and vascularity all favor on-site colposcopic diagnoses. Of the 2 telecolposcopy systems, network telecolposcopy had a slightly, but not significantly, greater sensitivity for detecting cervical cancer precursors compared with computer-based telecolposcopy.

Expert colposcopists’ accuracy with interpretation of network (real-time) cervical images was similar to that for on-site colposcopy, as might be expected. Network telecolposcopy might be equated with remote video colposcopy. Previously we have shown that traditional optical colposcopy is equivalent to video colposcopy with respect to colposcopic/histologic agreement.⁹

Comparison of telecolposcopy systems

The computer-based telecolposcopy system used in our study was, in all fairness, more similar to cervicography. Each method involves evaluation of 2 static images. Computer-based telecolposcopy provides 2 digitized images, but of a low- and high-power magnification view of the cervix. In comparison, cervicography produces dual low-power magnification celluloid images (2 x 2 slides) of the cervix. The provision of a high-power cervical image may explain the better sensitivity of computer-based telecolposcopy. This one feature may be more valuable than the better image resolution obtained from cervicography. However, computer-based resolution appears to be sufficient to render diagnoses at a level equivalent to or better than cervicography.

These 2 “static” systems differ in other aspects as well. First, computer-based systems are nonproprietary. Several systems are commercially available and other colposcopists have devised their own unique systems using modifications of off-the-shelf technology. Although not available at the initiation of our trial, computerbased systems now have the capability of capturing short video streams. These video segments should help improve the diagnostic ability of consulting colposcopists as demonstrated by our study.

Second, computer-based telecolposcopy can provide instantaneous consultation as opposed to cervicography, which generally takes a minimum of several weeks to receive a report. Computerbased telecolposcopy also allows interaction between the on-site provider and remote expert.

Third, cervicography is a screening test adjunct. The computer-based system was used as a colposcopy diagnostic adjunct. However, colposcopy could easily be adapted to provide the function of cervicography. A simple handheld miniature change-coupled device camera and light source could potentially replace a more expensive colposcope and video camera, or video colposcope. With an average laptop computer (with appropriate software) and cellular phone, health care providers of potentially all women in the world could have access to expert-level cervical evaluation services.

Finally, computer-based telecolposcopy images and associated data automatically become part of a modern electronic medical record. This format is more conducive to the direction toward which contemporary medicine is rapidly shifting. Consequently, computer-based telecolposcopy may offer clinicians superior, modern diagnostic services not previously available to women.

Acknowledgments

Special thanks to Dr. Debra Crawley and Diane Watson, MSN, for rural site participation.

Telemedicine enables doctors in rural areas or areas with poor medical service to consult with experts at distant locations. Telecolposcopy and cervicography both enable remote diagnoses of the cervix. The 2 methods differ in equipment, operations, image format, timeliness of consultation, and probably cost. However, these diagnostic approaches have not been compared previously. The purpose of this study was to compare the accuracy of telecolposcopy and cervicography with on-site colposcopy in the remote evaluation of women with potential cervical neoplasia.

Telecolposcopy and cervicography

Telecolposcopy involves a distant expert colposcopist’s evaluation of women with potential lower genital tract neoplasia.¹ Existing telemedicine network and computer systems provide an audiovisual interface between local colposcopists and expert colposcopists at other locations.² For health systems already using computer or video networks, telecolposcopic consultation can be implemented with only small additional charges per examination.² Telecolposcopy services may improve health care access for women in medically underserved areas.¹

Cervicography is distant evaluation of 2 photographs taken of the cervix following 5% acetic acid application.³ A special 35-mm camera is used to take these images. The end product, developed at a central processing center, resembles a low-magnification colposcopic photograph. Certified evaluators interpret these images, classifying them as negative, atypical, or positive. Cervicography is used primarily as an adjunct test to the Papanicolaou (Pap) smear.⁴ It has also been evaluated as an intermediate triage test for evaluating women with mildly abnormal Pap smear results.^5-8

Methods

Women aged 18 years or older who came to 1 of 2 rural clinic sites for a colposcopic examination were enrolled in the trial after signing an institutional review board–approved informed consent document. We included women with a recent abnormal Pap smear report or a lower genital tract finding that required further evaluation by colposcopy. The exclusion criteria were pregnancy, severe cervicitis, heavy menses, refusal to participate, or technical problems with the telecolposcopy or cervicography equipment.

Both clinics were part of the Medical College of Georgia Telemedicine Network. This system uses sophisticated telecommunications equipment to provide distant consultation services to clinicians practicing in rural areas of the state.¹ Small change-coupled device cameras were attached to the colposcopes at the 2 clinics.

For network telecolposcopy, images were transmitted using the network’s existing hardware and high-speed telecommunication lines. For computer telecolposcopy, personal computers (DIMS, DenVu, Tucson, Ariz) were also used to capture and transmit images to a computer at the Telemedicine Center. These digitized images were transmitted by modem via telephone lines.² Cerviscopes (35-mm cameras) supplied by the manufacturer (NTL Worldwide, Fenton, Mo) were used to acquire cervigrams (photographs).

Pertinent clinicians received appropriate training to take cervigrams. Certified evaluators interpreted the images according to company protocol and returned a standardized report to the investigators at a later date.

Study design

The study design has been described in detail previously.^1,2 Briefly, subjects were initially examined by 1 of 3 on-site, university-based expert colposcopists, who took 2 cervigrams of each patient, and then conducted a colposcopic examination independently.

A local clinician then completed another colposcopic examination, including histologic sampling, if indicated. This examination was observed simultaneously by another expert at a telemedicine center. Prior to obtaining histologic samples or using dilute Lugol’s iodine solution, the local clinician captured 2 cervical images (low and high magnification) using the computer telemedicine system. These images were then transmitted to the expert at the telemedicine center for independent interpretation.

A third expert colposcopist interpreted the video and computer images at a later time. However, these third interpretations were not considered in this report. Colposcopists were blinded to each other’s clinical diagnoses. However, all colposcopists were informed of the subject’s referral cervical cytology results and other pertinent history.

Data analysis

Each subject had 2 observations using each of the 3 colposcopy methods (on-site, network, and computer-based), and a single observation using cervicography. On-site colposcopy, consisting of the observations of the on-site expert and local colposcopist, was considered for reference purposes. Agreement with histologic results was calculated for each method, across all histologic diagnoses together and separately by diagnosis.

Sensitivity and specificity estimates were calculated using 2 definitions of disease: (1) normal versus any other histologic diagnosis, and (2) normal or cervical intraepithelial neoplasia 1 (CIN 1) versus any more severe diagnosis. The primary analysis model was complete block analysis of variance, with subjects included as blocks in the analysis to account for the multiple observations on the same subjects. Nonparametric comparisons of proportions of agreement with histology, sensitivity, and specificity among the methods were made using permutation tests. Post-hoc comparisons were made using a Tukey test; 95% confidence intervals (CIs) were calculated for all point estimates. Adjustment for dependence among multiple observations per subject was made by basing these tests and CIs on least-squares means.

The available sample sizes for all analyses were adequate to ensure approximate normality of the estimated means. Power to detect, at Α=.05, a difference in agreement of 15% between cervigram and the other evaluation methods, was estimated using Monte Carlo simulations. Data were simulated using the observed levels of agreement for on-site, network, and computer telecolposcopy, and specifying a difference of 15% between cervicography agreement and the maximum of the other methods’ agreement. Power estimates were based on analysis of 1000 simulations. SAS release 8.02 was used for all calculations (SAS, Inc, Cary, NC).

Results

A total of 264 subjects were enrolled in the trial, but the total number of subjects considered differed depending on the various analyses of interest. The demographic data of this study cohort have been published previously.¹

Briefly, the subjects’ mean age was 31.7 years and mean parity was 2.1. Subjects presented with a wide range of prior cervical cytology results: 20.4% normal, 29.2% atypical squamous cells of undetermined significance, 40.4% low-grade squamous intraepithelial lesion, 7.3% high-grade squamous intraepithelial lesion, and 2.7% atypical glandular cells of undetermined significance. Histology results included all levels of CIN (52.9% CIN 1 and 13.4% CIN 2 or 3), and endocervical histologic sampling results were reported as both positive and negative for neoplasia.

The agreement between telecolposcopic/cervicography impressions and histology were estimated (Table 1). Data for on-site colposcopy was also considered for reference purposes.

When all histologic diagnoses were considered, there was no statistically significant difference in the rates of agreement for colposcopy, the 2 types of telecolposcopy, and cervicography. This was also true if only cases of CIN 1 were examined.

However, a statistically significant difference was noted between agreement rates for computer-based telecolposcopy (63.95%) and on-site colposcopy (47.7%, P=.03, Tukey test) for normal histology. A statistically significant difference was also found between agreement rates for on-site colposcopy (50.0%) and cervicography (19.1%, P=.04, Tukey test) for women with biopsy-proven CIN 2 or 3. If all histologic diagnoses were considered, the study provided 85% power to detect a difference in agreement of 15% among the evaluation methods.

We also estimated the sensitivity and specificity of the four diagnostic methods to detect cervical neoplasia (Table 2). A statistically significant difference was found in observed sensitivity between on-site colposcopy (47.7%) and cervicography (18.2%, P=.04, Tukey test) when a positive threshold of at least CIN 2 was considered. The difference was not significant, however, if the lower positive test threshold of at least CIN 1 was considered.

A statistically significant difference in specificity was noted between computer-based telecolposcopy (64.0%) and on-site colposcopy (47.7%, P=.03, Tukey test) at a positive threshold of at least CIN 1. The study provided a power of 71% and 60% to detect differences of 15% in sensitivity and specificity, respectively, using the CIN 1 threshold. Using CIN 2 as the positive threshold, the power to detect this 15% difference was 24% and 81% for sensitivity and specificity, respectively.

TABLE 1
Colposcopic, telecolposcopic, and cervicographic agreement with histology

TABLE 2
Sensitivity and specificity of tests to detect cervical neoplasia

Discussion

Until recently, cervicography had been the only type of remote diagnostic system available for the evaluation of women with potential lower genital tract neoplasia. With the advent of telemedicine during the past decade, expert-level health care has now become more readily available to patients previously isolated from this important resource.

The future of telecolposcopy

Because of its nature, telecolposcopy may also be well suited to assist in the evaluation and management of women with lower genital tract neoplasia. Computer-based telecolposcopy has the potential to support clinical sites located wherever standard telephone service exists. Cellular telephone systems now broaden access to nearglobal availability. Soon, assuming sufficient funding is obtained, the provision of expertenhanced colposcopy may become a reality for all women. However, universal availability may be irrelevant if computer-based telecolposcopy performs at a substandard level.

Telecolposcopy vs cervicography

We have demonstrated that telecolposcopy was at least as effective as cervicography for detecting cervical cancer precursors. Although the difference was not statistically significant, both network and computer-based telecolposcopy systems detected a higher percentage of women with CIN 2 or 3 than cervicography.

Our results also included on-site colposcopy. As anticipated, on-site colposcopy had the greatest sensitivity for disease detection at either positive test thresholds (at least CIN 1 and CIN 2). Ability to manipulate the cervix, stereoscopic viewing, longitudinal observation after 5% acetic acid application, and better resolution of the cervical epithelium and vascularity all favor on-site colposcopic diagnoses. Of the 2 telecolposcopy systems, network telecolposcopy had a slightly, but not significantly, greater sensitivity for detecting cervical cancer precursors compared with computer-based telecolposcopy.

Expert colposcopists’ accuracy with interpretation of network (real-time) cervical images was similar to that for on-site colposcopy, as might be expected. Network telecolposcopy might be equated with remote video colposcopy. Previously we have shown that traditional optical colposcopy is equivalent to video colposcopy with respect to colposcopic/histologic agreement.⁹

Comparison of telecolposcopy systems

The computer-based telecolposcopy system used in our study was, in all fairness, more similar to cervicography. Each method involves evaluation of 2 static images. Computer-based telecolposcopy provides 2 digitized images, but of a low- and high-power magnification view of the cervix. In comparison, cervicography produces dual low-power magnification celluloid images (2 x 2 slides) of the cervix. The provision of a high-power cervical image may explain the better sensitivity of computer-based telecolposcopy. This one feature may be more valuable than the better image resolution obtained from cervicography. However, computer-based resolution appears to be sufficient to render diagnoses at a level equivalent to or better than cervicography.

These 2 “static” systems differ in other aspects as well. First, computer-based systems are nonproprietary. Several systems are commercially available and other colposcopists have devised their own unique systems using modifications of off-the-shelf technology. Although not available at the initiation of our trial, computerbased systems now have the capability of capturing short video streams. These video segments should help improve the diagnostic ability of consulting colposcopists as demonstrated by our study.

Second, computer-based telecolposcopy can provide instantaneous consultation as opposed to cervicography, which generally takes a minimum of several weeks to receive a report. Computerbased telecolposcopy also allows interaction between the on-site provider and remote expert.

Third, cervicography is a screening test adjunct. The computer-based system was used as a colposcopy diagnostic adjunct. However, colposcopy could easily be adapted to provide the function of cervicography. A simple handheld miniature change-coupled device camera and light source could potentially replace a more expensive colposcope and video camera, or video colposcope. With an average laptop computer (with appropriate software) and cellular phone, health care providers of potentially all women in the world could have access to expert-level cervical evaluation services.

Finally, computer-based telecolposcopy images and associated data automatically become part of a modern electronic medical record. This format is more conducive to the direction toward which contemporary medicine is rapidly shifting. Consequently, computer-based telecolposcopy may offer clinicians superior, modern diagnostic services not previously available to women.

Acknowledgments

Special thanks to Dr. Debra Crawley and Diane Watson, MSN, for rural site participation.

Telemedicine enables doctors in rural areas or areas with poor medical service to consult with experts at distant locations. Telecolposcopy and cervicography both enable remote diagnoses of the cervix. The 2 methods differ in equipment, operations, image format, timeliness of consultation, and probably cost. However, these diagnostic approaches have not been compared previously. The purpose of this study was to compare the accuracy of telecolposcopy and cervicography with on-site colposcopy in the remote evaluation of women with potential cervical neoplasia.

Telecolposcopy and cervicography

Telecolposcopy involves a distant expert colposcopist’s evaluation of women with potential lower genital tract neoplasia.¹ Existing telemedicine network and computer systems provide an audiovisual interface between local colposcopists and expert colposcopists at other locations.² For health systems already using computer or video networks, telecolposcopic consultation can be implemented with only small additional charges per examination.² Telecolposcopy services may improve health care access for women in medically underserved areas.¹

Cervicography is distant evaluation of 2 photographs taken of the cervix following 5% acetic acid application.³ A special 35-mm camera is used to take these images. The end product, developed at a central processing center, resembles a low-magnification colposcopic photograph. Certified evaluators interpret these images, classifying them as negative, atypical, or positive. Cervicography is used primarily as an adjunct test to the Papanicolaou (Pap) smear.⁴ It has also been evaluated as an intermediate triage test for evaluating women with mildly abnormal Pap smear results.^5-8

Methods

Women aged 18 years or older who came to 1 of 2 rural clinic sites for a colposcopic examination were enrolled in the trial after signing an institutional review board–approved informed consent document. We included women with a recent abnormal Pap smear report or a lower genital tract finding that required further evaluation by colposcopy. The exclusion criteria were pregnancy, severe cervicitis, heavy menses, refusal to participate, or technical problems with the telecolposcopy or cervicography equipment.

Both clinics were part of the Medical College of Georgia Telemedicine Network. This system uses sophisticated telecommunications equipment to provide distant consultation services to clinicians practicing in rural areas of the state.¹ Small change-coupled device cameras were attached to the colposcopes at the 2 clinics.

For network telecolposcopy, images were transmitted using the network’s existing hardware and high-speed telecommunication lines. For computer telecolposcopy, personal computers (DIMS, DenVu, Tucson, Ariz) were also used to capture and transmit images to a computer at the Telemedicine Center. These digitized images were transmitted by modem via telephone lines.² Cerviscopes (35-mm cameras) supplied by the manufacturer (NTL Worldwide, Fenton, Mo) were used to acquire cervigrams (photographs).

Pertinent clinicians received appropriate training to take cervigrams. Certified evaluators interpreted the images according to company protocol and returned a standardized report to the investigators at a later date.

Study design

The study design has been described in detail previously.^1,2 Briefly, subjects were initially examined by 1 of 3 on-site, university-based expert colposcopists, who took 2 cervigrams of each patient, and then conducted a colposcopic examination independently.

A local clinician then completed another colposcopic examination, including histologic sampling, if indicated. This examination was observed simultaneously by another expert at a telemedicine center. Prior to obtaining histologic samples or using dilute Lugol’s iodine solution, the local clinician captured 2 cervical images (low and high magnification) using the computer telemedicine system. These images were then transmitted to the expert at the telemedicine center for independent interpretation.

A third expert colposcopist interpreted the video and computer images at a later time. However, these third interpretations were not considered in this report. Colposcopists were blinded to each other’s clinical diagnoses. However, all colposcopists were informed of the subject’s referral cervical cytology results and other pertinent history.

Data analysis

Each subject had 2 observations using each of the 3 colposcopy methods (on-site, network, and computer-based), and a single observation using cervicography. On-site colposcopy, consisting of the observations of the on-site expert and local colposcopist, was considered for reference purposes. Agreement with histologic results was calculated for each method, across all histologic diagnoses together and separately by diagnosis.

Sensitivity and specificity estimates were calculated using 2 definitions of disease: (1) normal versus any other histologic diagnosis, and (2) normal or cervical intraepithelial neoplasia 1 (CIN 1) versus any more severe diagnosis. The primary analysis model was complete block analysis of variance, with subjects included as blocks in the analysis to account for the multiple observations on the same subjects. Nonparametric comparisons of proportions of agreement with histology, sensitivity, and specificity among the methods were made using permutation tests. Post-hoc comparisons were made using a Tukey test; 95% confidence intervals (CIs) were calculated for all point estimates. Adjustment for dependence among multiple observations per subject was made by basing these tests and CIs on least-squares means.

The available sample sizes for all analyses were adequate to ensure approximate normality of the estimated means. Power to detect, at Α=.05, a difference in agreement of 15% between cervigram and the other evaluation methods, was estimated using Monte Carlo simulations. Data were simulated using the observed levels of agreement for on-site, network, and computer telecolposcopy, and specifying a difference of 15% between cervicography agreement and the maximum of the other methods’ agreement. Power estimates were based on analysis of 1000 simulations. SAS release 8.02 was used for all calculations (SAS, Inc, Cary, NC).

Results

A total of 264 subjects were enrolled in the trial, but the total number of subjects considered differed depending on the various analyses of interest. The demographic data of this study cohort have been published previously.¹

Briefly, the subjects’ mean age was 31.7 years and mean parity was 2.1. Subjects presented with a wide range of prior cervical cytology results: 20.4% normal, 29.2% atypical squamous cells of undetermined significance, 40.4% low-grade squamous intraepithelial lesion, 7.3% high-grade squamous intraepithelial lesion, and 2.7% atypical glandular cells of undetermined significance. Histology results included all levels of CIN (52.9% CIN 1 and 13.4% CIN 2 or 3), and endocervical histologic sampling results were reported as both positive and negative for neoplasia.

The agreement between telecolposcopic/cervicography impressions and histology were estimated (Table 1). Data for on-site colposcopy was also considered for reference purposes.

When all histologic diagnoses were considered, there was no statistically significant difference in the rates of agreement for colposcopy, the 2 types of telecolposcopy, and cervicography. This was also true if only cases of CIN 1 were examined.

However, a statistically significant difference was noted between agreement rates for computer-based telecolposcopy (63.95%) and on-site colposcopy (47.7%, P=.03, Tukey test) for normal histology. A statistically significant difference was also found between agreement rates for on-site colposcopy (50.0%) and cervicography (19.1%, P=.04, Tukey test) for women with biopsy-proven CIN 2 or 3. If all histologic diagnoses were considered, the study provided 85% power to detect a difference in agreement of 15% among the evaluation methods.

We also estimated the sensitivity and specificity of the four diagnostic methods to detect cervical neoplasia (Table 2). A statistically significant difference was found in observed sensitivity between on-site colposcopy (47.7%) and cervicography (18.2%, P=.04, Tukey test) when a positive threshold of at least CIN 2 was considered. The difference was not significant, however, if the lower positive test threshold of at least CIN 1 was considered.

A statistically significant difference in specificity was noted between computer-based telecolposcopy (64.0%) and on-site colposcopy (47.7%, P=.03, Tukey test) at a positive threshold of at least CIN 1. The study provided a power of 71% and 60% to detect differences of 15% in sensitivity and specificity, respectively, using the CIN 1 threshold. Using CIN 2 as the positive threshold, the power to detect this 15% difference was 24% and 81% for sensitivity and specificity, respectively.

TABLE 1
Colposcopic, telecolposcopic, and cervicographic agreement with histology

TABLE 2
Sensitivity and specificity of tests to detect cervical neoplasia

Discussion

Until recently, cervicography had been the only type of remote diagnostic system available for the evaluation of women with potential lower genital tract neoplasia. With the advent of telemedicine during the past decade, expert-level health care has now become more readily available to patients previously isolated from this important resource.

The future of telecolposcopy

Because of its nature, telecolposcopy may also be well suited to assist in the evaluation and management of women with lower genital tract neoplasia. Computer-based telecolposcopy has the potential to support clinical sites located wherever standard telephone service exists. Cellular telephone systems now broaden access to nearglobal availability. Soon, assuming sufficient funding is obtained, the provision of expertenhanced colposcopy may become a reality for all women. However, universal availability may be irrelevant if computer-based telecolposcopy performs at a substandard level.

Telecolposcopy vs cervicography

We have demonstrated that telecolposcopy was at least as effective as cervicography for detecting cervical cancer precursors. Although the difference was not statistically significant, both network and computer-based telecolposcopy systems detected a higher percentage of women with CIN 2 or 3 than cervicography.

Our results also included on-site colposcopy. As anticipated, on-site colposcopy had the greatest sensitivity for disease detection at either positive test thresholds (at least CIN 1 and CIN 2). Ability to manipulate the cervix, stereoscopic viewing, longitudinal observation after 5% acetic acid application, and better resolution of the cervical epithelium and vascularity all favor on-site colposcopic diagnoses. Of the 2 telecolposcopy systems, network telecolposcopy had a slightly, but not significantly, greater sensitivity for detecting cervical cancer precursors compared with computer-based telecolposcopy.

Expert colposcopists’ accuracy with interpretation of network (real-time) cervical images was similar to that for on-site colposcopy, as might be expected. Network telecolposcopy might be equated with remote video colposcopy. Previously we have shown that traditional optical colposcopy is equivalent to video colposcopy with respect to colposcopic/histologic agreement.⁹

Comparison of telecolposcopy systems

The computer-based telecolposcopy system used in our study was, in all fairness, more similar to cervicography. Each method involves evaluation of 2 static images. Computer-based telecolposcopy provides 2 digitized images, but of a low- and high-power magnification view of the cervix. In comparison, cervicography produces dual low-power magnification celluloid images (2 x 2 slides) of the cervix. The provision of a high-power cervical image may explain the better sensitivity of computer-based telecolposcopy. This one feature may be more valuable than the better image resolution obtained from cervicography. However, computer-based resolution appears to be sufficient to render diagnoses at a level equivalent to or better than cervicography.

These 2 “static” systems differ in other aspects as well. First, computer-based systems are nonproprietary. Several systems are commercially available and other colposcopists have devised their own unique systems using modifications of off-the-shelf technology. Although not available at the initiation of our trial, computerbased systems now have the capability of capturing short video streams. These video segments should help improve the diagnostic ability of consulting colposcopists as demonstrated by our study.

Second, computer-based telecolposcopy can provide instantaneous consultation as opposed to cervicography, which generally takes a minimum of several weeks to receive a report. Computerbased telecolposcopy also allows interaction between the on-site provider and remote expert.

Third, cervicography is a screening test adjunct. The computer-based system was used as a colposcopy diagnostic adjunct. However, colposcopy could easily be adapted to provide the function of cervicography. A simple handheld miniature change-coupled device camera and light source could potentially replace a more expensive colposcope and video camera, or video colposcope. With an average laptop computer (with appropriate software) and cellular phone, health care providers of potentially all women in the world could have access to expert-level cervical evaluation services.

Finally, computer-based telecolposcopy images and associated data automatically become part of a modern electronic medical record. This format is more conducive to the direction toward which contemporary medicine is rapidly shifting. Consequently, computer-based telecolposcopy may offer clinicians superior, modern diagnostic services not previously available to women.

Acknowledgments

Special thanks to Dr. Debra Crawley and Diane Watson, MSN, for rural site participation.

Associate Editor, Journal of Family Practice.

About: “After-hours telephone triage affects patient safety,”

Will all of you who enjoy taking after-hours calls please stand up?

What? Everyone is still sitting? That’s what I thought. Although taking calls after hours is not one of our favorite duties, after-hours care is a crucial component of primary health care. The recent Institute of Medicine report, Crossing the Quality Chasm.,¹ cited 6 characteristics essential for a high-quality health care system for the 21st century:

• safe
• effective
• efficient
• equitable
• timely
• patient-centered.

After-hours call coverage systems should pass muster on all 6 qualities. Do they?

Telephone triage after hours not up to standard

Hildebrandt, Westfall, and Smith provide evidence that the after-hours primary care call systems in the United States are not up to standard.² They investigated call coverage systems of 91 primary care practices in the Denver area by phoning the office numbers, following the recorded instructions, and asking how calls were managed when they spoke to a live person. More than two thirds of the offices used answering services to triage calls, and 93% of these required patients to decide whether the condition was serious enough to warrant contacting the physician on call (this is correct!).

I suppose one could call this approach “patient-centered,” but I suspect this strategy is more to lessen the burden of the on-call physician rather than to promote safe and effective patientcentered care.

The investigators then reviewed reports of all calls not forwarded to the physician on call from 1 of these 91 practices. (A list of calls not forwarded to the physician on call is routinely forwarded to the office the next day by fax.) The physician reviewers in this study judged 50% of these calls to be potentially serious; the patients should have been referred immediately to the physician on call. Clearly, our patients are not making good decisions about the potentially serious nature of their complaints.

To be fair, only 10% of all calls were not forwarded to the on-call physician. Further, the researchers did not investigate each case to determine whether the delay in contact resulted in any untoward events that might have been prevented by immediate referral to the on-call doctor. Perhaps all of the patients needing immediate attention found appropriate care on their own by going to an emergency department or urgent care center. Further research is needed to explore the extent to which medical errors related to afterhours call procedures contribute to adverse patient outcomes.

The Institute of Medicine’s report, To Err is Human,. reminds us that the best way to prevent errors is by improving care systems rather than by attributing personal blame.³ If systems are inadequate for the job, then even the best-intentioned practitioner will provide suboptimal care. Hildebrandt and associates spotted a weakness in the system, a latent error that is easily correctable.

Solutions

What is the solution? I agree with the authors: all after-hours calls for clinical questions should be referred in a timely manner to a clinician. The clinician may be a physician, a physician assistant, or a nurse practitioner. After-hours call systems should be monitored periodically to ensure the systems are safe, effective, efficient, equitable, timely, and patient-centered. Patient complaints about suboptimal after-hours care should be investigated promptly. Continuous quality improvement principles should be applied to assess and improve after-hours care systems, just as we use them to improve office care.

I see no reason to wait. Check out your own after-hours coverage system today to ensure that all clinical calls reach the attention of a competent clinician as soon as possible. You might get another call or two each night you are on call, but I believe the gain will be worth the pain.

Associate Editor, Journal of Family Practice.

About: “After-hours telephone triage affects patient safety,”

Will all of you who enjoy taking after-hours calls please stand up?

What? Everyone is still sitting? That’s what I thought. Although taking calls after hours is not one of our favorite duties, after-hours care is a crucial component of primary health care. The recent Institute of Medicine report, Crossing the Quality Chasm.,¹ cited 6 characteristics essential for a high-quality health care system for the 21st century:

• safe
• effective
• efficient
• equitable
• timely
• patient-centered.

After-hours call coverage systems should pass muster on all 6 qualities. Do they?

Telephone triage after hours not up to standard

Hildebrandt, Westfall, and Smith provide evidence that the after-hours primary care call systems in the United States are not up to standard.² They investigated call coverage systems of 91 primary care practices in the Denver area by phoning the office numbers, following the recorded instructions, and asking how calls were managed when they spoke to a live person. More than two thirds of the offices used answering services to triage calls, and 93% of these required patients to decide whether the condition was serious enough to warrant contacting the physician on call (this is correct!).

I suppose one could call this approach “patient-centered,” but I suspect this strategy is more to lessen the burden of the on-call physician rather than to promote safe and effective patientcentered care.

The investigators then reviewed reports of all calls not forwarded to the physician on call from 1 of these 91 practices. (A list of calls not forwarded to the physician on call is routinely forwarded to the office the next day by fax.) The physician reviewers in this study judged 50% of these calls to be potentially serious; the patients should have been referred immediately to the physician on call. Clearly, our patients are not making good decisions about the potentially serious nature of their complaints.

To be fair, only 10% of all calls were not forwarded to the on-call physician. Further, the researchers did not investigate each case to determine whether the delay in contact resulted in any untoward events that might have been prevented by immediate referral to the on-call doctor. Perhaps all of the patients needing immediate attention found appropriate care on their own by going to an emergency department or urgent care center. Further research is needed to explore the extent to which medical errors related to afterhours call procedures contribute to adverse patient outcomes.

The Institute of Medicine’s report, To Err is Human,. reminds us that the best way to prevent errors is by improving care systems rather than by attributing personal blame.³ If systems are inadequate for the job, then even the best-intentioned practitioner will provide suboptimal care. Hildebrandt and associates spotted a weakness in the system, a latent error that is easily correctable.

Solutions

What is the solution? I agree with the authors: all after-hours calls for clinical questions should be referred in a timely manner to a clinician. The clinician may be a physician, a physician assistant, or a nurse practitioner. After-hours call systems should be monitored periodically to ensure the systems are safe, effective, efficient, equitable, timely, and patient-centered. Patient complaints about suboptimal after-hours care should be investigated promptly. Continuous quality improvement principles should be applied to assess and improve after-hours care systems, just as we use them to improve office care.

I see no reason to wait. Check out your own after-hours coverage system today to ensure that all clinical calls reach the attention of a competent clinician as soon as possible. You might get another call or two each night you are on call, but I believe the gain will be worth the pain.

Associate Editor, Journal of Family Practice.

About: “After-hours telephone triage affects patient safety,”

Will all of you who enjoy taking after-hours calls please stand up?

What? Everyone is still sitting? That’s what I thought. Although taking calls after hours is not one of our favorite duties, after-hours care is a crucial component of primary health care. The recent Institute of Medicine report, Crossing the Quality Chasm.,¹ cited 6 characteristics essential for a high-quality health care system for the 21st century:

• safe
• effective
• efficient
• equitable
• timely
• patient-centered.

After-hours call coverage systems should pass muster on all 6 qualities. Do they?

Telephone triage after hours not up to standard

Hildebrandt, Westfall, and Smith provide evidence that the after-hours primary care call systems in the United States are not up to standard.² They investigated call coverage systems of 91 primary care practices in the Denver area by phoning the office numbers, following the recorded instructions, and asking how calls were managed when they spoke to a live person. More than two thirds of the offices used answering services to triage calls, and 93% of these required patients to decide whether the condition was serious enough to warrant contacting the physician on call (this is correct!).

I suppose one could call this approach “patient-centered,” but I suspect this strategy is more to lessen the burden of the on-call physician rather than to promote safe and effective patientcentered care.

The investigators then reviewed reports of all calls not forwarded to the physician on call from 1 of these 91 practices. (A list of calls not forwarded to the physician on call is routinely forwarded to the office the next day by fax.) The physician reviewers in this study judged 50% of these calls to be potentially serious; the patients should have been referred immediately to the physician on call. Clearly, our patients are not making good decisions about the potentially serious nature of their complaints.

To be fair, only 10% of all calls were not forwarded to the on-call physician. Further, the researchers did not investigate each case to determine whether the delay in contact resulted in any untoward events that might have been prevented by immediate referral to the on-call doctor. Perhaps all of the patients needing immediate attention found appropriate care on their own by going to an emergency department or urgent care center. Further research is needed to explore the extent to which medical errors related to afterhours call procedures contribute to adverse patient outcomes.

The Institute of Medicine’s report, To Err is Human,. reminds us that the best way to prevent errors is by improving care systems rather than by attributing personal blame.³ If systems are inadequate for the job, then even the best-intentioned practitioner will provide suboptimal care. Hildebrandt and associates spotted a weakness in the system, a latent error that is easily correctable.

Solutions

What is the solution? I agree with the authors: all after-hours calls for clinical questions should be referred in a timely manner to a clinician. The clinician may be a physician, a physician assistant, or a nurse practitioner. After-hours call systems should be monitored periodically to ensure the systems are safe, effective, efficient, equitable, timely, and patient-centered. Patient complaints about suboptimal after-hours care should be investigated promptly. Continuous quality improvement principles should be applied to assess and improve after-hours care systems, just as we use them to improve office care.

I see no reason to wait. Check out your own after-hours coverage system today to ensure that all clinical calls reach the attention of a competent clinician as soon as possible. You might get another call or two each night you are on call, but I believe the gain will be worth the pain.

We found recently that about 10% of after-hours calls from patients were not forwarded by the answering service to the physician on call because the patient did not think the problem was an emergency.¹ In reviewing these calls, it became evident that many were indeed serious enough to require immediate contact with a medical professional.

The purpose of this study was to evaluate the management of after-hours phone calls made to primary care physicians’ offices and their answering services in a large metropolitan area. General descriptions of after-hours calls have been reported,^2,3,4 and the management of these calls by professional and nurse triage services have been studied.^5,6 However, the management of telephone triage by answering services has not been examined. No published data exist on the number of after-hours phone calls to US physicians.

Methods

This study had 2 components. In part 1, we surveyed 91 primary care offices (in family practice, internal medicine, obstetrics, and pediatrics) to determine how they handle after-hours phone calls. In part 2, we analyzed all calls from our previous study¹ that were not identified by the patient as an emergency and, hence, not forwarded to the on-call physician.

Survey of primary care physicians’ answering services

The physicians in each specialty were identified in their respective section of the telephone book,⁷ and, by using a systematic sampling technique, every fifth name was selected and surveyed. All surveys were completed in October and November 2001 after regular office hours, generally between 10:00 PM and 1:00 AM.

Using a structured survey interview form, the principal investigator indicated during each call that this was an anonymous research survey and asked if the answering service personnel could answer several questions. The information collected in each 3- to 5-minute interview included: whether there was a recorded message, whether the patient was instructed to call 911, who answered the call after the recorded message, what information was requested, who made the decision to initiate contact with the on-call physician, and what happened to calls that were not forwarded.

If the patient was instructed to choose an “option” from the medical office telephone system, this option was selected if it would lead to an answering service. If it offered to call or page the physician directly, then that survey was terminated. The name of the answering service was recorded to determine how many different services were used in this metropolitan area. We did not survey offices on how they managed the phone call reports received the next day or how they managed clinical calls during regular office hours.

Analysis of phone calls classified by patients as nonemergent

In our previous study,¹ we entered the chief complaint of all after-hours telephone calls made to our community-based family practice training program between April 2000 and March 2001 into an Access database program (Microsoft Access 97, Microsoft Corporation, Redmond, WA). These after-hours calls were routed to an answering service when the office was closed. Patients were asked by the answering service: “Is this an emergency?” Patients who were not certain were asked if they needed to speak directly with the physician. The calls were sent to the physician on call only if the patient stated to the answering service operator that the problem was an “emergency” or if they were uncertain and requested to speak directly with the physician.

For this study, we analyzed only the nonemergency calls that were not forwarded to a physician. We chose 4 local family physicians who were unaware of the purpose of the study to review these calls. We asked them to: “Indicate which of these complaints you want your after-hours answering service to forward to the physician on call and which can wait to be faxed to the office the following morning.” We analyzed their responses with descriptive statistics (SAS 8.0, SAS Institute, Cary, NC) and an overall multirater statistic (Magree macro 1.0, SAS Institute). The HealthOne Institutional Review Board approved this study.

Results

Survey of primary care physicians’ answering services

Table 1 presents the results of our survey of primary care physicians. Most physicians had a recorded message instructing the patient how to reach the physician after hours. In 4 cases, the message implied that the patient should not call unless that person had a “true emergency.”

After calling 5 pediatricians, it became clear that the pediatricians used a single, well-described nurse triage service for managing after-hours calls,⁵ and the pediatric offices were not included in further analysis. We have only partial data for 2 physicians because their answering service was too busy to complete the survey.

Fifty-six percent of the offices had recorded messages that instructed the caller to hang up and dial 911 if the problem was a “life-threatening” emergency. After the initial recorded message, 67% of the calls were answered by an answering service.

A full 93% of the answering services required the patient to decide whether to initiate contact with the on-call physician. Only those calls reported by the caller to be an “emergency” were forwarded to the on-call physician. In 2 cases, the answering service operator suggested to us that they were instructed to “use their judgment” in forwarding calls to the on-call physician. Five of the answering services commented that about 90% of the calls are forwarded to the physician and 10% are not forwarded, closely matching our previous findings.¹

Ninety-five percent of the answering services faxed reports on all calls, including those not forwarded during the night, to the offices the following business day. Twelve answering services were used by the 91 practices in our study: 2 handled only family practice offices, 1 handled only internal medicine offices, 1 handled only obstetric offices, and 8 handled calls for multiple specialties.

Analysis of phone calls classified by patients as nonemergent

Over 1 year, 2835 clinical calls (eg, not administrative or appointment cancellations) were made to the office after hours, and 90% were considered to be an emergency and forwarded to the oncall physician. The remaining 10% (288 calls) were faxed to the office the next day. Table 2 shows examples of those calls that were not forwarded. Our 4 physician reviewers of the nonemergency calls wanted to speak to the patient immediately at a mean of 50% of the calls rather than wait until the following business day (range, 22%%–77%, κ=.45).

TABLE 1
Telephone triage summary by specialty

TABLE 2
Sample of calls classified as nonemergent by patients

Discussion

In studying after-hours phone calls, we found several systematic barriers between patients and physicians: wrong numbers, messages necessitating a second phone call, and requirements that the patient decide whether the medical complaint was serious enough to initiate contact with the oncall physician. These barriers may negatively affect patient health due to unnecessary delays in evaluation and treatment.

Most patients asked to speak with the physician immediately about important clinical matters: medications, chest pain, contractions, or fever. However, some patients appeared unable to make appropriate triage decisions or persevere long enough to overcome the systematic barriers that prevented them from talking to a physician.

Our physician panel would have wanted to talk to the “no emergency” patients immediately in approximately half the cases. If 10% of 50 million to 100 million after-hours phone calls each year in the United States are not forwarded to the physician because the caller feels the complaint is not emergent, and if half those calls are potentially serious, there may be as many as 2.5 million to 5 million potentially dangerous delays in care each year.

We cannot expect an answering service operator or a parent to know how to triage an infant with a fever when physicians disagree on appropriate disposition.⁸ New parents with a sick infant, an older patient with chest pain, or a woman having preterm contractions during her first pregnancy might be uncertain as to what constitutes an “emergency.”

Solutions

Several solutions to this problem exist. We made a change in our office and now have all clinical calls forwarded to the on-call physician. No triage decisions are made by the patient or the answering service. This has led to an average increase of only 1 to 2 more patient calls per night. Offices also could become part of a citywide network in which all calls are managed by a trained nursing staff, as the pediatricians have done in Denver, Colorado.⁵

Interpretations

This study should be interpreted in light of several limitations. First, it was conducted in 1 metropolitan region. It is possible that other regions of the US have different mechanisms or standards for handling after-hours calls. However, given the overwhelming number of offices in our study that required patients to make their own triage decisions, we believe this barrier is likely widespread.

Second, the answering services we surveyed knew we were not patients, and this may have affected their answers. However, even if only 10% of these calls were not forwarded to the physician on call, a significant number of calls might have been unnecessarily delayed and potentially put patients at risk.

The Institute of Medicine’s report on medical errors states: “Errors can be prevented by designing systems that make it hard for people to do the wrong thing and easy for people to do the right thing.”⁹ Errors in triage by the patient or the answering service may lead to dangerous delays in necessary patient care.

Our future research will focus on identifying adverse outcomes in this study population and prospectively in a practice-based research network. When a patient calls the primary care office after hours, the decisions should be simple and left to those who have the training to make those decisions based on their best medical judgment. We strongly urge all clinicians who use an answering service to examine their policies and procedures for potential sources of medical error.

Acknowledgments

We express our thanks to Tarek Arja, DO, Dan O’Brien, DO, Mark Cucuzzella, MD, and Jacqueline Stern, MD; for agreeing to review nonemergent calls, and Pamela Sullivan for her assistance in preparing the manuscript.

We found recently that about 10% of after-hours calls from patients were not forwarded by the answering service to the physician on call because the patient did not think the problem was an emergency.¹ In reviewing these calls, it became evident that many were indeed serious enough to require immediate contact with a medical professional.

The purpose of this study was to evaluate the management of after-hours phone calls made to primary care physicians’ offices and their answering services in a large metropolitan area. General descriptions of after-hours calls have been reported,^2,3,4 and the management of these calls by professional and nurse triage services have been studied.^5,6 However, the management of telephone triage by answering services has not been examined. No published data exist on the number of after-hours phone calls to US physicians.

Methods

This study had 2 components. In part 1, we surveyed 91 primary care offices (in family practice, internal medicine, obstetrics, and pediatrics) to determine how they handle after-hours phone calls. In part 2, we analyzed all calls from our previous study¹ that were not identified by the patient as an emergency and, hence, not forwarded to the on-call physician.

Survey of primary care physicians’ answering services

The physicians in each specialty were identified in their respective section of the telephone book,⁷ and, by using a systematic sampling technique, every fifth name was selected and surveyed. All surveys were completed in October and November 2001 after regular office hours, generally between 10:00 PM and 1:00 AM.

Using a structured survey interview form, the principal investigator indicated during each call that this was an anonymous research survey and asked if the answering service personnel could answer several questions. The information collected in each 3- to 5-minute interview included: whether there was a recorded message, whether the patient was instructed to call 911, who answered the call after the recorded message, what information was requested, who made the decision to initiate contact with the on-call physician, and what happened to calls that were not forwarded.

If the patient was instructed to choose an “option” from the medical office telephone system, this option was selected if it would lead to an answering service. If it offered to call or page the physician directly, then that survey was terminated. The name of the answering service was recorded to determine how many different services were used in this metropolitan area. We did not survey offices on how they managed the phone call reports received the next day or how they managed clinical calls during regular office hours.

Analysis of phone calls classified by patients as nonemergent

In our previous study,¹ we entered the chief complaint of all after-hours telephone calls made to our community-based family practice training program between April 2000 and March 2001 into an Access database program (Microsoft Access 97, Microsoft Corporation, Redmond, WA). These after-hours calls were routed to an answering service when the office was closed. Patients were asked by the answering service: “Is this an emergency?” Patients who were not certain were asked if they needed to speak directly with the physician. The calls were sent to the physician on call only if the patient stated to the answering service operator that the problem was an “emergency” or if they were uncertain and requested to speak directly with the physician.

For this study, we analyzed only the nonemergency calls that were not forwarded to a physician. We chose 4 local family physicians who were unaware of the purpose of the study to review these calls. We asked them to: “Indicate which of these complaints you want your after-hours answering service to forward to the physician on call and which can wait to be faxed to the office the following morning.” We analyzed their responses with descriptive statistics (SAS 8.0, SAS Institute, Cary, NC) and an overall multirater statistic (Magree macro 1.0, SAS Institute). The HealthOne Institutional Review Board approved this study.

Results

Survey of primary care physicians’ answering services

Table 1 presents the results of our survey of primary care physicians. Most physicians had a recorded message instructing the patient how to reach the physician after hours. In 4 cases, the message implied that the patient should not call unless that person had a “true emergency.”

After calling 5 pediatricians, it became clear that the pediatricians used a single, well-described nurse triage service for managing after-hours calls,⁵ and the pediatric offices were not included in further analysis. We have only partial data for 2 physicians because their answering service was too busy to complete the survey.

Fifty-six percent of the offices had recorded messages that instructed the caller to hang up and dial 911 if the problem was a “life-threatening” emergency. After the initial recorded message, 67% of the calls were answered by an answering service.

A full 93% of the answering services required the patient to decide whether to initiate contact with the on-call physician. Only those calls reported by the caller to be an “emergency” were forwarded to the on-call physician. In 2 cases, the answering service operator suggested to us that they were instructed to “use their judgment” in forwarding calls to the on-call physician. Five of the answering services commented that about 90% of the calls are forwarded to the physician and 10% are not forwarded, closely matching our previous findings.¹

Ninety-five percent of the answering services faxed reports on all calls, including those not forwarded during the night, to the offices the following business day. Twelve answering services were used by the 91 practices in our study: 2 handled only family practice offices, 1 handled only internal medicine offices, 1 handled only obstetric offices, and 8 handled calls for multiple specialties.

Analysis of phone calls classified by patients as nonemergent

Over 1 year, 2835 clinical calls (eg, not administrative or appointment cancellations) were made to the office after hours, and 90% were considered to be an emergency and forwarded to the oncall physician. The remaining 10% (288 calls) were faxed to the office the next day. Table 2 shows examples of those calls that were not forwarded. Our 4 physician reviewers of the nonemergency calls wanted to speak to the patient immediately at a mean of 50% of the calls rather than wait until the following business day (range, 22%%–77%, κ=.45).

TABLE 1
Telephone triage summary by specialty

TABLE 2
Sample of calls classified as nonemergent by patients

Discussion

In studying after-hours phone calls, we found several systematic barriers between patients and physicians: wrong numbers, messages necessitating a second phone call, and requirements that the patient decide whether the medical complaint was serious enough to initiate contact with the oncall physician. These barriers may negatively affect patient health due to unnecessary delays in evaluation and treatment.

Most patients asked to speak with the physician immediately about important clinical matters: medications, chest pain, contractions, or fever. However, some patients appeared unable to make appropriate triage decisions or persevere long enough to overcome the systematic barriers that prevented them from talking to a physician.

Our physician panel would have wanted to talk to the “no emergency” patients immediately in approximately half the cases. If 10% of 50 million to 100 million after-hours phone calls each year in the United States are not forwarded to the physician because the caller feels the complaint is not emergent, and if half those calls are potentially serious, there may be as many as 2.5 million to 5 million potentially dangerous delays in care each year.

We cannot expect an answering service operator or a parent to know how to triage an infant with a fever when physicians disagree on appropriate disposition.⁸ New parents with a sick infant, an older patient with chest pain, or a woman having preterm contractions during her first pregnancy might be uncertain as to what constitutes an “emergency.”

Solutions

Several solutions to this problem exist. We made a change in our office and now have all clinical calls forwarded to the on-call physician. No triage decisions are made by the patient or the answering service. This has led to an average increase of only 1 to 2 more patient calls per night. Offices also could become part of a citywide network in which all calls are managed by a trained nursing staff, as the pediatricians have done in Denver, Colorado.⁵

Interpretations

This study should be interpreted in light of several limitations. First, it was conducted in 1 metropolitan region. It is possible that other regions of the US have different mechanisms or standards for handling after-hours calls. However, given the overwhelming number of offices in our study that required patients to make their own triage decisions, we believe this barrier is likely widespread.

Second, the answering services we surveyed knew we were not patients, and this may have affected their answers. However, even if only 10% of these calls were not forwarded to the physician on call, a significant number of calls might have been unnecessarily delayed and potentially put patients at risk.

The Institute of Medicine’s report on medical errors states: “Errors can be prevented by designing systems that make it hard for people to do the wrong thing and easy for people to do the right thing.”⁹ Errors in triage by the patient or the answering service may lead to dangerous delays in necessary patient care.

Our future research will focus on identifying adverse outcomes in this study population and prospectively in a practice-based research network. When a patient calls the primary care office after hours, the decisions should be simple and left to those who have the training to make those decisions based on their best medical judgment. We strongly urge all clinicians who use an answering service to examine their policies and procedures for potential sources of medical error.

Acknowledgments

We express our thanks to Tarek Arja, DO, Dan O’Brien, DO, Mark Cucuzzella, MD, and Jacqueline Stern, MD; for agreeing to review nonemergent calls, and Pamela Sullivan for her assistance in preparing the manuscript.

We found recently that about 10% of after-hours calls from patients were not forwarded by the answering service to the physician on call because the patient did not think the problem was an emergency.¹ In reviewing these calls, it became evident that many were indeed serious enough to require immediate contact with a medical professional.

The purpose of this study was to evaluate the management of after-hours phone calls made to primary care physicians’ offices and their answering services in a large metropolitan area. General descriptions of after-hours calls have been reported,^2,3,4 and the management of these calls by professional and nurse triage services have been studied.^5,6 However, the management of telephone triage by answering services has not been examined. No published data exist on the number of after-hours phone calls to US physicians.

Methods

This study had 2 components. In part 1, we surveyed 91 primary care offices (in family practice, internal medicine, obstetrics, and pediatrics) to determine how they handle after-hours phone calls. In part 2, we analyzed all calls from our previous study¹ that were not identified by the patient as an emergency and, hence, not forwarded to the on-call physician.

Survey of primary care physicians’ answering services

The physicians in each specialty were identified in their respective section of the telephone book,⁷ and, by using a systematic sampling technique, every fifth name was selected and surveyed. All surveys were completed in October and November 2001 after regular office hours, generally between 10:00 PM and 1:00 AM.

Using a structured survey interview form, the principal investigator indicated during each call that this was an anonymous research survey and asked if the answering service personnel could answer several questions. The information collected in each 3- to 5-minute interview included: whether there was a recorded message, whether the patient was instructed to call 911, who answered the call after the recorded message, what information was requested, who made the decision to initiate contact with the on-call physician, and what happened to calls that were not forwarded.

If the patient was instructed to choose an “option” from the medical office telephone system, this option was selected if it would lead to an answering service. If it offered to call or page the physician directly, then that survey was terminated. The name of the answering service was recorded to determine how many different services were used in this metropolitan area. We did not survey offices on how they managed the phone call reports received the next day or how they managed clinical calls during regular office hours.

Analysis of phone calls classified by patients as nonemergent

In our previous study,¹ we entered the chief complaint of all after-hours telephone calls made to our community-based family practice training program between April 2000 and March 2001 into an Access database program (Microsoft Access 97, Microsoft Corporation, Redmond, WA). These after-hours calls were routed to an answering service when the office was closed. Patients were asked by the answering service: “Is this an emergency?” Patients who were not certain were asked if they needed to speak directly with the physician. The calls were sent to the physician on call only if the patient stated to the answering service operator that the problem was an “emergency” or if they were uncertain and requested to speak directly with the physician.

For this study, we analyzed only the nonemergency calls that were not forwarded to a physician. We chose 4 local family physicians who were unaware of the purpose of the study to review these calls. We asked them to: “Indicate which of these complaints you want your after-hours answering service to forward to the physician on call and which can wait to be faxed to the office the following morning.” We analyzed their responses with descriptive statistics (SAS 8.0, SAS Institute, Cary, NC) and an overall multirater statistic (Magree macro 1.0, SAS Institute). The HealthOne Institutional Review Board approved this study.

Results

Survey of primary care physicians’ answering services

Table 1 presents the results of our survey of primary care physicians. Most physicians had a recorded message instructing the patient how to reach the physician after hours. In 4 cases, the message implied that the patient should not call unless that person had a “true emergency.”

After calling 5 pediatricians, it became clear that the pediatricians used a single, well-described nurse triage service for managing after-hours calls,⁵ and the pediatric offices were not included in further analysis. We have only partial data for 2 physicians because their answering service was too busy to complete the survey.

Fifty-six percent of the offices had recorded messages that instructed the caller to hang up and dial 911 if the problem was a “life-threatening” emergency. After the initial recorded message, 67% of the calls were answered by an answering service.

A full 93% of the answering services required the patient to decide whether to initiate contact with the on-call physician. Only those calls reported by the caller to be an “emergency” were forwarded to the on-call physician. In 2 cases, the answering service operator suggested to us that they were instructed to “use their judgment” in forwarding calls to the on-call physician. Five of the answering services commented that about 90% of the calls are forwarded to the physician and 10% are not forwarded, closely matching our previous findings.¹

Ninety-five percent of the answering services faxed reports on all calls, including those not forwarded during the night, to the offices the following business day. Twelve answering services were used by the 91 practices in our study: 2 handled only family practice offices, 1 handled only internal medicine offices, 1 handled only obstetric offices, and 8 handled calls for multiple specialties.

Analysis of phone calls classified by patients as nonemergent

Over 1 year, 2835 clinical calls (eg, not administrative or appointment cancellations) were made to the office after hours, and 90% were considered to be an emergency and forwarded to the oncall physician. The remaining 10% (288 calls) were faxed to the office the next day. Table 2 shows examples of those calls that were not forwarded. Our 4 physician reviewers of the nonemergency calls wanted to speak to the patient immediately at a mean of 50% of the calls rather than wait until the following business day (range, 22%%–77%, κ=.45).

TABLE 1
Telephone triage summary by specialty

TABLE 2
Sample of calls classified as nonemergent by patients

Discussion

In studying after-hours phone calls, we found several systematic barriers between patients and physicians: wrong numbers, messages necessitating a second phone call, and requirements that the patient decide whether the medical complaint was serious enough to initiate contact with the oncall physician. These barriers may negatively affect patient health due to unnecessary delays in evaluation and treatment.

Most patients asked to speak with the physician immediately about important clinical matters: medications, chest pain, contractions, or fever. However, some patients appeared unable to make appropriate triage decisions or persevere long enough to overcome the systematic barriers that prevented them from talking to a physician.

Our physician panel would have wanted to talk to the “no emergency” patients immediately in approximately half the cases. If 10% of 50 million to 100 million after-hours phone calls each year in the United States are not forwarded to the physician because the caller feels the complaint is not emergent, and if half those calls are potentially serious, there may be as many as 2.5 million to 5 million potentially dangerous delays in care each year.

We cannot expect an answering service operator or a parent to know how to triage an infant with a fever when physicians disagree on appropriate disposition.⁸ New parents with a sick infant, an older patient with chest pain, or a woman having preterm contractions during her first pregnancy might be uncertain as to what constitutes an “emergency.”

Solutions

Several solutions to this problem exist. We made a change in our office and now have all clinical calls forwarded to the on-call physician. No triage decisions are made by the patient or the answering service. This has led to an average increase of only 1 to 2 more patient calls per night. Offices also could become part of a citywide network in which all calls are managed by a trained nursing staff, as the pediatricians have done in Denver, Colorado.⁵

Interpretations

This study should be interpreted in light of several limitations. First, it was conducted in 1 metropolitan region. It is possible that other regions of the US have different mechanisms or standards for handling after-hours calls. However, given the overwhelming number of offices in our study that required patients to make their own triage decisions, we believe this barrier is likely widespread.

Second, the answering services we surveyed knew we were not patients, and this may have affected their answers. However, even if only 10% of these calls were not forwarded to the physician on call, a significant number of calls might have been unnecessarily delayed and potentially put patients at risk.

The Institute of Medicine’s report on medical errors states: “Errors can be prevented by designing systems that make it hard for people to do the wrong thing and easy for people to do the right thing.”⁹ Errors in triage by the patient or the answering service may lead to dangerous delays in necessary patient care.

Our future research will focus on identifying adverse outcomes in this study population and prospectively in a practice-based research network. When a patient calls the primary care office after hours, the decisions should be simple and left to those who have the training to make those decisions based on their best medical judgment. We strongly urge all clinicians who use an answering service to examine their policies and procedures for potential sources of medical error.

Acknowledgments

We express our thanks to Tarek Arja, DO, Dan O’Brien, DO, Mark Cucuzzella, MD, and Jacqueline Stern, MD; for agreeing to review nonemergent calls, and Pamela Sullivan for her assistance in preparing the manuscript.

When physicians’ perceptions of patient expectations were examined as a predictor of prescribing, physicians were significantly more likely to provide a prescription if pre- or postvisit expectation for antibiotics was expressed by patients, even if antibiotics were inappropriate.^1-8 Patients who expected to receive a prescription were 30% to 45% more likely to receive one than patients who did not expect to receive one. Inappropriate prescribing of antibiotics for presumed viral infections is a serious problem,^9-11 particularly in the pediatric population.^12-14

Research in the pediatric context has shown similar results. Mangione-Smith et al found that physicians’ perceptions of parental expectations for antibiotics was the only significant predictor of prescribing when a viral diagnosis was assigned.¹³ When physicians thought parents expected antibiotic treatment for their child, they prescribed it 62% of the time vs 7% when they did not think antibiotics were expected (P=.02). In addition, when physicians thought parents expected antibiotics, they were significantly more likely to make a bacterial diagnosis (70% of the time vs 31% of the time; P=.04). Parents’ reports of their expectations were not significantly related to inappropriate prescribing. In all of these studies, physicians’ perceptions were stronger predictors of prescribing behavior than were patients’ reports of their expectations.

Missing from this line of research is an answer to the question: “How do physicians come to perceive that parents expect antibiotics?” Earlier work^15-18 identified and described several communication practices used by parents during acute pediatric encounters that may be related to physicians’ perceptions of parent expectations for antibiotics. This study examined the relations between 3 parent communication behaviors and parents’ reports of their expectations for anti-biotics the relations between these communication behaviors and physicians’ reports of their perceptions of parents’ expectations for antibiotics.

Methods

One community and 1 university pediatric practice were identified for possible inclusion in the study. Parents were eligible for study participation if they spoke and read English and their children were 2 to 10 years old, were being seen for upper respiratory tract infection symptoms (cough, rhinorrhea, throat pain, ear pain, or ear tugging), had not been taking antibiotics for the previous 2 weeks, and were seeing a participating physician. Approval for all study procedures was obtained from the UCLA human subjects’ protection committee.

Inventory of parents’ expectations

Before the encounter, parents completed a 15-item previsit expectations inventory that included 1 item about “how necessary” they thought it was for the physician to “prescribe antibiotics for your child (medicine for infection).” The other items in the inventory asked about previsit expectations for other medications (eg, cough medicine) and other tasks (eg, taking the child's temperature) and are described in detail elsewhere.¹³

The inventory was scored by using a 5-point scale: 1 = definitely necessary, 2 = probably necessary, 3 = uncertain, 4 = probably unnecessary, and 5 = definitely unnecessary. Parents who reported a score of 1 or 2 were coded as expecting antibiotics, and parents who reported a score of 3, 4, or 5 were coded as not expecting antibiotics. Each encounter was then audiotaped.

Physicians’ perceptions of expectations

Physicians completed a postvisit checklist to indicate diagnosis, treatment, and their perceptions of what the parents expected. One item asked the doctor to agree or disagree with the statement: “This parent expected me to prescribe antibiotics.” Other items asked whether the physician thought that the parent expected other medications (eg, cough medicine). This measure is also described in detail elsewhere.¹³ These items were scored on a 5-point Likert scale: 1 = strongly agree, 2 = somewhat agree, 3 = uncertain, 4 = somewhat disagree, and 5 = strongly disagree. Scores of 1 and 2 were coded as the physician perceiving the parent as expecting antibiotics, and scores of 3, 4, and 5 were coded as the physician perceiving the parent as not expecting antibiotics.

Analysis of the doctor–parent interaction

Conversation analysis was used as a qualitative method for analyzing the audiotaped data.¹⁹ Conversation analysis looks for patterns in the interaction that form evidence of systematic usage such that they can be identified as “practices.” To be identified as a practice, a particular communication behavior must be recurrently used and attract responses that systematically discriminate it from similar or related practices. For example, when a physician asks, “How are you feeling?,” patients recurrently respond with information about an ongoing health condition (usually the problem they were treated for in a prior visit) even if there were new problems to report to the physician.²⁰

By relying on conversation analysis as a methodology, for these data 6 primary communication practices were found to be related to antibiotics.¹⁵ Analyses of 3 of these practices have been published elsewhere.^17,18 For the purposes of this study, 4 communication practices that seemed most robust given the relatively small sample size were identified and operationalized in a coding scheme to test the relations between these behaviors and survey-based variables. All encounters were coded by 1 coder (T.S.), and a 15% sample was recoded by the same coder for intrarater reliability. All κ values exceeded .8 reliability, indicating substantial agreement above chance.²¹ The communication behaviors that were coded are outlined in Table 1.

TABLE 1
Parent communication behaviors

Analytic methods

The survey data were merged with the coded audiotape data to examine the relations between (1) parents’ reports of their expectations for antibiotics, (2) parents’ communication practices, and (3) physicians’ perceptions of parents’ expectations for antibiotics. We tested bivariate relationships between the main outcome variables and several hypothesized predictors by using the χ² test of independence and Fisher's exact test. Variables significant at the P=.05 level were included in a multivariate logistic regression predicting physicians’ perceptions of parents’ expectations for antibiotics. Whether the diagnosis was bacterial or viral was controlled for in the model. A similar multivariate logistic regression examining the relations between parental expectations and their communication behaviors was developed. Both included separate intercepts for each physician. All tests were 2-sided and conducted at the .05 level of significance. Results were then corrected for clustering with the Huber correction.^22,23 Results of the logistic regression models are reported as odds ratios (ORs) with 95% confidence intervals (CIs).

Results

As previously reported, 8 of the 10 full-time physicians in 2 practices agreed to participate, and 306 of the 356 eligible parents agreed to participate (response rate, 86%). Eleven visits were excluded because of incomplete data. Thus, there were 295 complete encounters. Data were collected between October 1996 and March 1997. Parents in the sample were highly educated (mean years of education, 16), older (mean age, 38 years), and had high incomes (75% had household annual incomes greater than $50,000). Nonwhites comprised one third of the sample, and 60% were enrolled in managed care plans.¹³ Parents reported having an expectation for antibiotics in 49% (n=144) of cases. In contrast, physicians reported perceiving parents to expect antibiotics in 34% (n=100) of cases.

Qualitative analysis of the audiotaped data identified 4 primary communication behaviors associated with prescribing of antibiotics (see Table 1). When a parent presented the child's problem by offering a possible or “candidate” diagnosis (45% of cases), physicians responded as though the parent was seeking antibiotics as contrasted with a “symptoms only” presentation (51% of cases). The results of the qualitative analysis have been described in detail elsewhere.¹⁷ Candidate diagnoses (eg, ear infection, sinus infection, pneumonia, or strep throat) imply bacterial infections. In response physicians behave as though parents are seeking antibiotics. For example, they routinely confirm or deny the need for antibiotic treatment. Other qualitative research has associated these behaviors with inappropriate prescribing of antibiotics.²⁴

When a physician announces a diagnosis (whether framed positively as a viral condition or negatively as not a bacterial condition), parents sometimes “resist” that diagnosis. This resistance typically involves questioning the physician's physical examination findings or questioning the actual diagnosis. As with candidate diagnoses, this behavior does not explicitly mention antibiotics, but physicians routinely respond to diagnosis resistance as having communicated that the parent is seeking antibiotics by confirming or denying a need for them. This behavior was found in 17% (n=50) of cases.

In response to physicians’ nonantibiotic treatment recommendations, parents may “resist” the recommended treatment. As with the other behaviors, this resistance usually does not involve an explicit request for antibiotics, but physicians nonetheless typically respond to treatment resistance as if parents are searching for antibiotics. This behavior was found in 12% of (n=35) cases.

After the qualitative analysis of these behaviors, each audiotaped encounter was coded for their presence so that these communication variables could be merged with survey data variables for quantitative analysis. Bivariate associations between each identified communication behavior and the 2 survey variables (parents’ reports of their expectations for antibiotics and physicians’ perceptions that parents expected antibiotics) were tested. The relation between candidate diagnoses and parents’ reports of their expectations trended toward, but did not reach, significance (n=295, 2 χ²₁=3.141, P=.08), and parents who reported an expectation for antibiotics were no more likely to resist a physician's treatment recommendation (eg, for an over-the-counter or nonantibiotic remedy) than parents who did not expect antibiotics (n=295, χ²₁=0.29, P=.59). The strongest trend shown in these data was that, when parents expected antibiotics, they were more likely to resist a viral diagnosis (n=259, χ²₁=3.71, P=.59, P=.05).

Although none of the identified parental communication behaviors were significantly associated with parents’ reports of their expectations for antibiotics, there were significant associations between 2 of the 4 communication behaviors and physicians’ perceptions that parents expected antibiotics: when parents offered candidate diagnoses, physicians were significantly more likely to perceive the parents as expecting antibiotics. If a parent offered a candidate diagnosis in the problem presentation, the physician was 62% more likely to think the parent expected antibiotics (an increase from 29% to 47%; P=.04).

“Symptoms only” problem presentations were more frequent than “candidate diagnosis” presentations. However, among the candidate diagnosis presentations (n=132), 82% were for conditions that could be treated appropriately with antibiotics.

In cases in which a viral diagnosis was assigned, a physician was more likely to perceive a parent to expect an antibiotic if the parent resisted the diagnosis. When parents offered resistance to the diagnosis, physicians perceived them to expect antibiotics 20% of the time vs 7% of the time when they did not offer resistance (Fisher exact test, P=.047).

Parent resistance to nonantibiotic treatment recommendations was not associated with physicians’ perceptions of parents’ expectations for antibiotics (Fisher exact test, P=.122).

Each communication behavior was included in a multivariate logistic regression model predicting physicians’ perceptions that parents expected antibiotics. For parallelism, all were also included in a model predicting parents’ reports of their expectations for antibiotics. The type of diagnosis (ie, bacterial or viral) was also controlled for.

In the model predicting parents’ expectations, none of the communication behaviors reached significance as predictors. The results are shown in Table 2. After controlling for diagnosis and other communication behaviors, the odds that a physician would perceive a parent as expecting antibiotics were more than 5 times higher if the parent used a candidate diagnosis problem presentation. Similarly, the odds that a physician would perceive a parent as expecting antibiotics were nearly 3 times higher if the parent resisted a viral diagnosis.

The CIs for the associations of these 2 measures with physicians’ perceptions of expectations did not overlap with the corresponding CIs for parent-reported expectations, suggesting significantly stronger associations with physicians’ perceptions than with parents’ expectations. Neither treatment resistance nor resistance to a bacterial diagnosis reached significance as a predictor of physicians’ perceptions that parents expected antibiotics within the multivariate model.

TABLE 2
Multivariate logistic regression model predicting physicians’ perceptions that parents expected antibiotics and parents’ reports of their expectations*

Discussion

Prior research has suggested that parents commonly pressure physicians for antibiotics by overtly requesting antibiotics.^6,7,25,26 In this study this overt parent behavior was quite rare (for further discussion of these cases, see work by Mangione-Smith et al¹⁶ and Stivers¹⁸). This study suggests that physicians form their perceptions of parents’ expectations for antibiotics from far less direct communication behaviors such as parents’ candidate diagnoses or diagnosis resistance. Given the association between a physician's perception of a patient's or parent's expectation for antibiotics with increased rates of inappropriate antibiotic prescribing,^1-3,13 it appears that when a parent exhibits one of these behaviors, physicians may feel pressure to prescribe. Qualitative analyses of these data support this analysis.^17,24 Physicians appear to treat parents who use these communication practices as indicating an expectation and a desire for antibiotic treatment. However, parents may not always be intending to communicate pressure or even an expectation for antibiotics. This study found no association between the communication behaviors described and parents’ reports of their expectations for antibiotics.

This finding suggests 2 possible interpretations. Parents may not be accurate reporters of their expectations; they may be unwilling to admit to an expectation for antibiotics before the visit. Possibly, parents may accurately report their expectations before their medical encounters, but physicians misunderstand their behaviors as indicating such an expectation. Some parents may offer a candidate diagnosis because they feel that antibiotics are necessary; others may offer a candidate diagnosis to show competent parenting, or as a reflection of their concern that their child has a more serious illness, or of their concern that their visit may have been premature or unjustified. In the latter cases parents may be seeking reassurance from the physician, and they may not realize that they may be understood by physicians as pressuring for antibiotics.

However, as this study suggests, physicians do not differentiate between these alternative motivations and may tend to understand these behaviors as pressure to prescribe. The problem of mismatched parental expectations and physicians’ perceptions of those expectations is further exacerbated because it is rare for parents to explicitly state their desire for, or opposition to, antibiotic treatment.

Limitations

Because the data for this study were from 2 practices in the same geographic area and with a relatively homogeneous group of parents and physicians, we do not know whether the findings will generalize to other settings involving participants from more diverse backgrounds. In addition, we may have failed to detect associations that could exist between treatment resistance or diagnosis resistance and physicians’ perceptions of parents’ expectations or parent-reported expectations due to the relatively small sample size, the rarity of some of the behaviors, and the association of parental communication behaviors with one another. For these behaviors, we had 80% power to detect only true multivariate odds ratios that were relatively large.^1,11-14 Further research on these behaviors with larger sample sizes is indicated.

We may have introduced measurement error through reliance on parent and physician self-reports of 2 of the variables we studied. In relying on a single-item measurement of parents’ expectations for antibiotics, there may be some unreliability in the assessment of parents’ expectations.

Conclusions

Two parental communication behaviors in particular resulted in physicians feeling pressured to prescribe antibiotics: the use of candidate diagnoses and resistance to viral diagnoses were more strongly associated with physicians’ perceptions of parents’ expectations than with parents’ reports of their expectations. This finding indicates an incongruity between physicians’ perceptions of parents’ expectations and parents’ reports of their expectations. Future research needs to determine when physicians are accurate in perceiving antibiotic pressure, and when they should perceive other parental concerns for which reassurance would be the most desirable responsive action. Although antibiotics clearly are relevant in these pediatric encounters, physicians may be overly sensitive and thus too quick to understand certain communication behaviors as in search of antibiotics. Not only do such perceptions lead to inappropriate prescribing, but they also potentially contribute to dissatisfaction because parents who are in search of reassurance are not necessarily appeased by medication.^3,13,16,25

Further, parents who were in search of reassurance but who receive neither medication nor reassurance may be still less satisfied. This study has provided an initial step toward linking communication behaviors with survey reports of parents’ expectations and physicians’ perceptions. Future research is needed to translate these findings into communication-based interventions to decrease inappropriate prescribing. Physicians who recognize parental communication behaviors as communicating pressure for antibiotic treatment may be able to directly communicate with parents about their expectations and thus more directly assess and address parents’ expectations or desires. Future interventions should consider alternative communication practices physicians can use as resources for addressing perceived parental pressure.¹⁶

Acknowledgments

Thanks to Jeff Robinson for helpful comments on an earlier version of this manuscript.

Corresponding author
Tanya Stivers, PhD, Department of Pediatrics, University of California, Los Angeles, 12-358 Marion Davies Children's Center, 10833 LeConte Avenue, Los Angeles, CA 90095-1752. [email protected]

When physicians’ perceptions of patient expectations were examined as a predictor of prescribing, physicians were significantly more likely to provide a prescription if pre- or postvisit expectation for antibiotics was expressed by patients, even if antibiotics were inappropriate.^1-8 Patients who expected to receive a prescription were 30% to 45% more likely to receive one than patients who did not expect to receive one. Inappropriate prescribing of antibiotics for presumed viral infections is a serious problem,^9-11 particularly in the pediatric population.^12-14

Research in the pediatric context has shown similar results. Mangione-Smith et al found that physicians’ perceptions of parental expectations for antibiotics was the only significant predictor of prescribing when a viral diagnosis was assigned.¹³ When physicians thought parents expected antibiotic treatment for their child, they prescribed it 62% of the time vs 7% when they did not think antibiotics were expected (P=.02). In addition, when physicians thought parents expected antibiotics, they were significantly more likely to make a bacterial diagnosis (70% of the time vs 31% of the time; P=.04). Parents’ reports of their expectations were not significantly related to inappropriate prescribing. In all of these studies, physicians’ perceptions were stronger predictors of prescribing behavior than were patients’ reports of their expectations.

Missing from this line of research is an answer to the question: “How do physicians come to perceive that parents expect antibiotics?” Earlier work^15-18 identified and described several communication practices used by parents during acute pediatric encounters that may be related to physicians’ perceptions of parent expectations for antibiotics. This study examined the relations between 3 parent communication behaviors and parents’ reports of their expectations for anti-biotics the relations between these communication behaviors and physicians’ reports of their perceptions of parents’ expectations for antibiotics.

Methods

One community and 1 university pediatric practice were identified for possible inclusion in the study. Parents were eligible for study participation if they spoke and read English and their children were 2 to 10 years old, were being seen for upper respiratory tract infection symptoms (cough, rhinorrhea, throat pain, ear pain, or ear tugging), had not been taking antibiotics for the previous 2 weeks, and were seeing a participating physician. Approval for all study procedures was obtained from the UCLA human subjects’ protection committee.

Inventory of parents’ expectations

Before the encounter, parents completed a 15-item previsit expectations inventory that included 1 item about “how necessary” they thought it was for the physician to “prescribe antibiotics for your child (medicine for infection).” The other items in the inventory asked about previsit expectations for other medications (eg, cough medicine) and other tasks (eg, taking the child's temperature) and are described in detail elsewhere.¹³

The inventory was scored by using a 5-point scale: 1 = definitely necessary, 2 = probably necessary, 3 = uncertain, 4 = probably unnecessary, and 5 = definitely unnecessary. Parents who reported a score of 1 or 2 were coded as expecting antibiotics, and parents who reported a score of 3, 4, or 5 were coded as not expecting antibiotics. Each encounter was then audiotaped.

Physicians’ perceptions of expectations

Physicians completed a postvisit checklist to indicate diagnosis, treatment, and their perceptions of what the parents expected. One item asked the doctor to agree or disagree with the statement: “This parent expected me to prescribe antibiotics.” Other items asked whether the physician thought that the parent expected other medications (eg, cough medicine). This measure is also described in detail elsewhere.¹³ These items were scored on a 5-point Likert scale: 1 = strongly agree, 2 = somewhat agree, 3 = uncertain, 4 = somewhat disagree, and 5 = strongly disagree. Scores of 1 and 2 were coded as the physician perceiving the parent as expecting antibiotics, and scores of 3, 4, and 5 were coded as the physician perceiving the parent as not expecting antibiotics.

Analysis of the doctor–parent interaction

Conversation analysis was used as a qualitative method for analyzing the audiotaped data.¹⁹ Conversation analysis looks for patterns in the interaction that form evidence of systematic usage such that they can be identified as “practices.” To be identified as a practice, a particular communication behavior must be recurrently used and attract responses that systematically discriminate it from similar or related practices. For example, when a physician asks, “How are you feeling?,” patients recurrently respond with information about an ongoing health condition (usually the problem they were treated for in a prior visit) even if there were new problems to report to the physician.²⁰

By relying on conversation analysis as a methodology, for these data 6 primary communication practices were found to be related to antibiotics.¹⁵ Analyses of 3 of these practices have been published elsewhere.^17,18 For the purposes of this study, 4 communication practices that seemed most robust given the relatively small sample size were identified and operationalized in a coding scheme to test the relations between these behaviors and survey-based variables. All encounters were coded by 1 coder (T.S.), and a 15% sample was recoded by the same coder for intrarater reliability. All κ values exceeded .8 reliability, indicating substantial agreement above chance.²¹ The communication behaviors that were coded are outlined in Table 1.

TABLE 1
Parent communication behaviors

Analytic methods

The survey data were merged with the coded audiotape data to examine the relations between (1) parents’ reports of their expectations for antibiotics, (2) parents’ communication practices, and (3) physicians’ perceptions of parents’ expectations for antibiotics. We tested bivariate relationships between the main outcome variables and several hypothesized predictors by using the χ² test of independence and Fisher's exact test. Variables significant at the P=.05 level were included in a multivariate logistic regression predicting physicians’ perceptions of parents’ expectations for antibiotics. Whether the diagnosis was bacterial or viral was controlled for in the model. A similar multivariate logistic regression examining the relations between parental expectations and their communication behaviors was developed. Both included separate intercepts for each physician. All tests were 2-sided and conducted at the .05 level of significance. Results were then corrected for clustering with the Huber correction.^22,23 Results of the logistic regression models are reported as odds ratios (ORs) with 95% confidence intervals (CIs).

Results

As previously reported, 8 of the 10 full-time physicians in 2 practices agreed to participate, and 306 of the 356 eligible parents agreed to participate (response rate, 86%). Eleven visits were excluded because of incomplete data. Thus, there were 295 complete encounters. Data were collected between October 1996 and March 1997. Parents in the sample were highly educated (mean years of education, 16), older (mean age, 38 years), and had high incomes (75% had household annual incomes greater than $50,000). Nonwhites comprised one third of the sample, and 60% were enrolled in managed care plans.¹³ Parents reported having an expectation for antibiotics in 49% (n=144) of cases. In contrast, physicians reported perceiving parents to expect antibiotics in 34% (n=100) of cases.

Qualitative analysis of the audiotaped data identified 4 primary communication behaviors associated with prescribing of antibiotics (see Table 1). When a parent presented the child's problem by offering a possible or “candidate” diagnosis (45% of cases), physicians responded as though the parent was seeking antibiotics as contrasted with a “symptoms only” presentation (51% of cases). The results of the qualitative analysis have been described in detail elsewhere.¹⁷ Candidate diagnoses (eg, ear infection, sinus infection, pneumonia, or strep throat) imply bacterial infections. In response physicians behave as though parents are seeking antibiotics. For example, they routinely confirm or deny the need for antibiotic treatment. Other qualitative research has associated these behaviors with inappropriate prescribing of antibiotics.²⁴

When a physician announces a diagnosis (whether framed positively as a viral condition or negatively as not a bacterial condition), parents sometimes “resist” that diagnosis. This resistance typically involves questioning the physician's physical examination findings or questioning the actual diagnosis. As with candidate diagnoses, this behavior does not explicitly mention antibiotics, but physicians routinely respond to diagnosis resistance as having communicated that the parent is seeking antibiotics by confirming or denying a need for them. This behavior was found in 17% (n=50) of cases.

In response to physicians’ nonantibiotic treatment recommendations, parents may “resist” the recommended treatment. As with the other behaviors, this resistance usually does not involve an explicit request for antibiotics, but physicians nonetheless typically respond to treatment resistance as if parents are searching for antibiotics. This behavior was found in 12% of (n=35) cases.

After the qualitative analysis of these behaviors, each audiotaped encounter was coded for their presence so that these communication variables could be merged with survey data variables for quantitative analysis. Bivariate associations between each identified communication behavior and the 2 survey variables (parents’ reports of their expectations for antibiotics and physicians’ perceptions that parents expected antibiotics) were tested. The relation between candidate diagnoses and parents’ reports of their expectations trended toward, but did not reach, significance (n=295, 2 χ²₁=3.141, P=.08), and parents who reported an expectation for antibiotics were no more likely to resist a physician's treatment recommendation (eg, for an over-the-counter or nonantibiotic remedy) than parents who did not expect antibiotics (n=295, χ²₁=0.29, P=.59). The strongest trend shown in these data was that, when parents expected antibiotics, they were more likely to resist a viral diagnosis (n=259, χ²₁=3.71, P=.59, P=.05).

Although none of the identified parental communication behaviors were significantly associated with parents’ reports of their expectations for antibiotics, there were significant associations between 2 of the 4 communication behaviors and physicians’ perceptions that parents expected antibiotics: when parents offered candidate diagnoses, physicians were significantly more likely to perceive the parents as expecting antibiotics. If a parent offered a candidate diagnosis in the problem presentation, the physician was 62% more likely to think the parent expected antibiotics (an increase from 29% to 47%; P=.04).

“Symptoms only” problem presentations were more frequent than “candidate diagnosis” presentations. However, among the candidate diagnosis presentations (n=132), 82% were for conditions that could be treated appropriately with antibiotics.

In cases in which a viral diagnosis was assigned, a physician was more likely to perceive a parent to expect an antibiotic if the parent resisted the diagnosis. When parents offered resistance to the diagnosis, physicians perceived them to expect antibiotics 20% of the time vs 7% of the time when they did not offer resistance (Fisher exact test, P=.047).

Parent resistance to nonantibiotic treatment recommendations was not associated with physicians’ perceptions of parents’ expectations for antibiotics (Fisher exact test, P=.122).

Each communication behavior was included in a multivariate logistic regression model predicting physicians’ perceptions that parents expected antibiotics. For parallelism, all were also included in a model predicting parents’ reports of their expectations for antibiotics. The type of diagnosis (ie, bacterial or viral) was also controlled for.

In the model predicting parents’ expectations, none of the communication behaviors reached significance as predictors. The results are shown in Table 2. After controlling for diagnosis and other communication behaviors, the odds that a physician would perceive a parent as expecting antibiotics were more than 5 times higher if the parent used a candidate diagnosis problem presentation. Similarly, the odds that a physician would perceive a parent as expecting antibiotics were nearly 3 times higher if the parent resisted a viral diagnosis.

The CIs for the associations of these 2 measures with physicians’ perceptions of expectations did not overlap with the corresponding CIs for parent-reported expectations, suggesting significantly stronger associations with physicians’ perceptions than with parents’ expectations. Neither treatment resistance nor resistance to a bacterial diagnosis reached significance as a predictor of physicians’ perceptions that parents expected antibiotics within the multivariate model.

TABLE 2
Multivariate logistic regression model predicting physicians’ perceptions that parents expected antibiotics and parents’ reports of their expectations*

Discussion

Prior research has suggested that parents commonly pressure physicians for antibiotics by overtly requesting antibiotics.^6,7,25,26 In this study this overt parent behavior was quite rare (for further discussion of these cases, see work by Mangione-Smith et al¹⁶ and Stivers¹⁸). This study suggests that physicians form their perceptions of parents’ expectations for antibiotics from far less direct communication behaviors such as parents’ candidate diagnoses or diagnosis resistance. Given the association between a physician's perception of a patient's or parent's expectation for antibiotics with increased rates of inappropriate antibiotic prescribing,^1-3,13 it appears that when a parent exhibits one of these behaviors, physicians may feel pressure to prescribe. Qualitative analyses of these data support this analysis.^17,24 Physicians appear to treat parents who use these communication practices as indicating an expectation and a desire for antibiotic treatment. However, parents may not always be intending to communicate pressure or even an expectation for antibiotics. This study found no association between the communication behaviors described and parents’ reports of their expectations for antibiotics.

This finding suggests 2 possible interpretations. Parents may not be accurate reporters of their expectations; they may be unwilling to admit to an expectation for antibiotics before the visit. Possibly, parents may accurately report their expectations before their medical encounters, but physicians misunderstand their behaviors as indicating such an expectation. Some parents may offer a candidate diagnosis because they feel that antibiotics are necessary; others may offer a candidate diagnosis to show competent parenting, or as a reflection of their concern that their child has a more serious illness, or of their concern that their visit may have been premature or unjustified. In the latter cases parents may be seeking reassurance from the physician, and they may not realize that they may be understood by physicians as pressuring for antibiotics.

However, as this study suggests, physicians do not differentiate between these alternative motivations and may tend to understand these behaviors as pressure to prescribe. The problem of mismatched parental expectations and physicians’ perceptions of those expectations is further exacerbated because it is rare for parents to explicitly state their desire for, or opposition to, antibiotic treatment.

Limitations

Because the data for this study were from 2 practices in the same geographic area and with a relatively homogeneous group of parents and physicians, we do not know whether the findings will generalize to other settings involving participants from more diverse backgrounds. In addition, we may have failed to detect associations that could exist between treatment resistance or diagnosis resistance and physicians’ perceptions of parents’ expectations or parent-reported expectations due to the relatively small sample size, the rarity of some of the behaviors, and the association of parental communication behaviors with one another. For these behaviors, we had 80% power to detect only true multivariate odds ratios that were relatively large.^1,11-14 Further research on these behaviors with larger sample sizes is indicated.

We may have introduced measurement error through reliance on parent and physician self-reports of 2 of the variables we studied. In relying on a single-item measurement of parents’ expectations for antibiotics, there may be some unreliability in the assessment of parents’ expectations.

Conclusions

Two parental communication behaviors in particular resulted in physicians feeling pressured to prescribe antibiotics: the use of candidate diagnoses and resistance to viral diagnoses were more strongly associated with physicians’ perceptions of parents’ expectations than with parents’ reports of their expectations. This finding indicates an incongruity between physicians’ perceptions of parents’ expectations and parents’ reports of their expectations. Future research needs to determine when physicians are accurate in perceiving antibiotic pressure, and when they should perceive other parental concerns for which reassurance would be the most desirable responsive action. Although antibiotics clearly are relevant in these pediatric encounters, physicians may be overly sensitive and thus too quick to understand certain communication behaviors as in search of antibiotics. Not only do such perceptions lead to inappropriate prescribing, but they also potentially contribute to dissatisfaction because parents who are in search of reassurance are not necessarily appeased by medication.^3,13,16,25

Further, parents who were in search of reassurance but who receive neither medication nor reassurance may be still less satisfied. This study has provided an initial step toward linking communication behaviors with survey reports of parents’ expectations and physicians’ perceptions. Future research is needed to translate these findings into communication-based interventions to decrease inappropriate prescribing. Physicians who recognize parental communication behaviors as communicating pressure for antibiotic treatment may be able to directly communicate with parents about their expectations and thus more directly assess and address parents’ expectations or desires. Future interventions should consider alternative communication practices physicians can use as resources for addressing perceived parental pressure.¹⁶

Acknowledgments

Thanks to Jeff Robinson for helpful comments on an earlier version of this manuscript.

Corresponding author
Tanya Stivers, PhD, Department of Pediatrics, University of California, Los Angeles, 12-358 Marion Davies Children's Center, 10833 LeConte Avenue, Los Angeles, CA 90095-1752. [email protected]

When physicians’ perceptions of patient expectations were examined as a predictor of prescribing, physicians were significantly more likely to provide a prescription if pre- or postvisit expectation for antibiotics was expressed by patients, even if antibiotics were inappropriate.^1-8 Patients who expected to receive a prescription were 30% to 45% more likely to receive one than patients who did not expect to receive one. Inappropriate prescribing of antibiotics for presumed viral infections is a serious problem,^9-11 particularly in the pediatric population.^12-14

Research in the pediatric context has shown similar results. Mangione-Smith et al found that physicians’ perceptions of parental expectations for antibiotics was the only significant predictor of prescribing when a viral diagnosis was assigned.¹³ When physicians thought parents expected antibiotic treatment for their child, they prescribed it 62% of the time vs 7% when they did not think antibiotics were expected (P=.02). In addition, when physicians thought parents expected antibiotics, they were significantly more likely to make a bacterial diagnosis (70% of the time vs 31% of the time; P=.04). Parents’ reports of their expectations were not significantly related to inappropriate prescribing. In all of these studies, physicians’ perceptions were stronger predictors of prescribing behavior than were patients’ reports of their expectations.

Missing from this line of research is an answer to the question: “How do physicians come to perceive that parents expect antibiotics?” Earlier work^15-18 identified and described several communication practices used by parents during acute pediatric encounters that may be related to physicians’ perceptions of parent expectations for antibiotics. This study examined the relations between 3 parent communication behaviors and parents’ reports of their expectations for anti-biotics the relations between these communication behaviors and physicians’ reports of their perceptions of parents’ expectations for antibiotics.

Methods

One community and 1 university pediatric practice were identified for possible inclusion in the study. Parents were eligible for study participation if they spoke and read English and their children were 2 to 10 years old, were being seen for upper respiratory tract infection symptoms (cough, rhinorrhea, throat pain, ear pain, or ear tugging), had not been taking antibiotics for the previous 2 weeks, and were seeing a participating physician. Approval for all study procedures was obtained from the UCLA human subjects’ protection committee.

Inventory of parents’ expectations

Before the encounter, parents completed a 15-item previsit expectations inventory that included 1 item about “how necessary” they thought it was for the physician to “prescribe antibiotics for your child (medicine for infection).” The other items in the inventory asked about previsit expectations for other medications (eg, cough medicine) and other tasks (eg, taking the child's temperature) and are described in detail elsewhere.¹³

The inventory was scored by using a 5-point scale: 1 = definitely necessary, 2 = probably necessary, 3 = uncertain, 4 = probably unnecessary, and 5 = definitely unnecessary. Parents who reported a score of 1 or 2 were coded as expecting antibiotics, and parents who reported a score of 3, 4, or 5 were coded as not expecting antibiotics. Each encounter was then audiotaped.

Physicians’ perceptions of expectations

Physicians completed a postvisit checklist to indicate diagnosis, treatment, and their perceptions of what the parents expected. One item asked the doctor to agree or disagree with the statement: “This parent expected me to prescribe antibiotics.” Other items asked whether the physician thought that the parent expected other medications (eg, cough medicine). This measure is also described in detail elsewhere.¹³ These items were scored on a 5-point Likert scale: 1 = strongly agree, 2 = somewhat agree, 3 = uncertain, 4 = somewhat disagree, and 5 = strongly disagree. Scores of 1 and 2 were coded as the physician perceiving the parent as expecting antibiotics, and scores of 3, 4, and 5 were coded as the physician perceiving the parent as not expecting antibiotics.

Analysis of the doctor–parent interaction

Conversation analysis was used as a qualitative method for analyzing the audiotaped data.¹⁹ Conversation analysis looks for patterns in the interaction that form evidence of systematic usage such that they can be identified as “practices.” To be identified as a practice, a particular communication behavior must be recurrently used and attract responses that systematically discriminate it from similar or related practices. For example, when a physician asks, “How are you feeling?,” patients recurrently respond with information about an ongoing health condition (usually the problem they were treated for in a prior visit) even if there were new problems to report to the physician.²⁰

By relying on conversation analysis as a methodology, for these data 6 primary communication practices were found to be related to antibiotics.¹⁵ Analyses of 3 of these practices have been published elsewhere.^17,18 For the purposes of this study, 4 communication practices that seemed most robust given the relatively small sample size were identified and operationalized in a coding scheme to test the relations between these behaviors and survey-based variables. All encounters were coded by 1 coder (T.S.), and a 15% sample was recoded by the same coder for intrarater reliability. All κ values exceeded .8 reliability, indicating substantial agreement above chance.²¹ The communication behaviors that were coded are outlined in Table 1.

TABLE 1
Parent communication behaviors

Analytic methods

The survey data were merged with the coded audiotape data to examine the relations between (1) parents’ reports of their expectations for antibiotics, (2) parents’ communication practices, and (3) physicians’ perceptions of parents’ expectations for antibiotics. We tested bivariate relationships between the main outcome variables and several hypothesized predictors by using the χ² test of independence and Fisher's exact test. Variables significant at the P=.05 level were included in a multivariate logistic regression predicting physicians’ perceptions of parents’ expectations for antibiotics. Whether the diagnosis was bacterial or viral was controlled for in the model. A similar multivariate logistic regression examining the relations between parental expectations and their communication behaviors was developed. Both included separate intercepts for each physician. All tests were 2-sided and conducted at the .05 level of significance. Results were then corrected for clustering with the Huber correction.^22,23 Results of the logistic regression models are reported as odds ratios (ORs) with 95% confidence intervals (CIs).

Results

As previously reported, 8 of the 10 full-time physicians in 2 practices agreed to participate, and 306 of the 356 eligible parents agreed to participate (response rate, 86%). Eleven visits were excluded because of incomplete data. Thus, there were 295 complete encounters. Data were collected between October 1996 and March 1997. Parents in the sample were highly educated (mean years of education, 16), older (mean age, 38 years), and had high incomes (75% had household annual incomes greater than $50,000). Nonwhites comprised one third of the sample, and 60% were enrolled in managed care plans.¹³ Parents reported having an expectation for antibiotics in 49% (n=144) of cases. In contrast, physicians reported perceiving parents to expect antibiotics in 34% (n=100) of cases.

Qualitative analysis of the audiotaped data identified 4 primary communication behaviors associated with prescribing of antibiotics (see Table 1). When a parent presented the child's problem by offering a possible or “candidate” diagnosis (45% of cases), physicians responded as though the parent was seeking antibiotics as contrasted with a “symptoms only” presentation (51% of cases). The results of the qualitative analysis have been described in detail elsewhere.¹⁷ Candidate diagnoses (eg, ear infection, sinus infection, pneumonia, or strep throat) imply bacterial infections. In response physicians behave as though parents are seeking antibiotics. For example, they routinely confirm or deny the need for antibiotic treatment. Other qualitative research has associated these behaviors with inappropriate prescribing of antibiotics.²⁴

When a physician announces a diagnosis (whether framed positively as a viral condition or negatively as not a bacterial condition), parents sometimes “resist” that diagnosis. This resistance typically involves questioning the physician's physical examination findings or questioning the actual diagnosis. As with candidate diagnoses, this behavior does not explicitly mention antibiotics, but physicians routinely respond to diagnosis resistance as having communicated that the parent is seeking antibiotics by confirming or denying a need for them. This behavior was found in 17% (n=50) of cases.

In response to physicians’ nonantibiotic treatment recommendations, parents may “resist” the recommended treatment. As with the other behaviors, this resistance usually does not involve an explicit request for antibiotics, but physicians nonetheless typically respond to treatment resistance as if parents are searching for antibiotics. This behavior was found in 12% of (n=35) cases.

After the qualitative analysis of these behaviors, each audiotaped encounter was coded for their presence so that these communication variables could be merged with survey data variables for quantitative analysis. Bivariate associations between each identified communication behavior and the 2 survey variables (parents’ reports of their expectations for antibiotics and physicians’ perceptions that parents expected antibiotics) were tested. The relation between candidate diagnoses and parents’ reports of their expectations trended toward, but did not reach, significance (n=295, 2 χ²₁=3.141, P=.08), and parents who reported an expectation for antibiotics were no more likely to resist a physician's treatment recommendation (eg, for an over-the-counter or nonantibiotic remedy) than parents who did not expect antibiotics (n=295, χ²₁=0.29, P=.59). The strongest trend shown in these data was that, when parents expected antibiotics, they were more likely to resist a viral diagnosis (n=259, χ²₁=3.71, P=.59, P=.05).

Although none of the identified parental communication behaviors were significantly associated with parents’ reports of their expectations for antibiotics, there were significant associations between 2 of the 4 communication behaviors and physicians’ perceptions that parents expected antibiotics: when parents offered candidate diagnoses, physicians were significantly more likely to perceive the parents as expecting antibiotics. If a parent offered a candidate diagnosis in the problem presentation, the physician was 62% more likely to think the parent expected antibiotics (an increase from 29% to 47%; P=.04).

“Symptoms only” problem presentations were more frequent than “candidate diagnosis” presentations. However, among the candidate diagnosis presentations (n=132), 82% were for conditions that could be treated appropriately with antibiotics.

In cases in which a viral diagnosis was assigned, a physician was more likely to perceive a parent to expect an antibiotic if the parent resisted the diagnosis. When parents offered resistance to the diagnosis, physicians perceived them to expect antibiotics 20% of the time vs 7% of the time when they did not offer resistance (Fisher exact test, P=.047).

Parent resistance to nonantibiotic treatment recommendations was not associated with physicians’ perceptions of parents’ expectations for antibiotics (Fisher exact test, P=.122).

Each communication behavior was included in a multivariate logistic regression model predicting physicians’ perceptions that parents expected antibiotics. For parallelism, all were also included in a model predicting parents’ reports of their expectations for antibiotics. The type of diagnosis (ie, bacterial or viral) was also controlled for.

In the model predicting parents’ expectations, none of the communication behaviors reached significance as predictors. The results are shown in Table 2. After controlling for diagnosis and other communication behaviors, the odds that a physician would perceive a parent as expecting antibiotics were more than 5 times higher if the parent used a candidate diagnosis problem presentation. Similarly, the odds that a physician would perceive a parent as expecting antibiotics were nearly 3 times higher if the parent resisted a viral diagnosis.

The CIs for the associations of these 2 measures with physicians’ perceptions of expectations did not overlap with the corresponding CIs for parent-reported expectations, suggesting significantly stronger associations with physicians’ perceptions than with parents’ expectations. Neither treatment resistance nor resistance to a bacterial diagnosis reached significance as a predictor of physicians’ perceptions that parents expected antibiotics within the multivariate model.

TABLE 2
Multivariate logistic regression model predicting physicians’ perceptions that parents expected antibiotics and parents’ reports of their expectations*

Discussion

Prior research has suggested that parents commonly pressure physicians for antibiotics by overtly requesting antibiotics.^6,7,25,26 In this study this overt parent behavior was quite rare (for further discussion of these cases, see work by Mangione-Smith et al¹⁶ and Stivers¹⁸). This study suggests that physicians form their perceptions of parents’ expectations for antibiotics from far less direct communication behaviors such as parents’ candidate diagnoses or diagnosis resistance. Given the association between a physician's perception of a patient's or parent's expectation for antibiotics with increased rates of inappropriate antibiotic prescribing,^1-3,13 it appears that when a parent exhibits one of these behaviors, physicians may feel pressure to prescribe. Qualitative analyses of these data support this analysis.^17,24 Physicians appear to treat parents who use these communication practices as indicating an expectation and a desire for antibiotic treatment. However, parents may not always be intending to communicate pressure or even an expectation for antibiotics. This study found no association between the communication behaviors described and parents’ reports of their expectations for antibiotics.

This finding suggests 2 possible interpretations. Parents may not be accurate reporters of their expectations; they may be unwilling to admit to an expectation for antibiotics before the visit. Possibly, parents may accurately report their expectations before their medical encounters, but physicians misunderstand their behaviors as indicating such an expectation. Some parents may offer a candidate diagnosis because they feel that antibiotics are necessary; others may offer a candidate diagnosis to show competent parenting, or as a reflection of their concern that their child has a more serious illness, or of their concern that their visit may have been premature or unjustified. In the latter cases parents may be seeking reassurance from the physician, and they may not realize that they may be understood by physicians as pressuring for antibiotics.

However, as this study suggests, physicians do not differentiate between these alternative motivations and may tend to understand these behaviors as pressure to prescribe. The problem of mismatched parental expectations and physicians’ perceptions of those expectations is further exacerbated because it is rare for parents to explicitly state their desire for, or opposition to, antibiotic treatment.

Limitations

Because the data for this study were from 2 practices in the same geographic area and with a relatively homogeneous group of parents and physicians, we do not know whether the findings will generalize to other settings involving participants from more diverse backgrounds. In addition, we may have failed to detect associations that could exist between treatment resistance or diagnosis resistance and physicians’ perceptions of parents’ expectations or parent-reported expectations due to the relatively small sample size, the rarity of some of the behaviors, and the association of parental communication behaviors with one another. For these behaviors, we had 80% power to detect only true multivariate odds ratios that were relatively large.^1,11-14 Further research on these behaviors with larger sample sizes is indicated.

We may have introduced measurement error through reliance on parent and physician self-reports of 2 of the variables we studied. In relying on a single-item measurement of parents’ expectations for antibiotics, there may be some unreliability in the assessment of parents’ expectations.

Conclusions

Two parental communication behaviors in particular resulted in physicians feeling pressured to prescribe antibiotics: the use of candidate diagnoses and resistance to viral diagnoses were more strongly associated with physicians’ perceptions of parents’ expectations than with parents’ reports of their expectations. This finding indicates an incongruity between physicians’ perceptions of parents’ expectations and parents’ reports of their expectations. Future research needs to determine when physicians are accurate in perceiving antibiotic pressure, and when they should perceive other parental concerns for which reassurance would be the most desirable responsive action. Although antibiotics clearly are relevant in these pediatric encounters, physicians may be overly sensitive and thus too quick to understand certain communication behaviors as in search of antibiotics. Not only do such perceptions lead to inappropriate prescribing, but they also potentially contribute to dissatisfaction because parents who are in search of reassurance are not necessarily appeased by medication.^3,13,16,25

Further, parents who were in search of reassurance but who receive neither medication nor reassurance may be still less satisfied. This study has provided an initial step toward linking communication behaviors with survey reports of parents’ expectations and physicians’ perceptions. Future research is needed to translate these findings into communication-based interventions to decrease inappropriate prescribing. Physicians who recognize parental communication behaviors as communicating pressure for antibiotic treatment may be able to directly communicate with parents about their expectations and thus more directly assess and address parents’ expectations or desires. Future interventions should consider alternative communication practices physicians can use as resources for addressing perceived parental pressure.¹⁶

Acknowledgments

Thanks to Jeff Robinson for helpful comments on an earlier version of this manuscript.

Corresponding author
Tanya Stivers, PhD, Department of Pediatrics, University of California, Los Angeles, 12-358 Marion Davies Children's Center, 10833 LeConte Avenue, Los Angeles, CA 90095-1752. [email protected]

Handicapping hearing loss is one of the most common health problems of older people. Because hearing loss leads to social isolation, depression, and withdrawal from life activities,¹ screening for hearing loss should be included in the health assessment of older people. Although primary care physicians endorse the desirability of screening for hearing loss, screening methods vary widely in strategy, technique, application, and effectiveness.² Since improved methods for remediation of hearing loss have evolved over the past decade, renewed efforts for detecting and referring people with possible handicapping hearing loss are appropriate.

The gold standard for the clinical evaluation of people reporting hearing loss is a formal audiogram. However, obtaining audiometry is difficult in many locales because of problems with access, referral, and reimbursement. Therefore, many practices rely on self-administered questionnaires to screen for hearing loss.

In 1982, Ventry and Weinstein³ introduced the 25-item Hearing Handicap Inventory for the Elderly (HHIE), which was designed to assess the self-perceived psychosocial handicap of hearing impairment in the elderly as a supplement to pure tone audiometry in the evaluation of hearing aid effectiveness (Appendix.) A shorter 10-item version of the HHIE, the Hearing Handicap Inventory for the Elderly–Screening (HHIE-S), was introduced in 1986 as a screening instrument for handicapping hearing loss and is widely used.²

The reliability and validity of the HHIE-S has been established.^4,5 However, the HHIE was not developed as a screening instrument but as a method to assess the effectiveness of amplification; the subset of 10 HHIE items was extracted later for use as a screening instrument. Even shorter questionnaires and questions^6,7 have been shown to be valid and effective in hearing screening.

The purpose of this report was to determine whether the single question might be as effective and efficient a method as the formal questionnaire to screen for handicapping hearing loss. We describe the associations among the global hearing history question, the HHIE-S results, and formal hearing testing in 546 people (mean age ± SD, 78.3 ± 4.1 years) from a population-based cohort of elderly subjects (Framingham Heart Study Cohort).

Methods

The data for this report were derived from our ongoing hearing study of the Framingham Heart Study cohort. The Framingham Heart Study members comprise a population-based cohort that has been studied biennially since the first cycle from 1948 to 1950.⁸ The cohort has a substantial history of environmental noise exposure and noise-induced hearing loss.⁹ Hearing tests were offered to all members of the cohort at biennial examinations (E) E15,⁹ E18,¹⁰ and E22 (from 1983 to 1985).

Subjects in this study had a hearing test at E22 and completed the HHIE. Of the 927 people who were willing and able to take part in the E22 health examination, 723 volunteered to have a pure tone audiogram and all were asked to take the 25-item HHIE. The HHIE was completed by 672 subjects before the hearing testing, and the answers were reviewed by the audiologist for completeness. The global question was asked separately on an otologic history intake form, which also inquired about hearing aid use at the time of hearing testing. There was no provision for family members’ opinions about the subject’s hearing status. Of the 723 participants, 51 did not take the questionnaire. Reasons for noncompliance varied and included time constraints, fatigue, and malaise.

Of the 672 individuals who took the HHIE, the results from 126 were excluded because of known hearing loss for which hearing aids had been previously fitted. Of the remaining 546 participants, 502 completed all items, 29 had 1 to 4 missing items, and 15 individuals had 9 or more missing items. The number of responses per item of the HHIE ranged from to 527 to 546. The HHIE items probe the functional (social) and emotional difficulties experienced by people with hearing loss. The responses are scored 0 for a no response, 2 for a sometimes response, and 4 for a yes response. The score is the sum of all responses. Ten items from the HHIE are also used as the short or screening version (HHIES). We used these 10 items for this report.

The global history measure—the answer to the question, “Do you have a hearing problem now?”— was used as the subjective criterion of hearing loss.

The criterion handicapping hearing level used was recommended by Ventry and Weinstein,¹¹ namely an audiometric screening threshold level of 40 dB HL or greater at 1 and 2 kHz in one ear or at 1 or 2 kHz in both ears.

The HHIE-S scores were converted to a bivariate categorical variable by using the cutoff scores of 0 to 8 vs 10 and higher¹²; the sensitivity, specificity, and predictive values for a handicapping hearing loss were computed and compared with the same indicators for the global question. Exploratory models were developed to combine both screening measures. Statistical tests were performed with STATA 6.0 by using Spearman rank correlation for the categorical variables, the χ² test for proportions, and the t test for continuous variables.

Results