Cleveland Clinic Journal of Medicine

The amount of information generated in medical research is becoming overwhelming, even for experienced researchers. New studies are constantly being published, and clinicians are finding it nearly impossible to stay current, even in their own area of specialty.

To help make sense of the information, we are seeing more and more review articles that pool the results of multiple studies. When certain principles are followed and the data are quantitatively analyzed, these reviews are called meta-analyses. A PubMed search of the word “meta-analysis” in the title yielded 1,473 articles in the year 2007.

Combining available information to generate an integrated result seems reasonable and can save a considerable amount of resources. Nowadays, meta-analyses are being used to design future research, to provide evidence in the regulatory process,¹ and even to modify clinical practice.

Meta-analysis is powerful but also controversial—controversial because several conditions are critical to a sound meta-analysis, and small violations of those conditions can lead to misleading results. Summarizing large amounts of varied information using a single number is another controversial aspect of meta-analysis. Under scrutiny, some meta-analyses have been inappropriate, and their conclusions not fully warranted.^2,3

This article introduces the basic concepts of meta-analysis and discusses its caveats, with the aim of helping clinicians assess the merits of the results. We will use several recent meta-analyses to illustrate the issues, including a controversial one⁴ with potentially far-reaching consequences.

OBJECTIVES OF META-ANALYSIS

The main objectives of a meta-analysis are to:

• Summarize and integrate results from a number of individual studies
• Analyze differences in the results among studies
• Overcome small sample sizes of individual studies to detect effects of interest, and analyze end points that require larger sample sizes
• Increase precision in estimating effects
• Evaluate effects in subsets of patients
• Determine if new studies are needed to further investigate an issue
• Generate new hypotheses for future studies.

These lofty objectives can only be achieved when the meta-analysis satisfactorily addresses certain critical issues, which we will discuss next.

CRITICAL ISSUES IN META-ANALYSIS DESIGN

Four critical issues need to be addressed in a meta-analysis:

• Identification and selection of studies
• Heterogeneity of results
• Availability of information
• Analysis of the data.

IDENTIFICATION AND SELECTION OF STUDIES

The outcome of a meta-analysis depends on the studies included. The critical aspect of selecting studies to be included in a meta-analysis consists of two phases. The first is the identification phase or literature search, in which potential studies are identified. In the second phase, further criteria are used to create a list of studies for inclusion. Three insidious problems plague this aspect of meta-analysis: publication bias and search bias in the identification phase, and selection bias in the selection phase. These biases are discussed below.

Publication bias: ‘Positive’ studies are more likely to be printed

Searches of databases such as PubMed or Embase can yield long lists of studies. However, these databases include only studies that have been published. Such searches are unlikely to yield a representative sample because studies that show a “positive” result (usually in favor of a new treatment or against a well-established one) are more likely to be published than those that do not. This selective publication of studies is called publication bias.

In a recent article, Turner et al⁵ analyzed the publication status of studies of antidepressants. Based on studies registered with the US Food and Drug Administration (FDA), they found that 97% of the positive studies were published vs only 12% of the negative ones. Furthermore, when the nonpublished studies were not included in the analysis, the positive effects of individual drugs increased between 11% and 69%.

One reason for publication bias is that drug manufacturers are not generally interested in publishing negative studies. Another may be that editors favor positive studies because these are the ones that make the headlines and give the publication visibility. In some medical areas, the exclusion of studies conducted in non-English-speaking countries can increase publication bias.⁶

To ameliorate the effect of publication bias on the results of a meta-analysis, a serious effort should be made to identify unpublished studies. Identifying unpublished studies is easier now, thanks to improved communication between researchers worldwide, and thanks to registries in which all the studies of a certain disease or treatment are reported regardless of the result.

The National Institutes of Health maintains a registry of all the studies it supports, and the FDA keeps a registry and database in which drug companies must register all trials they intend to use in applying for marketing approval or a change in labeling. “Banks” of published and unpublished trials supported by pharmaceutical companies are also available (eg, http://ctr.gsk.co.uk/welcome.asp). The Cochrane collaboration (www.cochrane.org/) keeps records of systematic reviews and meta-analyses of many diseases and procedures.

Even in the ideal case that all relevant studies were available (ie, no publication bias), a faulty search can miss some of them. In searching databases, much care should be taken to assure that the set of key words used for searching is as complete as possible. This step is so critical that most recent meta-analyses include the list of key words used. The search engine (eg, PubMed, Google) is also critical, affecting the type and number of studies that are found.⁷ Small differences in search strategies can produce large differences in the set of studies found.⁸

Selection bias: Choosing the studies to be included

The identification phase usually yields a long list of potential studies, many of which are not directly relevant to the topic of the meta-analysis. This list is then subject to additional criteria to select the studies to be included. This critical step is also designed to reduce differences among studies, eliminate replication of data or studies, and improve data quality, and thus enhance the validity of the results.

To reduce the possibility of selection bias in this phase, it is crucial for the criteria to be clearly defined and for the studies to be scored by more than one researcher, with the final list chosen by consensus.^9,10 Frequently used criteria in this phase are in the areas of:

• Objectives
• Populations studied
• Study design (eg, experimental vs observational)
• Sample size
• Treatment (eg, type and dosage)
• Criteria for selection of controls
• Outcomes measured
• Quality of the data
• Analysis and reporting of results
• Accounting and reporting of attrition rates
• Length of follow-up
• When the study was conducted.

The objective in this phase is to select studies that are as similar as possible with respect to these criteria. It is a fact that even with careful selection, differences among studies will remain. But when the dissimilarities are large it becomes hard to justify pooling the results to obtain a “unified” conclusion.

In some cases, it is particularly difficult to find similar studies,^10,11 and sometimes the discrepancies and low quality of the studies can prevent a reasonable integration of results. In a systematic review of advanced lung cancer, Nicolucci et al¹² decided not to pool the results, in view of “systematic qualitative inadequacy of almost all trials” and lack of consistency in the studies and their methods. Marsoni et al¹³ came to a similar conclusion in attempting to summarize results in advanced ovarian cancer.

Stratification is an effective way to deal with inherent differences among studies and to improve the quality and usefulness of the conclusions. An added advantage to stratification is that insight can be gained by investigating discrepancies among strata.

There are many ways to create coherent subgroups of studies. For example, studies can be stratified according to their “quality,” assigned by certain scoring systems. Commonly used systems award points on the basis of how patients were selected and randomized, the type of blinding, the dropout rate, the outcome measurement, and the type of analysis (eg, intention-to-treat). However, these criteria, and therefore the scores, are somewhat subjective. Moher et al¹⁴ expand on this issue.

Large differences in sample sizes among studies are not uncommon and can cause problems in the analysis. Depending on the type of model used (see below), meta-analyses combine results based on the size of each study, but when the studies vary significantly in size, the large studies can still have an unduly large influence on the results. Stratifying by sample size is done sometimes to verify the stability of the results.⁴

On the other hand, the presence of dissimilarities among studies can have advantages by increasing the generalizability of the conclusions. Berlin and Colditz¹ point out that “we gain strength in inference when the range of patient characteristics has been broadened by replicating findings in studies with populations that vary in age range, geographic region, severity of underlying illness, and the like.”

Funnel plot: Detecting biases in the identification and selection of studies

The funnel plot is a technique used to investigate the possibility of biases in the identification and selection phases. In a funnel plot the size of the effect (defined as a measure of the difference between treatment and control) in each study is plotted on the horizontal axis against standard error¹⁵ or sample size⁹ on the vertical axis. If there are no biases, the graph will tend to have a symmetrical funnel shape centered in the average effect of the studies. When negative studies are missing, the graph shows lack of symmetry.

Funnel plots are appealing because they are simple, but their objective is to detect a complex effect, and they can be misleading. For example, lack of symmetry in a funnel plot can also be caused by heterogeneity in the studies.¹⁶ Another problem with funnel plots is that they are difficult to interpret when the number of studies is small. In some cases, however, the researcher may not have any option but to perform the analysis and report the presence of bias.¹¹

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

HETEROGENEITY OF RESULTS

In meta-analysis, heterogeneity refers to the degree of dissimilarity in the results of individual studies. In some cases, the dissimilarities in results can be traced back to inherent differences in the individual studies. In other situations, however, causes for the dissimilarities might not be easy to elucidate. In any case, as the level of heterogeneity increases, the justification for an integrated result becomes more difficult. A tool that is very effective to display the level of heterogeneity is the forest plot. In a forest plot, the estimated effect of each study along with a line representing a confidence interval is drawn. When the effects are similar, the confidence intervals overlap, and heterogeneity is low. The forest plot includes a reference line at the point of no effect (eg, one for relative risks and odds ratios). When some effects lie on opposite sides of the reference line, it means that the studies are contradictory and heterogeneity is high. In such cases, the conclusions of a meta-analysis are compromised.

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

AVAILABILITY OF INFORMATION

Most reports of individual studies include only summary results, such as means, standard deviations, proportions, odds ratios, and relative risks. Other than the possibility of errors in reporting, the lack of information can severely limit the type of analyses and conclusions that can be reached in a meta-analysis. For example, lack of information from individual studies can preclude the comparison of effects in predetermined subgroups of patients.

The best scenario is when data at the patient level are available. In such cases, the researcher has great flexibility in the analysis of the information. Trivella et al²⁰ performed a meta-analysis of the value of microvessel density in predicting survival in non-small-cell lung cancer. They obtained information on individual patients by contacting research centers directly. The data allowed them to vary the cutoff point to classify microvessel density as high or low and to use statistical methods to ameliorate heterogeneity.

A frequent problem in meta-analysis is the lack of uniformity in how outcomes are measured. In the study by Trivella et al,²⁰ the microvessel density was measured by two methods. The microvessel density was a significant prognostic factor when measured by one of the methods, but not the other.

RANDOMIZED CONTROLLED TRIALS VS OBSERVATIONAL STUDIES

Some researchers believe that meta-analyses should be conducted only on randomized controlled trials.^3,21 Their reasoning is that meta-analyses should include only reasonably well-conducted studies to reduce the risk of a misleading conclusion. However, many important diseases can only be studied observationally. If these studies have a certain level of quality, there is no technical reason not to include them in a meta-analysis.

Gillum et al²² performed a meta-analysis published in 2000 on the risk of ischemic stroke in users of oral contraceptives, based on observational studies (since no randomized trials were available). Studies were identified and selected by multiple researchers using strict criteria to make sure that only studies fulfilling certain standards were included. Of 804 potentially relevant studies identified, only 16 were included in the final analysis. A funnel plot showed no evidence of bias and the level of heterogeneity was fairly low. The meta-analysis result confirmed the results of individual studies, but the precision with which the effect was estimated was much higher. The overall relative risk of stroke in women taking oral contraceptives was 2.75, with a 95% confidence interval of 2.24 to 3.38.

A more recent meta-analysis²³ (published in 2004) on the same issue found no significant increase in the risk of ischemic stroke with the use of oral contraceptives. Gillum and Johnston²⁴ suggest that the main reason for the discrepancy is the lower amount of estrogen in newer oral contraceptives. They also point out differences in the control groups and study outcomes as reasons for the discrepancies between the two studies.

Bhutta et al²⁵ performed a meta-analysis of case-control (observational) studies of the effect of preterm birth on cognitive and behavioral outcomes. Studies were included only if the children were evaluated after their fifth birthday and the attrition rate was less than 30%. The studies were grouped according to criteria of quality devised specifically for case-control studies. The high-quality studies tended to show a larger effect than the low-quality studies, but the difference was not significant. Seventeen studies were included, and all of them found that children born preterm had lower cognitive scores; the difference was statistically significant in 15 of the studies. As expected, the meta-analysis confirmed these findings (95% confidence interval for the difference 9.2–12.5). The number of patients (1,556 cases and 1,720 controls) in the meta-analysis allowed the researchers to conclude further that the mean cognitive scores were directly proportional to the mean birth weight (R² = 0.51, P < .001) and gestational age (R² = 0.49, P < .001).

ANALYSIS OF DATA

There are specific statistical techniques that are used in meta-analysis to analyze and integrate the information. The data from the individual studies can be analyzed using either of two models: fixed effects or random effects.

The fixed-effects model assumes that the treatment effect is the same across studies. This common effect is unknown, and the purpose of the analysis is to estimate it with more precision than in the individual studies.

The random-effects model, on the other hand, assumes that the treatment effect is not the same across studies. The goal is to estimate the average effect in the studies.

In the fixed-effects model, the results of individual studies are pooled using weights that depend on the sample size of the study, whereas in the random-effects model each study is weighted equally. Due to the heterogeneity among studies, the random-effects model yields wider confidence intervals.

Both models have pros and cons. In many cases, the assumption that the treatment effect is the same in all the studies is not tenable, and the random-effects model is preferable. When the effect of interest is large, the results of both models tend to agree, particularly when the studies are balanced (ie, they have a similar number of patients in the treatment group as in the control group) and the study sizes are similar. But when the effect is small or when the level of heterogeneity of the studies is high, the result of the meta-analysis is likely to depend on the model used. In those cases, the analysis should be done and presented using both models.

It is highly desirable for a meta-analysis to include a sensitivity analysis to determine the “robustness” of the results. Two common ways to perform sensitivity analysis are to analyze the data using various methods and to present the results when some studies are removed from the analysis.²⁶ If these actions cause serious changes in the overall results, the credibility of the results is compromised.

The strength of meta-analysis is that, by pooling many studies, the effective sample size is greatly increased, and consequently more variables and outcomes can be examined. For example, analysis in subsets of patients and regression analyses⁹ that could not be done in individual trials can be performed in a meta-analysis.

A word of caution should be given with respect to larger samples and the possibility of multiple analyses of the data in meta-analysis. Much care must be exercised when examining the significance of effects that are not considered prior to the meta-analysis. The testing of effects suggested by the data and not planned a priori (sometimes called “data-mining”) increases considerably the risk of false-positive results. One common problem with large samples is the temptation to perform many so-called “subgroup analyses” in which subgroups of patients formed according to multiple baseline characteristics are compared.²⁷ The best way to minimize the possibility of false-positive results is to determine the effects to be tested before the data are collected and analyzed. Another method is to adjust the P value according to the number of analyses performed. In general, post hoc analyses should be deemed exploratory, and the reader should be made aware of this fact in order to judge the validity of the conclusion.

META-ANALYSIS OF RARE EVENTS

Lately, meta-analysis has been used to analyze outcomes that are rare and that individual studies were not designed to test. In general, the sample size of individual studies provides inadequate power to test rare outcomes. Adverse events are prime examples of important rare outcomes that are not always formally analyzed statistically. The problem in the analysis of adverse events is their low incidence. Paucity of events causes serious problems in any statistical analysis (see Shuster et al²⁸). The reason is that, with rare events, small changes in the data can cause dramatic changes in the results. This problem can persist even after pooling data from many studies. Instability of results is also exacerbated by the use of relative measures (eg, relative risk and odds ratio) instead of absolute measures of risk (eg, risk difference).

In a controversial meta-analysis, Nissen and Wolski⁴ combined 42 studies to examine the effect of rosiglitazone (Avandia) on the risk of myocardial infarction and death from cardiovascular causes. The overall estimated incidence of myocardial infarction in the treatment groups was 0.006 (86/14,376), or 6 in 1,000. Furthermore, 4 studies did not have any occurrences in either group, and 2 of the 42 studies accounted for 28.4% of the patients in the study.

Using a fixed-effect model, the odds ratio was 1.42, ie, the odds of myocardial infarction was 42% higher in patients using rosiglitazone, and the difference was statistically significant (95% confidence interval 1.03–1.98). Given the low frequency of myocardial infarction, this translates into an increase of only 1.78 myocardial infarctions per 1,000 patients (from 4.22 to 6 per 1,000). Furthermore, when the data were analyzed using other methods or if the two large studies were removed, the effect became nonsignificant.²⁹ Nissen and Wolski’s study⁴ is valuable and raises an important issue. However, the medical community would have been better served if a sensitivity analysis had been presented to highlight the fragility of the conclusions.

META-ANALYSIS VS LARGE RANDOMIZED CONTROLLED TRIALS

There is debate about how meta-analyses compare with large randomized controlled trials. In situations where a meta-analysis and a subsequent large randomized controlled trial are available, discrepancies are not uncommon.

LeLorier et al⁶ compared the results of 19 meta-analyses and 12 subsequent large randomized controlled trials on the same topics. In 5 (12%) of the 40 outcomes studied, the results of the trials were significantly different than those of the meta-analysis. The authors mentioned publication bias, study heterogeneity, and differences in populations as plausible explanations for the disagreements. However, they correctly commented: “this does not appear to be a large percentage, since a divergence in 5 percent of cases would be expected on the basis of chance alone.”⁶

A key reason for discrepancies is that meta-analyses are based on heterogeneous, often small studies. The results of a meta-analysis can be generalized to a target population similar to the target population in each of the studies. The patients in the individual studies can be substantially different with respect to diagnostic criteria, comorbidities, severity of disease, geographic region, and the time when the trial was conducted, among other factors. On the other hand, even in a large randomized controlled trial, the target population is necessarily more limited. These differences can explain many of the disagreements in the results.

A large, well-designed, randomized controlled trial is considered the gold standard in the sense that it provides the most reliable information on the specific target population from which the sample was drawn. Within that population the results of a randomized controlled trial supersede those of a meta-analysis. However, a well conducted meta-analysis can provide complementary information that is valuable to a researcher, clinician, or policy-maker.

CONCLUSION

Like many other statistical techniques, meta-analysis is a powerful tool when used judiciously; however, there are many caveats in its application. Clearly, meta-analysis has an important role in medical research, public policy, and clinical practice. Its use and value will likely increase, given the amount of new knowledge, the speed at which it is being created, and the availability of specialized software for performing it.³⁰ A meta-analysis needs to fulfill several key requirements to ensure the validity of its results:

• Well-defined objectives, including precise definitions of clinical variables and outcomes
• An appropriate and well-documented study identification and selection strategy
• Evaluation of bias in the identification and selection of studies
• Description and evaluation of heterogeneity
• Justification of data analytic techniques
• Use of sensitivity analysis.

It is imperative that researchers, policy-makers, and clinicians be able to critically assess the value and reliability of the conclusions of meta-analyses.

The amount of information generated in medical research is becoming overwhelming, even for experienced researchers. New studies are constantly being published, and clinicians are finding it nearly impossible to stay current, even in their own area of specialty.

To help make sense of the information, we are seeing more and more review articles that pool the results of multiple studies. When certain principles are followed and the data are quantitatively analyzed, these reviews are called meta-analyses. A PubMed search of the word “meta-analysis” in the title yielded 1,473 articles in the year 2007.

Combining available information to generate an integrated result seems reasonable and can save a considerable amount of resources. Nowadays, meta-analyses are being used to design future research, to provide evidence in the regulatory process,¹ and even to modify clinical practice.

Meta-analysis is powerful but also controversial—controversial because several conditions are critical to a sound meta-analysis, and small violations of those conditions can lead to misleading results. Summarizing large amounts of varied information using a single number is another controversial aspect of meta-analysis. Under scrutiny, some meta-analyses have been inappropriate, and their conclusions not fully warranted.^2,3

This article introduces the basic concepts of meta-analysis and discusses its caveats, with the aim of helping clinicians assess the merits of the results. We will use several recent meta-analyses to illustrate the issues, including a controversial one⁴ with potentially far-reaching consequences.

OBJECTIVES OF META-ANALYSIS

The main objectives of a meta-analysis are to:

• Summarize and integrate results from a number of individual studies
• Analyze differences in the results among studies
• Overcome small sample sizes of individual studies to detect effects of interest, and analyze end points that require larger sample sizes
• Increase precision in estimating effects
• Evaluate effects in subsets of patients
• Determine if new studies are needed to further investigate an issue
• Generate new hypotheses for future studies.

These lofty objectives can only be achieved when the meta-analysis satisfactorily addresses certain critical issues, which we will discuss next.

CRITICAL ISSUES IN META-ANALYSIS DESIGN

Four critical issues need to be addressed in a meta-analysis:

• Identification and selection of studies
• Heterogeneity of results
• Availability of information
• Analysis of the data.

IDENTIFICATION AND SELECTION OF STUDIES

The outcome of a meta-analysis depends on the studies included. The critical aspect of selecting studies to be included in a meta-analysis consists of two phases. The first is the identification phase or literature search, in which potential studies are identified. In the second phase, further criteria are used to create a list of studies for inclusion. Three insidious problems plague this aspect of meta-analysis: publication bias and search bias in the identification phase, and selection bias in the selection phase. These biases are discussed below.

Publication bias: ‘Positive’ studies are more likely to be printed

Searches of databases such as PubMed or Embase can yield long lists of studies. However, these databases include only studies that have been published. Such searches are unlikely to yield a representative sample because studies that show a “positive” result (usually in favor of a new treatment or against a well-established one) are more likely to be published than those that do not. This selective publication of studies is called publication bias.

In a recent article, Turner et al⁵ analyzed the publication status of studies of antidepressants. Based on studies registered with the US Food and Drug Administration (FDA), they found that 97% of the positive studies were published vs only 12% of the negative ones. Furthermore, when the nonpublished studies were not included in the analysis, the positive effects of individual drugs increased between 11% and 69%.

One reason for publication bias is that drug manufacturers are not generally interested in publishing negative studies. Another may be that editors favor positive studies because these are the ones that make the headlines and give the publication visibility. In some medical areas, the exclusion of studies conducted in non-English-speaking countries can increase publication bias.⁶

To ameliorate the effect of publication bias on the results of a meta-analysis, a serious effort should be made to identify unpublished studies. Identifying unpublished studies is easier now, thanks to improved communication between researchers worldwide, and thanks to registries in which all the studies of a certain disease or treatment are reported regardless of the result.

The National Institutes of Health maintains a registry of all the studies it supports, and the FDA keeps a registry and database in which drug companies must register all trials they intend to use in applying for marketing approval or a change in labeling. “Banks” of published and unpublished trials supported by pharmaceutical companies are also available (eg, http://ctr.gsk.co.uk/welcome.asp). The Cochrane collaboration (www.cochrane.org/) keeps records of systematic reviews and meta-analyses of many diseases and procedures.

Even in the ideal case that all relevant studies were available (ie, no publication bias), a faulty search can miss some of them. In searching databases, much care should be taken to assure that the set of key words used for searching is as complete as possible. This step is so critical that most recent meta-analyses include the list of key words used. The search engine (eg, PubMed, Google) is also critical, affecting the type and number of studies that are found.⁷ Small differences in search strategies can produce large differences in the set of studies found.⁸

Selection bias: Choosing the studies to be included

The identification phase usually yields a long list of potential studies, many of which are not directly relevant to the topic of the meta-analysis. This list is then subject to additional criteria to select the studies to be included. This critical step is also designed to reduce differences among studies, eliminate replication of data or studies, and improve data quality, and thus enhance the validity of the results.

To reduce the possibility of selection bias in this phase, it is crucial for the criteria to be clearly defined and for the studies to be scored by more than one researcher, with the final list chosen by consensus.^9,10 Frequently used criteria in this phase are in the areas of:

• Objectives
• Populations studied
• Study design (eg, experimental vs observational)
• Sample size
• Treatment (eg, type and dosage)
• Criteria for selection of controls
• Outcomes measured
• Quality of the data
• Analysis and reporting of results
• Accounting and reporting of attrition rates
• Length of follow-up
• When the study was conducted.

The objective in this phase is to select studies that are as similar as possible with respect to these criteria. It is a fact that even with careful selection, differences among studies will remain. But when the dissimilarities are large it becomes hard to justify pooling the results to obtain a “unified” conclusion.

In some cases, it is particularly difficult to find similar studies,^10,11 and sometimes the discrepancies and low quality of the studies can prevent a reasonable integration of results. In a systematic review of advanced lung cancer, Nicolucci et al¹² decided not to pool the results, in view of “systematic qualitative inadequacy of almost all trials” and lack of consistency in the studies and their methods. Marsoni et al¹³ came to a similar conclusion in attempting to summarize results in advanced ovarian cancer.

Stratification is an effective way to deal with inherent differences among studies and to improve the quality and usefulness of the conclusions. An added advantage to stratification is that insight can be gained by investigating discrepancies among strata.

There are many ways to create coherent subgroups of studies. For example, studies can be stratified according to their “quality,” assigned by certain scoring systems. Commonly used systems award points on the basis of how patients were selected and randomized, the type of blinding, the dropout rate, the outcome measurement, and the type of analysis (eg, intention-to-treat). However, these criteria, and therefore the scores, are somewhat subjective. Moher et al¹⁴ expand on this issue.

Large differences in sample sizes among studies are not uncommon and can cause problems in the analysis. Depending on the type of model used (see below), meta-analyses combine results based on the size of each study, but when the studies vary significantly in size, the large studies can still have an unduly large influence on the results. Stratifying by sample size is done sometimes to verify the stability of the results.⁴

On the other hand, the presence of dissimilarities among studies can have advantages by increasing the generalizability of the conclusions. Berlin and Colditz¹ point out that “we gain strength in inference when the range of patient characteristics has been broadened by replicating findings in studies with populations that vary in age range, geographic region, severity of underlying illness, and the like.”

Funnel plot: Detecting biases in the identification and selection of studies

The funnel plot is a technique used to investigate the possibility of biases in the identification and selection phases. In a funnel plot the size of the effect (defined as a measure of the difference between treatment and control) in each study is plotted on the horizontal axis against standard error¹⁵ or sample size⁹ on the vertical axis. If there are no biases, the graph will tend to have a symmetrical funnel shape centered in the average effect of the studies. When negative studies are missing, the graph shows lack of symmetry.

Funnel plots are appealing because they are simple, but their objective is to detect a complex effect, and they can be misleading. For example, lack of symmetry in a funnel plot can also be caused by heterogeneity in the studies.¹⁶ Another problem with funnel plots is that they are difficult to interpret when the number of studies is small. In some cases, however, the researcher may not have any option but to perform the analysis and report the presence of bias.¹¹

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

HETEROGENEITY OF RESULTS

In meta-analysis, heterogeneity refers to the degree of dissimilarity in the results of individual studies. In some cases, the dissimilarities in results can be traced back to inherent differences in the individual studies. In other situations, however, causes for the dissimilarities might not be easy to elucidate. In any case, as the level of heterogeneity increases, the justification for an integrated result becomes more difficult. A tool that is very effective to display the level of heterogeneity is the forest plot. In a forest plot, the estimated effect of each study along with a line representing a confidence interval is drawn. When the effects are similar, the confidence intervals overlap, and heterogeneity is low. The forest plot includes a reference line at the point of no effect (eg, one for relative risks and odds ratios). When some effects lie on opposite sides of the reference line, it means that the studies are contradictory and heterogeneity is high. In such cases, the conclusions of a meta-analysis are compromised.

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

AVAILABILITY OF INFORMATION

Most reports of individual studies include only summary results, such as means, standard deviations, proportions, odds ratios, and relative risks. Other than the possibility of errors in reporting, the lack of information can severely limit the type of analyses and conclusions that can be reached in a meta-analysis. For example, lack of information from individual studies can preclude the comparison of effects in predetermined subgroups of patients.

The best scenario is when data at the patient level are available. In such cases, the researcher has great flexibility in the analysis of the information. Trivella et al²⁰ performed a meta-analysis of the value of microvessel density in predicting survival in non-small-cell lung cancer. They obtained information on individual patients by contacting research centers directly. The data allowed them to vary the cutoff point to classify microvessel density as high or low and to use statistical methods to ameliorate heterogeneity.

A frequent problem in meta-analysis is the lack of uniformity in how outcomes are measured. In the study by Trivella et al,²⁰ the microvessel density was measured by two methods. The microvessel density was a significant prognostic factor when measured by one of the methods, but not the other.

RANDOMIZED CONTROLLED TRIALS VS OBSERVATIONAL STUDIES

Some researchers believe that meta-analyses should be conducted only on randomized controlled trials.^3,21 Their reasoning is that meta-analyses should include only reasonably well-conducted studies to reduce the risk of a misleading conclusion. However, many important diseases can only be studied observationally. If these studies have a certain level of quality, there is no technical reason not to include them in a meta-analysis.

Gillum et al²² performed a meta-analysis published in 2000 on the risk of ischemic stroke in users of oral contraceptives, based on observational studies (since no randomized trials were available). Studies were identified and selected by multiple researchers using strict criteria to make sure that only studies fulfilling certain standards were included. Of 804 potentially relevant studies identified, only 16 were included in the final analysis. A funnel plot showed no evidence of bias and the level of heterogeneity was fairly low. The meta-analysis result confirmed the results of individual studies, but the precision with which the effect was estimated was much higher. The overall relative risk of stroke in women taking oral contraceptives was 2.75, with a 95% confidence interval of 2.24 to 3.38.

A more recent meta-analysis²³ (published in 2004) on the same issue found no significant increase in the risk of ischemic stroke with the use of oral contraceptives. Gillum and Johnston²⁴ suggest that the main reason for the discrepancy is the lower amount of estrogen in newer oral contraceptives. They also point out differences in the control groups and study outcomes as reasons for the discrepancies between the two studies.

Bhutta et al²⁵ performed a meta-analysis of case-control (observational) studies of the effect of preterm birth on cognitive and behavioral outcomes. Studies were included only if the children were evaluated after their fifth birthday and the attrition rate was less than 30%. The studies were grouped according to criteria of quality devised specifically for case-control studies. The high-quality studies tended to show a larger effect than the low-quality studies, but the difference was not significant. Seventeen studies were included, and all of them found that children born preterm had lower cognitive scores; the difference was statistically significant in 15 of the studies. As expected, the meta-analysis confirmed these findings (95% confidence interval for the difference 9.2–12.5). The number of patients (1,556 cases and 1,720 controls) in the meta-analysis allowed the researchers to conclude further that the mean cognitive scores were directly proportional to the mean birth weight (R² = 0.51, P < .001) and gestational age (R² = 0.49, P < .001).

ANALYSIS OF DATA

There are specific statistical techniques that are used in meta-analysis to analyze and integrate the information. The data from the individual studies can be analyzed using either of two models: fixed effects or random effects.

The fixed-effects model assumes that the treatment effect is the same across studies. This common effect is unknown, and the purpose of the analysis is to estimate it with more precision than in the individual studies.

The random-effects model, on the other hand, assumes that the treatment effect is not the same across studies. The goal is to estimate the average effect in the studies.

In the fixed-effects model, the results of individual studies are pooled using weights that depend on the sample size of the study, whereas in the random-effects model each study is weighted equally. Due to the heterogeneity among studies, the random-effects model yields wider confidence intervals.

Both models have pros and cons. In many cases, the assumption that the treatment effect is the same in all the studies is not tenable, and the random-effects model is preferable. When the effect of interest is large, the results of both models tend to agree, particularly when the studies are balanced (ie, they have a similar number of patients in the treatment group as in the control group) and the study sizes are similar. But when the effect is small or when the level of heterogeneity of the studies is high, the result of the meta-analysis is likely to depend on the model used. In those cases, the analysis should be done and presented using both models.

It is highly desirable for a meta-analysis to include a sensitivity analysis to determine the “robustness” of the results. Two common ways to perform sensitivity analysis are to analyze the data using various methods and to present the results when some studies are removed from the analysis.²⁶ If these actions cause serious changes in the overall results, the credibility of the results is compromised.

The strength of meta-analysis is that, by pooling many studies, the effective sample size is greatly increased, and consequently more variables and outcomes can be examined. For example, analysis in subsets of patients and regression analyses⁹ that could not be done in individual trials can be performed in a meta-analysis.

A word of caution should be given with respect to larger samples and the possibility of multiple analyses of the data in meta-analysis. Much care must be exercised when examining the significance of effects that are not considered prior to the meta-analysis. The testing of effects suggested by the data and not planned a priori (sometimes called “data-mining”) increases considerably the risk of false-positive results. One common problem with large samples is the temptation to perform many so-called “subgroup analyses” in which subgroups of patients formed according to multiple baseline characteristics are compared.²⁷ The best way to minimize the possibility of false-positive results is to determine the effects to be tested before the data are collected and analyzed. Another method is to adjust the P value according to the number of analyses performed. In general, post hoc analyses should be deemed exploratory, and the reader should be made aware of this fact in order to judge the validity of the conclusion.

META-ANALYSIS OF RARE EVENTS

Lately, meta-analysis has been used to analyze outcomes that are rare and that individual studies were not designed to test. In general, the sample size of individual studies provides inadequate power to test rare outcomes. Adverse events are prime examples of important rare outcomes that are not always formally analyzed statistically. The problem in the analysis of adverse events is their low incidence. Paucity of events causes serious problems in any statistical analysis (see Shuster et al²⁸). The reason is that, with rare events, small changes in the data can cause dramatic changes in the results. This problem can persist even after pooling data from many studies. Instability of results is also exacerbated by the use of relative measures (eg, relative risk and odds ratio) instead of absolute measures of risk (eg, risk difference).

In a controversial meta-analysis, Nissen and Wolski⁴ combined 42 studies to examine the effect of rosiglitazone (Avandia) on the risk of myocardial infarction and death from cardiovascular causes. The overall estimated incidence of myocardial infarction in the treatment groups was 0.006 (86/14,376), or 6 in 1,000. Furthermore, 4 studies did not have any occurrences in either group, and 2 of the 42 studies accounted for 28.4% of the patients in the study.

Using a fixed-effect model, the odds ratio was 1.42, ie, the odds of myocardial infarction was 42% higher in patients using rosiglitazone, and the difference was statistically significant (95% confidence interval 1.03–1.98). Given the low frequency of myocardial infarction, this translates into an increase of only 1.78 myocardial infarctions per 1,000 patients (from 4.22 to 6 per 1,000). Furthermore, when the data were analyzed using other methods or if the two large studies were removed, the effect became nonsignificant.²⁹ Nissen and Wolski’s study⁴ is valuable and raises an important issue. However, the medical community would have been better served if a sensitivity analysis had been presented to highlight the fragility of the conclusions.

META-ANALYSIS VS LARGE RANDOMIZED CONTROLLED TRIALS

There is debate about how meta-analyses compare with large randomized controlled trials. In situations where a meta-analysis and a subsequent large randomized controlled trial are available, discrepancies are not uncommon.

LeLorier et al⁶ compared the results of 19 meta-analyses and 12 subsequent large randomized controlled trials on the same topics. In 5 (12%) of the 40 outcomes studied, the results of the trials were significantly different than those of the meta-analysis. The authors mentioned publication bias, study heterogeneity, and differences in populations as plausible explanations for the disagreements. However, they correctly commented: “this does not appear to be a large percentage, since a divergence in 5 percent of cases would be expected on the basis of chance alone.”⁶

A key reason for discrepancies is that meta-analyses are based on heterogeneous, often small studies. The results of a meta-analysis can be generalized to a target population similar to the target population in each of the studies. The patients in the individual studies can be substantially different with respect to diagnostic criteria, comorbidities, severity of disease, geographic region, and the time when the trial was conducted, among other factors. On the other hand, even in a large randomized controlled trial, the target population is necessarily more limited. These differences can explain many of the disagreements in the results.

A large, well-designed, randomized controlled trial is considered the gold standard in the sense that it provides the most reliable information on the specific target population from which the sample was drawn. Within that population the results of a randomized controlled trial supersede those of a meta-analysis. However, a well conducted meta-analysis can provide complementary information that is valuable to a researcher, clinician, or policy-maker.

CONCLUSION

Like many other statistical techniques, meta-analysis is a powerful tool when used judiciously; however, there are many caveats in its application. Clearly, meta-analysis has an important role in medical research, public policy, and clinical practice. Its use and value will likely increase, given the amount of new knowledge, the speed at which it is being created, and the availability of specialized software for performing it.³⁰ A meta-analysis needs to fulfill several key requirements to ensure the validity of its results:

• Well-defined objectives, including precise definitions of clinical variables and outcomes
• An appropriate and well-documented study identification and selection strategy
• Evaluation of bias in the identification and selection of studies
• Description and evaluation of heterogeneity
• Justification of data analytic techniques
• Use of sensitivity analysis.

It is imperative that researchers, policy-makers, and clinicians be able to critically assess the value and reliability of the conclusions of meta-analyses.

The amount of information generated in medical research is becoming overwhelming, even for experienced researchers. New studies are constantly being published, and clinicians are finding it nearly impossible to stay current, even in their own area of specialty.

To help make sense of the information, we are seeing more and more review articles that pool the results of multiple studies. When certain principles are followed and the data are quantitatively analyzed, these reviews are called meta-analyses. A PubMed search of the word “meta-analysis” in the title yielded 1,473 articles in the year 2007.

Combining available information to generate an integrated result seems reasonable and can save a considerable amount of resources. Nowadays, meta-analyses are being used to design future research, to provide evidence in the regulatory process,¹ and even to modify clinical practice.

Meta-analysis is powerful but also controversial—controversial because several conditions are critical to a sound meta-analysis, and small violations of those conditions can lead to misleading results. Summarizing large amounts of varied information using a single number is another controversial aspect of meta-analysis. Under scrutiny, some meta-analyses have been inappropriate, and their conclusions not fully warranted.^2,3

This article introduces the basic concepts of meta-analysis and discusses its caveats, with the aim of helping clinicians assess the merits of the results. We will use several recent meta-analyses to illustrate the issues, including a controversial one⁴ with potentially far-reaching consequences.

OBJECTIVES OF META-ANALYSIS

The main objectives of a meta-analysis are to:

• Summarize and integrate results from a number of individual studies
• Analyze differences in the results among studies
• Overcome small sample sizes of individual studies to detect effects of interest, and analyze end points that require larger sample sizes
• Increase precision in estimating effects
• Evaluate effects in subsets of patients
• Determine if new studies are needed to further investigate an issue
• Generate new hypotheses for future studies.

These lofty objectives can only be achieved when the meta-analysis satisfactorily addresses certain critical issues, which we will discuss next.

CRITICAL ISSUES IN META-ANALYSIS DESIGN

Four critical issues need to be addressed in a meta-analysis:

• Identification and selection of studies
• Heterogeneity of results
• Availability of information
• Analysis of the data.

IDENTIFICATION AND SELECTION OF STUDIES

The outcome of a meta-analysis depends on the studies included. The critical aspect of selecting studies to be included in a meta-analysis consists of two phases. The first is the identification phase or literature search, in which potential studies are identified. In the second phase, further criteria are used to create a list of studies for inclusion. Three insidious problems plague this aspect of meta-analysis: publication bias and search bias in the identification phase, and selection bias in the selection phase. These biases are discussed below.

Publication bias: ‘Positive’ studies are more likely to be printed

Searches of databases such as PubMed or Embase can yield long lists of studies. However, these databases include only studies that have been published. Such searches are unlikely to yield a representative sample because studies that show a “positive” result (usually in favor of a new treatment or against a well-established one) are more likely to be published than those that do not. This selective publication of studies is called publication bias.

In a recent article, Turner et al⁵ analyzed the publication status of studies of antidepressants. Based on studies registered with the US Food and Drug Administration (FDA), they found that 97% of the positive studies were published vs only 12% of the negative ones. Furthermore, when the nonpublished studies were not included in the analysis, the positive effects of individual drugs increased between 11% and 69%.

One reason for publication bias is that drug manufacturers are not generally interested in publishing negative studies. Another may be that editors favor positive studies because these are the ones that make the headlines and give the publication visibility. In some medical areas, the exclusion of studies conducted in non-English-speaking countries can increase publication bias.⁶

To ameliorate the effect of publication bias on the results of a meta-analysis, a serious effort should be made to identify unpublished studies. Identifying unpublished studies is easier now, thanks to improved communication between researchers worldwide, and thanks to registries in which all the studies of a certain disease or treatment are reported regardless of the result.

The National Institutes of Health maintains a registry of all the studies it supports, and the FDA keeps a registry and database in which drug companies must register all trials they intend to use in applying for marketing approval or a change in labeling. “Banks” of published and unpublished trials supported by pharmaceutical companies are also available (eg, http://ctr.gsk.co.uk/welcome.asp). The Cochrane collaboration (www.cochrane.org/) keeps records of systematic reviews and meta-analyses of many diseases and procedures.

Even in the ideal case that all relevant studies were available (ie, no publication bias), a faulty search can miss some of them. In searching databases, much care should be taken to assure that the set of key words used for searching is as complete as possible. This step is so critical that most recent meta-analyses include the list of key words used. The search engine (eg, PubMed, Google) is also critical, affecting the type and number of studies that are found.⁷ Small differences in search strategies can produce large differences in the set of studies found.⁸

Selection bias: Choosing the studies to be included

The identification phase usually yields a long list of potential studies, many of which are not directly relevant to the topic of the meta-analysis. This list is then subject to additional criteria to select the studies to be included. This critical step is also designed to reduce differences among studies, eliminate replication of data or studies, and improve data quality, and thus enhance the validity of the results.

To reduce the possibility of selection bias in this phase, it is crucial for the criteria to be clearly defined and for the studies to be scored by more than one researcher, with the final list chosen by consensus.^9,10 Frequently used criteria in this phase are in the areas of:

• Objectives
• Populations studied
• Study design (eg, experimental vs observational)
• Sample size
• Treatment (eg, type and dosage)
• Criteria for selection of controls
• Outcomes measured
• Quality of the data
• Analysis and reporting of results
• Accounting and reporting of attrition rates
• Length of follow-up
• When the study was conducted.

The objective in this phase is to select studies that are as similar as possible with respect to these criteria. It is a fact that even with careful selection, differences among studies will remain. But when the dissimilarities are large it becomes hard to justify pooling the results to obtain a “unified” conclusion.

In some cases, it is particularly difficult to find similar studies,^10,11 and sometimes the discrepancies and low quality of the studies can prevent a reasonable integration of results. In a systematic review of advanced lung cancer, Nicolucci et al¹² decided not to pool the results, in view of “systematic qualitative inadequacy of almost all trials” and lack of consistency in the studies and their methods. Marsoni et al¹³ came to a similar conclusion in attempting to summarize results in advanced ovarian cancer.

Stratification is an effective way to deal with inherent differences among studies and to improve the quality and usefulness of the conclusions. An added advantage to stratification is that insight can be gained by investigating discrepancies among strata.

There are many ways to create coherent subgroups of studies. For example, studies can be stratified according to their “quality,” assigned by certain scoring systems. Commonly used systems award points on the basis of how patients were selected and randomized, the type of blinding, the dropout rate, the outcome measurement, and the type of analysis (eg, intention-to-treat). However, these criteria, and therefore the scores, are somewhat subjective. Moher et al¹⁴ expand on this issue.

Large differences in sample sizes among studies are not uncommon and can cause problems in the analysis. Depending on the type of model used (see below), meta-analyses combine results based on the size of each study, but when the studies vary significantly in size, the large studies can still have an unduly large influence on the results. Stratifying by sample size is done sometimes to verify the stability of the results.⁴

On the other hand, the presence of dissimilarities among studies can have advantages by increasing the generalizability of the conclusions. Berlin and Colditz¹ point out that “we gain strength in inference when the range of patient characteristics has been broadened by replicating findings in studies with populations that vary in age range, geographic region, severity of underlying illness, and the like.”

Funnel plot: Detecting biases in the identification and selection of studies

The funnel plot is a technique used to investigate the possibility of biases in the identification and selection phases. In a funnel plot the size of the effect (defined as a measure of the difference between treatment and control) in each study is plotted on the horizontal axis against standard error¹⁵ or sample size⁹ on the vertical axis. If there are no biases, the graph will tend to have a symmetrical funnel shape centered in the average effect of the studies. When negative studies are missing, the graph shows lack of symmetry.

Funnel plots are appealing because they are simple, but their objective is to detect a complex effect, and they can be misleading. For example, lack of symmetry in a funnel plot can also be caused by heterogeneity in the studies.¹⁶ Another problem with funnel plots is that they are difficult to interpret when the number of studies is small. In some cases, however, the researcher may not have any option but to perform the analysis and report the presence of bias.¹¹

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

HETEROGENEITY OF RESULTS

In meta-analysis, heterogeneity refers to the degree of dissimilarity in the results of individual studies. In some cases, the dissimilarities in results can be traced back to inherent differences in the individual studies. In other situations, however, causes for the dissimilarities might not be easy to elucidate. In any case, as the level of heterogeneity increases, the justification for an integrated result becomes more difficult. A tool that is very effective to display the level of heterogeneity is the forest plot. In a forest plot, the estimated effect of each study along with a line representing a confidence interval is drawn. When the effects are similar, the confidence intervals overlap, and heterogeneity is low. The forest plot includes a reference line at the point of no effect (eg, one for relative risks and odds ratios). When some effects lie on opposite sides of the reference line, it means that the studies are contradictory and heterogeneity is high. In such cases, the conclusions of a meta-analysis are compromised.

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

Dentali F, et al. Meta-analysis: anticoagulant prophylaxis to prevent symptomatic venous thromboembolism in hospitalized medical patients. Ann Intern Med 2007; 146:278–288.

AVAILABILITY OF INFORMATION

Most reports of individual studies include only summary results, such as means, standard deviations, proportions, odds ratios, and relative risks. Other than the possibility of errors in reporting, the lack of information can severely limit the type of analyses and conclusions that can be reached in a meta-analysis. For example, lack of information from individual studies can preclude the comparison of effects in predetermined subgroups of patients.

The best scenario is when data at the patient level are available. In such cases, the researcher has great flexibility in the analysis of the information. Trivella et al²⁰ performed a meta-analysis of the value of microvessel density in predicting survival in non-small-cell lung cancer. They obtained information on individual patients by contacting research centers directly. The data allowed them to vary the cutoff point to classify microvessel density as high or low and to use statistical methods to ameliorate heterogeneity.

A frequent problem in meta-analysis is the lack of uniformity in how outcomes are measured. In the study by Trivella et al,²⁰ the microvessel density was measured by two methods. The microvessel density was a significant prognostic factor when measured by one of the methods, but not the other.

RANDOMIZED CONTROLLED TRIALS VS OBSERVATIONAL STUDIES

Some researchers believe that meta-analyses should be conducted only on randomized controlled trials.^3,21 Their reasoning is that meta-analyses should include only reasonably well-conducted studies to reduce the risk of a misleading conclusion. However, many important diseases can only be studied observationally. If these studies have a certain level of quality, there is no technical reason not to include them in a meta-analysis.

Gillum et al²² performed a meta-analysis published in 2000 on the risk of ischemic stroke in users of oral contraceptives, based on observational studies (since no randomized trials were available). Studies were identified and selected by multiple researchers using strict criteria to make sure that only studies fulfilling certain standards were included. Of 804 potentially relevant studies identified, only 16 were included in the final analysis. A funnel plot showed no evidence of bias and the level of heterogeneity was fairly low. The meta-analysis result confirmed the results of individual studies, but the precision with which the effect was estimated was much higher. The overall relative risk of stroke in women taking oral contraceptives was 2.75, with a 95% confidence interval of 2.24 to 3.38.

A more recent meta-analysis²³ (published in 2004) on the same issue found no significant increase in the risk of ischemic stroke with the use of oral contraceptives. Gillum and Johnston²⁴ suggest that the main reason for the discrepancy is the lower amount of estrogen in newer oral contraceptives. They also point out differences in the control groups and study outcomes as reasons for the discrepancies between the two studies.

Bhutta et al²⁵ performed a meta-analysis of case-control (observational) studies of the effect of preterm birth on cognitive and behavioral outcomes. Studies were included only if the children were evaluated after their fifth birthday and the attrition rate was less than 30%. The studies were grouped according to criteria of quality devised specifically for case-control studies. The high-quality studies tended to show a larger effect than the low-quality studies, but the difference was not significant. Seventeen studies were included, and all of them found that children born preterm had lower cognitive scores; the difference was statistically significant in 15 of the studies. As expected, the meta-analysis confirmed these findings (95% confidence interval for the difference 9.2–12.5). The number of patients (1,556 cases and 1,720 controls) in the meta-analysis allowed the researchers to conclude further that the mean cognitive scores were directly proportional to the mean birth weight (R² = 0.51, P < .001) and gestational age (R² = 0.49, P < .001).

ANALYSIS OF DATA

There are specific statistical techniques that are used in meta-analysis to analyze and integrate the information. The data from the individual studies can be analyzed using either of two models: fixed effects or random effects.

The fixed-effects model assumes that the treatment effect is the same across studies. This common effect is unknown, and the purpose of the analysis is to estimate it with more precision than in the individual studies.

The random-effects model, on the other hand, assumes that the treatment effect is not the same across studies. The goal is to estimate the average effect in the studies.

In the fixed-effects model, the results of individual studies are pooled using weights that depend on the sample size of the study, whereas in the random-effects model each study is weighted equally. Due to the heterogeneity among studies, the random-effects model yields wider confidence intervals.

Both models have pros and cons. In many cases, the assumption that the treatment effect is the same in all the studies is not tenable, and the random-effects model is preferable. When the effect of interest is large, the results of both models tend to agree, particularly when the studies are balanced (ie, they have a similar number of patients in the treatment group as in the control group) and the study sizes are similar. But when the effect is small or when the level of heterogeneity of the studies is high, the result of the meta-analysis is likely to depend on the model used. In those cases, the analysis should be done and presented using both models.

It is highly desirable for a meta-analysis to include a sensitivity analysis to determine the “robustness” of the results. Two common ways to perform sensitivity analysis are to analyze the data using various methods and to present the results when some studies are removed from the analysis.²⁶ If these actions cause serious changes in the overall results, the credibility of the results is compromised.

The strength of meta-analysis is that, by pooling many studies, the effective sample size is greatly increased, and consequently more variables and outcomes can be examined. For example, analysis in subsets of patients and regression analyses⁹ that could not be done in individual trials can be performed in a meta-analysis.

A word of caution should be given with respect to larger samples and the possibility of multiple analyses of the data in meta-analysis. Much care must be exercised when examining the significance of effects that are not considered prior to the meta-analysis. The testing of effects suggested by the data and not planned a priori (sometimes called “data-mining”) increases considerably the risk of false-positive results. One common problem with large samples is the temptation to perform many so-called “subgroup analyses” in which subgroups of patients formed according to multiple baseline characteristics are compared.²⁷ The best way to minimize the possibility of false-positive results is to determine the effects to be tested before the data are collected and analyzed. Another method is to adjust the P value according to the number of analyses performed. In general, post hoc analyses should be deemed exploratory, and the reader should be made aware of this fact in order to judge the validity of the conclusion.

META-ANALYSIS OF RARE EVENTS

Lately, meta-analysis has been used to analyze outcomes that are rare and that individual studies were not designed to test. In general, the sample size of individual studies provides inadequate power to test rare outcomes. Adverse events are prime examples of important rare outcomes that are not always formally analyzed statistically. The problem in the analysis of adverse events is their low incidence. Paucity of events causes serious problems in any statistical analysis (see Shuster et al²⁸). The reason is that, with rare events, small changes in the data can cause dramatic changes in the results. This problem can persist even after pooling data from many studies. Instability of results is also exacerbated by the use of relative measures (eg, relative risk and odds ratio) instead of absolute measures of risk (eg, risk difference).

In a controversial meta-analysis, Nissen and Wolski⁴ combined 42 studies to examine the effect of rosiglitazone (Avandia) on the risk of myocardial infarction and death from cardiovascular causes. The overall estimated incidence of myocardial infarction in the treatment groups was 0.006 (86/14,376), or 6 in 1,000. Furthermore, 4 studies did not have any occurrences in either group, and 2 of the 42 studies accounted for 28.4% of the patients in the study.

Using a fixed-effect model, the odds ratio was 1.42, ie, the odds of myocardial infarction was 42% higher in patients using rosiglitazone, and the difference was statistically significant (95% confidence interval 1.03–1.98). Given the low frequency of myocardial infarction, this translates into an increase of only 1.78 myocardial infarctions per 1,000 patients (from 4.22 to 6 per 1,000). Furthermore, when the data were analyzed using other methods or if the two large studies were removed, the effect became nonsignificant.²⁹ Nissen and Wolski’s study⁴ is valuable and raises an important issue. However, the medical community would have been better served if a sensitivity analysis had been presented to highlight the fragility of the conclusions.

META-ANALYSIS VS LARGE RANDOMIZED CONTROLLED TRIALS

There is debate about how meta-analyses compare with large randomized controlled trials. In situations where a meta-analysis and a subsequent large randomized controlled trial are available, discrepancies are not uncommon.

LeLorier et al⁶ compared the results of 19 meta-analyses and 12 subsequent large randomized controlled trials on the same topics. In 5 (12%) of the 40 outcomes studied, the results of the trials were significantly different than those of the meta-analysis. The authors mentioned publication bias, study heterogeneity, and differences in populations as plausible explanations for the disagreements. However, they correctly commented: “this does not appear to be a large percentage, since a divergence in 5 percent of cases would be expected on the basis of chance alone.”⁶

A key reason for discrepancies is that meta-analyses are based on heterogeneous, often small studies. The results of a meta-analysis can be generalized to a target population similar to the target population in each of the studies. The patients in the individual studies can be substantially different with respect to diagnostic criteria, comorbidities, severity of disease, geographic region, and the time when the trial was conducted, among other factors. On the other hand, even in a large randomized controlled trial, the target population is necessarily more limited. These differences can explain many of the disagreements in the results.

A large, well-designed, randomized controlled trial is considered the gold standard in the sense that it provides the most reliable information on the specific target population from which the sample was drawn. Within that population the results of a randomized controlled trial supersede those of a meta-analysis. However, a well conducted meta-analysis can provide complementary information that is valuable to a researcher, clinician, or policy-maker.

CONCLUSION

Like many other statistical techniques, meta-analysis is a powerful tool when used judiciously; however, there are many caveats in its application. Clearly, meta-analysis has an important role in medical research, public policy, and clinical practice. Its use and value will likely increase, given the amount of new knowledge, the speed at which it is being created, and the availability of specialized software for performing it.³⁰ A meta-analysis needs to fulfill several key requirements to ensure the validity of its results:

• Well-defined objectives, including precise definitions of clinical variables and outcomes
• An appropriate and well-documented study identification and selection strategy
• Evaluation of bias in the identification and selection of studies
• Description and evaluation of heterogeneity
• Justification of data analytic techniques
• Use of sensitivity analysis.

It is imperative that researchers, policy-makers, and clinicians be able to critically assess the value and reliability of the conclusions of meta-analyses.

Physicians and their women patients have faced a continuous, confusing mix of information about the risks and benefits of hormone therapy (HT) for perimenopausal and postmenopausal women, most of it without respect to age or the timing of HT relative to menopause. Initial data from the Women’s Health Initiative (WHI) estrogen + progestin (E+P) trial, a prevention study conducted predominantly in older postmenopausal women without menopausal symptoms,¹ resulted in questioning of the role of HT (unfortunately and inappropriately in younger symptomatic women). Cumulative trial data and further analyses of the WHI have refined and added to our understanding of the effects of HT, particularly with regard to cardiovascular health. This review will update physicians on the latest data on the risks and benefits of HT, with a particular focus on the heart, and will put the risks of HT into appropriate clinical context.

HORMONE THERAPY AND CARDIOVASCULAR DISEASE: A HISTORICAL PERSPECTIVE

Observational studies conducted prior to the WHI found consistently that women who self-selected to use HT had a reduction in mortality and in the incidence of cardiovascular disease relative to women who did not choose to use HT.^2–8 This reduction in risk was apparent whether the HT users had taken ET (estrogen therapy) or EPT (estrogen-progestogen therapy). In contrast, randomized controlled trials failed to confirm these findings from observational studies. However, the findings from randomized controlled trials were derived from older postmenopausal women who were many years past menopause. Often overlooked is the WHI observational study of ET and EPT,^9,10 in which women who chose to use HT had a reduction in the risk of coronary heart disease (CHD) similar to that observed among the HT users in other observational studies.

RECENT REPORTS FROM THE WHI

Since the original publication of the WHI E+P trial in 2002,¹ an extensive collection of data have been published in piecemeal fashion, contributing to the confusion and misperception of the effects of HT on risks and benefits. It is important to note that the WHI consists of both randomized and observational components, as detailed below, and that data have come from both. Together, these data help clarify the misperceptions generated from the first WHI report of 2002,¹ particularly misperceptions regarding the timing of HT initiation relative to menopause and the effect of HT duration on cardiovascular disease outcomes. In most instances, the conclusions drawn from this recent research run counter to the inaccurate but prevalent perception that HT use at any time and at any age is associated with cardiovascular harm, a perception that has unfortunately prevailed since the initial publication of the WHI E+P trial findings in 2002.

The WHI randomized trials and combined analysis

The WHI trials enrolled 27,347 postmenopausal women aged 50 to 79 years at baseline; almost two-thirds of the women enrolled were 60 years of age or older, and the majority of women were more than 10 years past menopause. WHI actually comprised two parallel randomized trials:

• One among 16,608 women who had not undergone hysterectomy (ie, with uterus intact), who were randomized to EPT or placebo (ie, WHI E+P trial) ¹
• One among 10,739 women who had undergone hysterectomy, who were randomized to ET or placebo.¹¹

Recent analyses from the WHI, published in 2007, assessed the cardiovascular effects of ET and EPT independently and combined, both overall and according to subject age and years since menopause when randomized.¹² Other analyses following the initial WHI E+P trial publication have analyzed the effects of HT according to duration of HT use and according to secondary end points. Many of these analyses have presented risks and benefits in terms of both nominal and adjusted confidence intervals (CIs). Nominal 95% CIs describe the variability in risk estimates that would arise from a simple trial for a single end point. Although nominal CIs are traditionally used, they do not take into account the multiple statistical testing issues (across time and across outcome categories) that occur in a trial. In contrast, adjusted 95% CIs correct for these stastical testing issues. From a clinical perspective, it is most appropriate to look at the adjusted CIs.

WHI: EPT vs placebo

CHD. Although the point estimate for CHD is increased, the 95% CI indicates that EPT has a non-significant effect on CHD outcome relative to placebo among all women randomized in the WHI E+P trial (mean age, 63 years).¹² This is a very important point for cardiologists and primary care physicians to note.

In a 2008 analysis of the WHI E+P trial that included a 2.4-year open-label follow-up subsequent to the randomized trial,¹⁹ the randomized trial data reported were again different than in previous reports, but remained nonsignificant. The hazard ratio (HR) reported for CHD in the 2008 analysis for the randomized portion of the trial was 1.22 (95% CI, 0.99 to 1.51), as compared with 1.23 (95% CI, 0.99 to 1.53) reported in 2007¹² (Table 1), 1.24 (95% CI, 1.00 to 1.54) reported in 2003,²⁰ and 1.29 (95% CI, 1.02 to 1.63) reported in 2002.¹ In the 2.4-year open-label follow-up period in which women were no longer on their randomized regimens (EPT or placebo), the HR for CHD between the original randomized treatment groups was nonsignificant at 0.95 (95% CI, 0.73 to 1.26) and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Stroke. The risk of stroke was increased significantly (by an additional 8 events per 10,000 women treated per year) in the EPT arm versus the placebo arm in the nominal analysis,¹² but this difference was nonsignificant after adjustment.¹³

In the 2008 WHI E+P analysis, the HR reported for stroke during the randomized trial phase was different than in previous reports—1.34 (95% CI, 1.05 to 1.71)¹⁹ versus 1.31 (95% CI, 1.03 to 1.68) reported in 2007¹² (Table 1)—and the adjusted analysis was not reported. In the 2.4-year open-label follow-up period in which women were no longer on their randomized regimens, the HR for stroke between the original randomized treatment groups was nonsignifi-cant at 1.16 (95% CI, 0.83 to 1.61) and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Breast cancer. Breast cancer risk was originally reported to be increased significantly (by an additional 8 cases per 10,000 women per year) in the EPT arm versus the placebo arm with the nominal statistic, but this increase was nonsignificant after adjustment.¹ This risk estimate was revised in a follow-up publication 1 year after the original data were reported, and the increase in risk in the EPT arm was still no longer significant in the adjusted analysis.²¹ Importantly, another subsequent analysis that adjusted for baseline risk factors for breast cancer resulted in a further revision of the risk estimate, which again showed a nonsignificant increase in the EPT arm relative to the placebo arm.¹⁴ This is very important since it is commonly accepted that EPT increases the risk of breast cancer when this has not been definitively proven in any randomized controlled trial.

Unfortunately, the most recent breast cancer data¹⁴ (Table 1) were not used in the 2008 WHI E+P analysis.¹⁹ However, even using the unadjusted data in the 2.4-year open-label follow-up in which women were no longer on their randomized regimens, the HR for breast cancer between the original randomized treatment groups was nonsignificant and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Venous thromboembolism (VTE). EPT was associated with a doubling of the risk of VTE (ie, deep vein thrombosis and pulmonary embolism) compared with placebo, resulting in an excess of 18 VTE events per 10,000 women per year of therapy.¹⁵ The risk of VTE was significant across the entire cohort of women (mean age, 63 years).

In the 2008 WHI E+P analysis, the HR reported for VTE during the randomized phase was different than in previous reports—1.98 (95% CI, 1.52 to 2.59)¹⁹ versus 2.06 (95% CI, 1.57 to 2.70) reported in 2004¹⁵ (Table 1)—and the HR during the 2.4-year open-label follow-up, in which women were no longer on their randomized regimens, was no longer significant (HR = 0.95; 95% CI, 0.63 to 1.44). This change in the HR over time from the randomized phase to the open-label phase was statistically significant.¹⁹

Fracture. The risk of hip fracture was reduced by 33% with EPT relative to placebo, which was statistically significant in the nominal analysis but not in the adjusted analysis.¹⁷ The risk of any fracture was reduced by 24% in women randomized to EPT compared with placebo, which was statistically significant and translated to 47 fewer fractures per 10,000 women per year of therapy.¹⁷ Clinically, these results are most impressive given that women randomized in the WHI were not selected on the basis of risk for osteoporosis or fracture. This claim cannot be made for any other therapy.

In the 2008 WHI E+P analysis of the 2.4-year open-label follow-up in which women were no longer on their randomized regimens, the HR between the original randomized treatment groups was nonsignificant for hip fracture and any fracture—0.92 (95% CI, 0.64 to 1.34) and 0.91 (95% CI, 0.78 to 1.06), respectively—and the change in the HR over time from the randomized phase to the open-label phase was not significant for either fracture outcome.¹⁹

Diabetes. The risk of new-onset diabetes was reduced by a statistically significant 21% in women randomized to EPT compared with placebo.¹⁸

CHD. Importantly, no significant difference was found between the ET and placebo arms with respect to CHD events in the overall cohort of women, whose average age was 64 years.¹²

Stroke. The risk of stroke was greater with ET than with placebo in the nominal analysis, but importantly, the difference in event rates (11 per 10,000 women per year of therapy) failed to reach significance in the adjusted analysis.^11,12

Breast cancer. A strong but nonsignificant trend toward a reduction in breast cancer risk was apparent in the ET arm (8 fewer breast cancer cases per 10,000 women per year of therapy). Among women who actually were adherent to their study regimen (ie, consuming ≥ 80% of their study medication), there was a statistically significant 33% reduction in breast cancer risk with ET relative to placebo.²² Importantly, the reduction in breast cancer risk relative to placebo was found across all the age ranges studied.¹¹

VTE. The excess risk of VTE with ET versus placebo (32%) was less than the excess risk of VTE associated with EPT (Table 1). Importantly, the risk of VTE associated with ET was not statistically significant.^11,23

Fracture. The risk of any fracture (hip or vertebral) was reduced significantly in the ET arm compared with the placebo arm.¹¹

Diabetes. In a nominal analysis, there was a trend toward a reduction in the risk of new-onset diabetes in women randomized to ET relative to placebo, which nearly achieved statistical significance.²⁴

WHAT EXPLAINS THE DISCORDANCE BETWEEN OBSERVATIONAL AND RANDOMIZED TRIALS OF HT?

How can the perceived discordance between the results from observational studies and those from randomized controlled trials be explained? There are currently three hypotheses:

• The populations differ in the two types of study designs (observational studies and randomized controlled trials)
• The duration of HT use differs
• The timing of HT initiation differs in relation to age, time since menopause, and stage of atherosclerosis.

Population characteristics

One obvious difference between randomized trials and observational studies of HT is the presence of menopausal symptoms. To maintain blinding, women with hot flashes were predominantly excluded from randomized trials of HT, whereas the presence of hot flashes is the predominant menopausal symptom of women included in observational studies and the main reason women seek HT from their providers.

Other consistent and possibly explanatory differences between clinical trials and observational studies of HT are patient age at enrollment, years since menopause, and body mass index (BMI). Comparing randomized controlled trials with observational studies, age at enrollment was much higher in the clinical trials (mean age ≥ 63 years) than in the observational studies (range of 30 to 55 years). Similarly, women enrolled in randomized trials were more than 10 years beyond menopause, whereas those in observational studies were less than 5 years beyond menopause. In fact, more than 80% of HT users in observational studies initiated HT within 1 or 2 years of menopause.

Additionally, women in randomized trials of HT tend to have higher BMIs than their counterparts in observational studies. For example, mean BMI was considerably higher in the WHI randomized trials (28.5 kg/m² and 30.1 kg/m²)^1,11 than in the observational Nurses’ Health Study (25.8 kg/m²),²⁵ and a full third (34%) of women in the WHI randomized trials were severely obese (BMI ≥ 30 kg/m²).

This point about BMI is noteworthy in light of findings on the effect of BMI on breast cancer risk from an analysis of the WHI observational study among 85,917 women aged 50 to 79 years old at enrollment.²⁶ This analysis found that BMI was unrelated to breast cancer risk among women who had used HT; however, among nonusers of HT, a baseline BMI greater than 31.1 kg/m² was associated with a 2.52 relative risk of breast cancer compared with a baseline BMI less than 22.6 kg/m². The risk of breast cancer with increasing BMI was most pronounced in younger postmenopausal women. One interpretation is that high endogenous estrogen levels in postmenopausal women with an elevated BMI serve to increase breast cancer risk to a level beyond which HT adds no further risk. Alternatively, conjugated equine estrogens may act through a selective estrogen receptor modulator mechanism to block any potential adverse breast tissue effects of elevated endogenous estrone and estradiol levels.

Another hypothesis for the discordant findings between observational and randomized studies of HT focuses on differences in duration of HT use between the two types of studies. Duration of HT use has been substantially longer in observational studies, ranging from 10 years to 40 years, compared with no more than 7 or 8 years in randomized trials to date. Moreover, the results from observational studies have suggested that the longer the duration of HT use, the greater the benefit in terms of CHD risk.

For example, a case-control study by Chilvers et al showed that HT protected against nonfatal myocardial infarction only when used for more than 60 months.²⁷ Analysis of data on EPT use from the Heart and Estrogen/progestin Replacement Study (HERS),²⁸ the randomized WHI E+P trial,²⁰ and the WHI observational study⁹ reveals an interesting and consistent trend: rates of CHD events increased during the first year of EPT therapy (compared with placebo or nonuse) but then declined over time, ending up below the rates of CHD events in placebo recipients or nonusers of HT after approximately 5 years of therapy. This trend was not as pronounced with ET use in either the WHI randomized trial²⁹ or the WHI observational study,¹⁰ but the incidence of CHD events did decline over time with ET use in both studies, and in the WHI observational study, greater than 5 years of ET use was associated with a significant 27% reduction (HR, 0.73; 95% CI, 0.61 to 0.84) in CHD events compared with nonuse.¹⁰

Timing of HT initiation

A third hypothesis for the discordance between randomized and observational studies of HT concerns the timing of HT initiation relative to patient age, time since menopause, and stage of atherosclerosis. As noted above, women enrolled in randomized trials have tended to be considerably older and further from menopause compared with their counterparts in observational studies. The effects of differences in timing of HT initiation on specific clinical end points, particularly in relation to cardiovascular health, are reviewed below.

Reprinted, with permission, from JAMA (Rossouw JE, et al. JAMA 2007; 297: 1465–1477). Copyright © 2007 American Medical Association. All rights reserved.

Early and continued HT use may interrupt the pathogenic sequence of vascular aging, potentially preventing progression of atherosclerosis to the late stage at which plaque rupture and clinical events occur. In contrast, late intervention with HT may have little effect on established plaque and, in the case of EPT, may actually predispose to plaque rupture. This concept has been demonstrated in a pair of sister studies, the Women’s Estrogen-Progestin Lipid-Lowering Hormone Atherosclerosis Regression Trial (WELLHART)³¹ and the Estrogen Prevention Atherosclerosis Trial (EPAT).³²

Mortality. The same Stanford University researchers who performed the above meta-analysis of CHD events also assessed odd ratios for overall mortality in a meta-analysis of 30 randomized trials of HT versus placebo that included a total of 26,708 postmenopausal women (representing 119,118 patient-years).³³ They found the timing of HT initiation to have an effect on mortality similar to its effect on CHD events. In the overall cohort of women, the odds of death were not different between the HT and placebo groups, but among women younger than 60 years (mean, 54 years), those randomized to HT had a significant 39% reduction in the risk of death (HR = 0.61; 95% CI, 0.39 to 0.95) compared with those randomized to placebo. HT had no effect on mortality among women older than 60 years (mean, 66 years) in this analysis.

These mortality data are consistent with those from the WHI randomized trials, in which both EPT and ET were associated with a 30% reduction in overall mortality relative to placebo among women 50 to 59 years old.¹² When both the EPT and ET portions of the WHI were combined to increase the sample size, HT was associated with a significant 30% reduction in mortality (HR = 0.70; 95% CI, 0.51 to 0.96) compared with placebo among women 50 to 59 years old.¹²

Stroke. The most recent WHI data indicate that stroke is not increased with ET in women 50 to 59 years of age, as there were 2 fewer events per 10,000 women per year of ET relative to placebo.¹² In this same age group, the risk of stroke from EPT was increased by 5 events per 10,000 women per year of therapy relative to placebo.¹² Among women randomized within 5 years of menopause, stroke risk was increased by 3 events per 10,000 women per year of EPT relative to placebo.13

VTE. Although age was not a significant contributing factor to the risk of VTE from HT in the WHI, the absolute risk of VTE was lower in younger versus older women. The additional absolute risk for VTE events per 10,000 women per year of EPT use was 11 events for women 50 to 59 years old at randomization, 16 events for women 60 to 69 years old at randomization, and 35 events for women 70 to 79 years old at randomization.¹⁵ The additional absolute risk for VTE events per 10,000 women per year of ET use was 4 events for women 50 to 59 years old at randomization, 7 events for women 60 to 69 years old at randomization, and 11 events for women 70 to 79 years old at randomization.²³ A history of a prior thromboembolic event increases the risk of VTE with postmenopausal HT use, which should be considered before HT is initiated.

Comparative effects of lipid-lowering therapy and HT

Examination of the evidence regarding lipid-lowering therapy for prevention of CHD, as well as its effects on breast cancer risk and coronary artery calcium scores in women, can add some much-needed perspective and context to the HT data reviewed above.

CHD prevention. Walsh and Pignone examined six randomized controlled trials (N = 11,435) of primary prevention with lipid-lowering medication in women and found no significant effect on CHD events, nonfatal myocardial infarction, CHD mortality, or total mortality.³⁴ In eight randomized controlled trials of secondary prevention (N = 8,272), lipid-lowering therapy in women resulted in significant reductions in CHD end points but had no effect on total mortality.³⁴

Atherosclerosis progression. In three randomized controlled trials, 1 to 4 years of statin therapy had no effect on the progression of coronary artery calcium compared with placebo.^41–43 Among 1,064 women 50 to 59 years old who participated in a WHI substudy called the WHI Coronary Artery Calcium Study (WHI-CACS), those who were randomized to ET had significantly less coronary artery calcium at year 7 compared with those assigned to placebo.⁴⁴ The mean Agatston coronary artery calcium scores were 83.1 with ET versus 123.1 with placebo (P = .02).

Comparing risk of HT with risk of other therapies

Other comparisons yield similar conclusions.³⁵ For instance, a comparison of the ET arm of the WHI randomized trial with raloxifene in the Raloxifene Use for the Heart (RUTH) trial⁵² reveals similar effects on CHD, stroke, VTE, pulmonary embolism, deep vein thrombosis, and breast cancer, with the only large difference being a greater reduction in bone fracture risk with ET compared with raloxifene.³⁵ Finally, in putting the risk of VTE with HT into perspective, consider that the risk of thromboembolic events with selective estrogen receptor modulators⁵² and with the fibric acid derivative fenofibrate in diabetics⁵⁰ is of similar magnitude as the risk with HT.

Risk must be viewed in light of age at HT initiation

CONCLUSIONS FROM RANDOMIZED TRIALS

HT’s effects on CHD and mortality: Timing is everything

Cumulative data from randomized trials indicate that in the overall population of women studied, HT, aspirin, and lipid-lowering therapy each have a null effect on the incidence of CHD and mortality. However, within this overall null effect, early initiation of HT (in terms of time from menopause [< 10 years] and age at initiation [< 60 years]) is associated with reductions in total mortality and CHD incidence. Additionally, a duration of HT use beyond 5 years is associated with a reduction in the incidence of CHD.^{9,10,20,28,29} These effects of the timing and duration of therapy are unique to HT. Unopposed ET may have an advantageous profile relative to EPT for reducing the incidence of CHD and mortality in postmenopausal women.

Gaining perspective on risks

The risks of stroke and VTE associated with HT are low for women overall and lower still for women who are within 10 years of menopause or younger than age 60 when they start HT. With respect to stroke, fewer cases of stroke developed in users of ET compared with placebo in women who started HT before age 60. The risks of EPT are comparable to those of other medications commonly used in this population of women. More broadly, the magnitude and types of risk associated with HT are similar to those associated with other commonly used therapies.

In addition, the underappreciated benefits of HT, such as potential prevention of diabetes mellitus (15 fewer cases of incident diabetes per 10,000 women per year with EPT and 14 fewer cases with ET), need to be recognized and discussed with our patients. These data are consistent in both observational studies and randomized trials.

The bottom line

As data from randomized trials of HT accumulate, the results are clearly similar to those from the more than 20 observational studies indicating that young, symptomatic postmenopausal women who use HT for long periods have lower rates of CHD and total mortality compared with postmenopausal women who do not use HT. Consistent with these data, the American Association of Clinical Endocrinologists issued a position statement in 2008 concluding that for symptomatic menopausal women under the age of 60, the benefits of HT exceed the risks.⁵³

Nevertheless, the bottom line remains that the estrogen-cardioprotective hypothesis has yet to be studied, since randomized trials have not been conducted in the same population of women from which the hypothesis was generated. This hypothesis will be directly evaluated, however, in the ongoing Early versus Late Intervention Trial with Estradiol (ELITE). This randomized trial, funded by the National Institute on Aging, is designed to determine the effects of 17β-estradiol on the progression of atherosclerosis, cognition, and other postmenopausal health issues in recently menopausal (< 6 years) and remotely menopausal (≥ 10 years) women with no history of cardiovascular disease or diabetes. Until data from trials like ELITE emerge, guidelines such as those from the North American Menopause Society⁵⁴ (reviewed in the next article in this supplement) are reasonable for clinical practice.

Physicians and their women patients have faced a continuous, confusing mix of information about the risks and benefits of hormone therapy (HT) for perimenopausal and postmenopausal women, most of it without respect to age or the timing of HT relative to menopause. Initial data from the Women’s Health Initiative (WHI) estrogen + progestin (E+P) trial, a prevention study conducted predominantly in older postmenopausal women without menopausal symptoms,¹ resulted in questioning of the role of HT (unfortunately and inappropriately in younger symptomatic women). Cumulative trial data and further analyses of the WHI have refined and added to our understanding of the effects of HT, particularly with regard to cardiovascular health. This review will update physicians on the latest data on the risks and benefits of HT, with a particular focus on the heart, and will put the risks of HT into appropriate clinical context.

HORMONE THERAPY AND CARDIOVASCULAR DISEASE: A HISTORICAL PERSPECTIVE

Observational studies conducted prior to the WHI found consistently that women who self-selected to use HT had a reduction in mortality and in the incidence of cardiovascular disease relative to women who did not choose to use HT.^2–8 This reduction in risk was apparent whether the HT users had taken ET (estrogen therapy) or EPT (estrogen-progestogen therapy). In contrast, randomized controlled trials failed to confirm these findings from observational studies. However, the findings from randomized controlled trials were derived from older postmenopausal women who were many years past menopause. Often overlooked is the WHI observational study of ET and EPT,^9,10 in which women who chose to use HT had a reduction in the risk of coronary heart disease (CHD) similar to that observed among the HT users in other observational studies.

RECENT REPORTS FROM THE WHI

Since the original publication of the WHI E+P trial in 2002,¹ an extensive collection of data have been published in piecemeal fashion, contributing to the confusion and misperception of the effects of HT on risks and benefits. It is important to note that the WHI consists of both randomized and observational components, as detailed below, and that data have come from both. Together, these data help clarify the misperceptions generated from the first WHI report of 2002,¹ particularly misperceptions regarding the timing of HT initiation relative to menopause and the effect of HT duration on cardiovascular disease outcomes. In most instances, the conclusions drawn from this recent research run counter to the inaccurate but prevalent perception that HT use at any time and at any age is associated with cardiovascular harm, a perception that has unfortunately prevailed since the initial publication of the WHI E+P trial findings in 2002.

The WHI randomized trials and combined analysis

The WHI trials enrolled 27,347 postmenopausal women aged 50 to 79 years at baseline; almost two-thirds of the women enrolled were 60 years of age or older, and the majority of women were more than 10 years past menopause. WHI actually comprised two parallel randomized trials:

• One among 16,608 women who had not undergone hysterectomy (ie, with uterus intact), who were randomized to EPT or placebo (ie, WHI E+P trial) ¹
• One among 10,739 women who had undergone hysterectomy, who were randomized to ET or placebo.¹¹

Recent analyses from the WHI, published in 2007, assessed the cardiovascular effects of ET and EPT independently and combined, both overall and according to subject age and years since menopause when randomized.¹² Other analyses following the initial WHI E+P trial publication have analyzed the effects of HT according to duration of HT use and according to secondary end points. Many of these analyses have presented risks and benefits in terms of both nominal and adjusted confidence intervals (CIs). Nominal 95% CIs describe the variability in risk estimates that would arise from a simple trial for a single end point. Although nominal CIs are traditionally used, they do not take into account the multiple statistical testing issues (across time and across outcome categories) that occur in a trial. In contrast, adjusted 95% CIs correct for these stastical testing issues. From a clinical perspective, it is most appropriate to look at the adjusted CIs.

WHI: EPT vs placebo

CHD. Although the point estimate for CHD is increased, the 95% CI indicates that EPT has a non-significant effect on CHD outcome relative to placebo among all women randomized in the WHI E+P trial (mean age, 63 years).¹² This is a very important point for cardiologists and primary care physicians to note.

In a 2008 analysis of the WHI E+P trial that included a 2.4-year open-label follow-up subsequent to the randomized trial,¹⁹ the randomized trial data reported were again different than in previous reports, but remained nonsignificant. The hazard ratio (HR) reported for CHD in the 2008 analysis for the randomized portion of the trial was 1.22 (95% CI, 0.99 to 1.51), as compared with 1.23 (95% CI, 0.99 to 1.53) reported in 2007¹² (Table 1), 1.24 (95% CI, 1.00 to 1.54) reported in 2003,²⁰ and 1.29 (95% CI, 1.02 to 1.63) reported in 2002.¹ In the 2.4-year open-label follow-up period in which women were no longer on their randomized regimens (EPT or placebo), the HR for CHD between the original randomized treatment groups was nonsignificant at 0.95 (95% CI, 0.73 to 1.26) and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Stroke. The risk of stroke was increased significantly (by an additional 8 events per 10,000 women treated per year) in the EPT arm versus the placebo arm in the nominal analysis,¹² but this difference was nonsignificant after adjustment.¹³

In the 2008 WHI E+P analysis, the HR reported for stroke during the randomized trial phase was different than in previous reports—1.34 (95% CI, 1.05 to 1.71)¹⁹ versus 1.31 (95% CI, 1.03 to 1.68) reported in 2007¹² (Table 1)—and the adjusted analysis was not reported. In the 2.4-year open-label follow-up period in which women were no longer on their randomized regimens, the HR for stroke between the original randomized treatment groups was nonsignifi-cant at 1.16 (95% CI, 0.83 to 1.61) and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Breast cancer. Breast cancer risk was originally reported to be increased significantly (by an additional 8 cases per 10,000 women per year) in the EPT arm versus the placebo arm with the nominal statistic, but this increase was nonsignificant after adjustment.¹ This risk estimate was revised in a follow-up publication 1 year after the original data were reported, and the increase in risk in the EPT arm was still no longer significant in the adjusted analysis.²¹ Importantly, another subsequent analysis that adjusted for baseline risk factors for breast cancer resulted in a further revision of the risk estimate, which again showed a nonsignificant increase in the EPT arm relative to the placebo arm.¹⁴ This is very important since it is commonly accepted that EPT increases the risk of breast cancer when this has not been definitively proven in any randomized controlled trial.

Unfortunately, the most recent breast cancer data¹⁴ (Table 1) were not used in the 2008 WHI E+P analysis.¹⁹ However, even using the unadjusted data in the 2.4-year open-label follow-up in which women were no longer on their randomized regimens, the HR for breast cancer between the original randomized treatment groups was nonsignificant and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Venous thromboembolism (VTE). EPT was associated with a doubling of the risk of VTE (ie, deep vein thrombosis and pulmonary embolism) compared with placebo, resulting in an excess of 18 VTE events per 10,000 women per year of therapy.¹⁵ The risk of VTE was significant across the entire cohort of women (mean age, 63 years).

In the 2008 WHI E+P analysis, the HR reported for VTE during the randomized phase was different than in previous reports—1.98 (95% CI, 1.52 to 2.59)¹⁹ versus 2.06 (95% CI, 1.57 to 2.70) reported in 2004¹⁵ (Table 1)—and the HR during the 2.4-year open-label follow-up, in which women were no longer on their randomized regimens, was no longer significant (HR = 0.95; 95% CI, 0.63 to 1.44). This change in the HR over time from the randomized phase to the open-label phase was statistically significant.¹⁹

Fracture. The risk of hip fracture was reduced by 33% with EPT relative to placebo, which was statistically significant in the nominal analysis but not in the adjusted analysis.¹⁷ The risk of any fracture was reduced by 24% in women randomized to EPT compared with placebo, which was statistically significant and translated to 47 fewer fractures per 10,000 women per year of therapy.¹⁷ Clinically, these results are most impressive given that women randomized in the WHI were not selected on the basis of risk for osteoporosis or fracture. This claim cannot be made for any other therapy.

In the 2008 WHI E+P analysis of the 2.4-year open-label follow-up in which women were no longer on their randomized regimens, the HR between the original randomized treatment groups was nonsignificant for hip fracture and any fracture—0.92 (95% CI, 0.64 to 1.34) and 0.91 (95% CI, 0.78 to 1.06), respectively—and the change in the HR over time from the randomized phase to the open-label phase was not significant for either fracture outcome.¹⁹

Diabetes. The risk of new-onset diabetes was reduced by a statistically significant 21% in women randomized to EPT compared with placebo.¹⁸

CHD. Importantly, no significant difference was found between the ET and placebo arms with respect to CHD events in the overall cohort of women, whose average age was 64 years.¹²

Stroke. The risk of stroke was greater with ET than with placebo in the nominal analysis, but importantly, the difference in event rates (11 per 10,000 women per year of therapy) failed to reach significance in the adjusted analysis.^11,12

Breast cancer. A strong but nonsignificant trend toward a reduction in breast cancer risk was apparent in the ET arm (8 fewer breast cancer cases per 10,000 women per year of therapy). Among women who actually were adherent to their study regimen (ie, consuming ≥ 80% of their study medication), there was a statistically significant 33% reduction in breast cancer risk with ET relative to placebo.²² Importantly, the reduction in breast cancer risk relative to placebo was found across all the age ranges studied.¹¹

VTE. The excess risk of VTE with ET versus placebo (32%) was less than the excess risk of VTE associated with EPT (Table 1). Importantly, the risk of VTE associated with ET was not statistically significant.^11,23

Fracture. The risk of any fracture (hip or vertebral) was reduced significantly in the ET arm compared with the placebo arm.¹¹

Diabetes. In a nominal analysis, there was a trend toward a reduction in the risk of new-onset diabetes in women randomized to ET relative to placebo, which nearly achieved statistical significance.²⁴

WHAT EXPLAINS THE DISCORDANCE BETWEEN OBSERVATIONAL AND RANDOMIZED TRIALS OF HT?

How can the perceived discordance between the results from observational studies and those from randomized controlled trials be explained? There are currently three hypotheses:

• The populations differ in the two types of study designs (observational studies and randomized controlled trials)
• The duration of HT use differs
• The timing of HT initiation differs in relation to age, time since menopause, and stage of atherosclerosis.

Population characteristics

One obvious difference between randomized trials and observational studies of HT is the presence of menopausal symptoms. To maintain blinding, women with hot flashes were predominantly excluded from randomized trials of HT, whereas the presence of hot flashes is the predominant menopausal symptom of women included in observational studies and the main reason women seek HT from their providers.

Other consistent and possibly explanatory differences between clinical trials and observational studies of HT are patient age at enrollment, years since menopause, and body mass index (BMI). Comparing randomized controlled trials with observational studies, age at enrollment was much higher in the clinical trials (mean age ≥ 63 years) than in the observational studies (range of 30 to 55 years). Similarly, women enrolled in randomized trials were more than 10 years beyond menopause, whereas those in observational studies were less than 5 years beyond menopause. In fact, more than 80% of HT users in observational studies initiated HT within 1 or 2 years of menopause.

Additionally, women in randomized trials of HT tend to have higher BMIs than their counterparts in observational studies. For example, mean BMI was considerably higher in the WHI randomized trials (28.5 kg/m² and 30.1 kg/m²)^1,11 than in the observational Nurses’ Health Study (25.8 kg/m²),²⁵ and a full third (34%) of women in the WHI randomized trials were severely obese (BMI ≥ 30 kg/m²).

This point about BMI is noteworthy in light of findings on the effect of BMI on breast cancer risk from an analysis of the WHI observational study among 85,917 women aged 50 to 79 years old at enrollment.²⁶ This analysis found that BMI was unrelated to breast cancer risk among women who had used HT; however, among nonusers of HT, a baseline BMI greater than 31.1 kg/m² was associated with a 2.52 relative risk of breast cancer compared with a baseline BMI less than 22.6 kg/m². The risk of breast cancer with increasing BMI was most pronounced in younger postmenopausal women. One interpretation is that high endogenous estrogen levels in postmenopausal women with an elevated BMI serve to increase breast cancer risk to a level beyond which HT adds no further risk. Alternatively, conjugated equine estrogens may act through a selective estrogen receptor modulator mechanism to block any potential adverse breast tissue effects of elevated endogenous estrone and estradiol levels.

Another hypothesis for the discordant findings between observational and randomized studies of HT focuses on differences in duration of HT use between the two types of studies. Duration of HT use has been substantially longer in observational studies, ranging from 10 years to 40 years, compared with no more than 7 or 8 years in randomized trials to date. Moreover, the results from observational studies have suggested that the longer the duration of HT use, the greater the benefit in terms of CHD risk.

For example, a case-control study by Chilvers et al showed that HT protected against nonfatal myocardial infarction only when used for more than 60 months.²⁷ Analysis of data on EPT use from the Heart and Estrogen/progestin Replacement Study (HERS),²⁸ the randomized WHI E+P trial,²⁰ and the WHI observational study⁹ reveals an interesting and consistent trend: rates of CHD events increased during the first year of EPT therapy (compared with placebo or nonuse) but then declined over time, ending up below the rates of CHD events in placebo recipients or nonusers of HT after approximately 5 years of therapy. This trend was not as pronounced with ET use in either the WHI randomized trial²⁹ or the WHI observational study,¹⁰ but the incidence of CHD events did decline over time with ET use in both studies, and in the WHI observational study, greater than 5 years of ET use was associated with a significant 27% reduction (HR, 0.73; 95% CI, 0.61 to 0.84) in CHD events compared with nonuse.¹⁰

Timing of HT initiation

A third hypothesis for the discordance between randomized and observational studies of HT concerns the timing of HT initiation relative to patient age, time since menopause, and stage of atherosclerosis. As noted above, women enrolled in randomized trials have tended to be considerably older and further from menopause compared with their counterparts in observational studies. The effects of differences in timing of HT initiation on specific clinical end points, particularly in relation to cardiovascular health, are reviewed below.

Reprinted, with permission, from JAMA (Rossouw JE, et al. JAMA 2007; 297: 1465–1477). Copyright © 2007 American Medical Association. All rights reserved.

Early and continued HT use may interrupt the pathogenic sequence of vascular aging, potentially preventing progression of atherosclerosis to the late stage at which plaque rupture and clinical events occur. In contrast, late intervention with HT may have little effect on established plaque and, in the case of EPT, may actually predispose to plaque rupture. This concept has been demonstrated in a pair of sister studies, the Women’s Estrogen-Progestin Lipid-Lowering Hormone Atherosclerosis Regression Trial (WELLHART)³¹ and the Estrogen Prevention Atherosclerosis Trial (EPAT).³²

Mortality. The same Stanford University researchers who performed the above meta-analysis of CHD events also assessed odd ratios for overall mortality in a meta-analysis of 30 randomized trials of HT versus placebo that included a total of 26,708 postmenopausal women (representing 119,118 patient-years).³³ They found the timing of HT initiation to have an effect on mortality similar to its effect on CHD events. In the overall cohort of women, the odds of death were not different between the HT and placebo groups, but among women younger than 60 years (mean, 54 years), those randomized to HT had a significant 39% reduction in the risk of death (HR = 0.61; 95% CI, 0.39 to 0.95) compared with those randomized to placebo. HT had no effect on mortality among women older than 60 years (mean, 66 years) in this analysis.

These mortality data are consistent with those from the WHI randomized trials, in which both EPT and ET were associated with a 30% reduction in overall mortality relative to placebo among women 50 to 59 years old.¹² When both the EPT and ET portions of the WHI were combined to increase the sample size, HT was associated with a significant 30% reduction in mortality (HR = 0.70; 95% CI, 0.51 to 0.96) compared with placebo among women 50 to 59 years old.¹²

Stroke. The most recent WHI data indicate that stroke is not increased with ET in women 50 to 59 years of age, as there were 2 fewer events per 10,000 women per year of ET relative to placebo.¹² In this same age group, the risk of stroke from EPT was increased by 5 events per 10,000 women per year of therapy relative to placebo.¹² Among women randomized within 5 years of menopause, stroke risk was increased by 3 events per 10,000 women per year of EPT relative to placebo.13

VTE. Although age was not a significant contributing factor to the risk of VTE from HT in the WHI, the absolute risk of VTE was lower in younger versus older women. The additional absolute risk for VTE events per 10,000 women per year of EPT use was 11 events for women 50 to 59 years old at randomization, 16 events for women 60 to 69 years old at randomization, and 35 events for women 70 to 79 years old at randomization.¹⁵ The additional absolute risk for VTE events per 10,000 women per year of ET use was 4 events for women 50 to 59 years old at randomization, 7 events for women 60 to 69 years old at randomization, and 11 events for women 70 to 79 years old at randomization.²³ A history of a prior thromboembolic event increases the risk of VTE with postmenopausal HT use, which should be considered before HT is initiated.

Comparative effects of lipid-lowering therapy and HT

Examination of the evidence regarding lipid-lowering therapy for prevention of CHD, as well as its effects on breast cancer risk and coronary artery calcium scores in women, can add some much-needed perspective and context to the HT data reviewed above.

CHD prevention. Walsh and Pignone examined six randomized controlled trials (N = 11,435) of primary prevention with lipid-lowering medication in women and found no significant effect on CHD events, nonfatal myocardial infarction, CHD mortality, or total mortality.³⁴ In eight randomized controlled trials of secondary prevention (N = 8,272), lipid-lowering therapy in women resulted in significant reductions in CHD end points but had no effect on total mortality.³⁴

Atherosclerosis progression. In three randomized controlled trials, 1 to 4 years of statin therapy had no effect on the progression of coronary artery calcium compared with placebo.^41–43 Among 1,064 women 50 to 59 years old who participated in a WHI substudy called the WHI Coronary Artery Calcium Study (WHI-CACS), those who were randomized to ET had significantly less coronary artery calcium at year 7 compared with those assigned to placebo.⁴⁴ The mean Agatston coronary artery calcium scores were 83.1 with ET versus 123.1 with placebo (P = .02).

Comparing risk of HT with risk of other therapies

Other comparisons yield similar conclusions.³⁵ For instance, a comparison of the ET arm of the WHI randomized trial with raloxifene in the Raloxifene Use for the Heart (RUTH) trial⁵² reveals similar effects on CHD, stroke, VTE, pulmonary embolism, deep vein thrombosis, and breast cancer, with the only large difference being a greater reduction in bone fracture risk with ET compared with raloxifene.³⁵ Finally, in putting the risk of VTE with HT into perspective, consider that the risk of thromboembolic events with selective estrogen receptor modulators⁵² and with the fibric acid derivative fenofibrate in diabetics⁵⁰ is of similar magnitude as the risk with HT.

Risk must be viewed in light of age at HT initiation

CONCLUSIONS FROM RANDOMIZED TRIALS

HT’s effects on CHD and mortality: Timing is everything

Cumulative data from randomized trials indicate that in the overall population of women studied, HT, aspirin, and lipid-lowering therapy each have a null effect on the incidence of CHD and mortality. However, within this overall null effect, early initiation of HT (in terms of time from menopause [< 10 years] and age at initiation [< 60 years]) is associated with reductions in total mortality and CHD incidence. Additionally, a duration of HT use beyond 5 years is associated with a reduction in the incidence of CHD.^{9,10,20,28,29} These effects of the timing and duration of therapy are unique to HT. Unopposed ET may have an advantageous profile relative to EPT for reducing the incidence of CHD and mortality in postmenopausal women.

Gaining perspective on risks

The risks of stroke and VTE associated with HT are low for women overall and lower still for women who are within 10 years of menopause or younger than age 60 when they start HT. With respect to stroke, fewer cases of stroke developed in users of ET compared with placebo in women who started HT before age 60. The risks of EPT are comparable to those of other medications commonly used in this population of women. More broadly, the magnitude and types of risk associated with HT are similar to those associated with other commonly used therapies.

In addition, the underappreciated benefits of HT, such as potential prevention of diabetes mellitus (15 fewer cases of incident diabetes per 10,000 women per year with EPT and 14 fewer cases with ET), need to be recognized and discussed with our patients. These data are consistent in both observational studies and randomized trials.

The bottom line

As data from randomized trials of HT accumulate, the results are clearly similar to those from the more than 20 observational studies indicating that young, symptomatic postmenopausal women who use HT for long periods have lower rates of CHD and total mortality compared with postmenopausal women who do not use HT. Consistent with these data, the American Association of Clinical Endocrinologists issued a position statement in 2008 concluding that for symptomatic menopausal women under the age of 60, the benefits of HT exceed the risks.⁵³

Nevertheless, the bottom line remains that the estrogen-cardioprotective hypothesis has yet to be studied, since randomized trials have not been conducted in the same population of women from which the hypothesis was generated. This hypothesis will be directly evaluated, however, in the ongoing Early versus Late Intervention Trial with Estradiol (ELITE). This randomized trial, funded by the National Institute on Aging, is designed to determine the effects of 17β-estradiol on the progression of atherosclerosis, cognition, and other postmenopausal health issues in recently menopausal (< 6 years) and remotely menopausal (≥ 10 years) women with no history of cardiovascular disease or diabetes. Until data from trials like ELITE emerge, guidelines such as those from the North American Menopause Society⁵⁴ (reviewed in the next article in this supplement) are reasonable for clinical practice.

Physicians and their women patients have faced a continuous, confusing mix of information about the risks and benefits of hormone therapy (HT) for perimenopausal and postmenopausal women, most of it without respect to age or the timing of HT relative to menopause. Initial data from the Women’s Health Initiative (WHI) estrogen + progestin (E+P) trial, a prevention study conducted predominantly in older postmenopausal women without menopausal symptoms,¹ resulted in questioning of the role of HT (unfortunately and inappropriately in younger symptomatic women). Cumulative trial data and further analyses of the WHI have refined and added to our understanding of the effects of HT, particularly with regard to cardiovascular health. This review will update physicians on the latest data on the risks and benefits of HT, with a particular focus on the heart, and will put the risks of HT into appropriate clinical context.

HORMONE THERAPY AND CARDIOVASCULAR DISEASE: A HISTORICAL PERSPECTIVE

Observational studies conducted prior to the WHI found consistently that women who self-selected to use HT had a reduction in mortality and in the incidence of cardiovascular disease relative to women who did not choose to use HT.^2–8 This reduction in risk was apparent whether the HT users had taken ET (estrogen therapy) or EPT (estrogen-progestogen therapy). In contrast, randomized controlled trials failed to confirm these findings from observational studies. However, the findings from randomized controlled trials were derived from older postmenopausal women who were many years past menopause. Often overlooked is the WHI observational study of ET and EPT,^9,10 in which women who chose to use HT had a reduction in the risk of coronary heart disease (CHD) similar to that observed among the HT users in other observational studies.

RECENT REPORTS FROM THE WHI

Since the original publication of the WHI E+P trial in 2002,¹ an extensive collection of data have been published in piecemeal fashion, contributing to the confusion and misperception of the effects of HT on risks and benefits. It is important to note that the WHI consists of both randomized and observational components, as detailed below, and that data have come from both. Together, these data help clarify the misperceptions generated from the first WHI report of 2002,¹ particularly misperceptions regarding the timing of HT initiation relative to menopause and the effect of HT duration on cardiovascular disease outcomes. In most instances, the conclusions drawn from this recent research run counter to the inaccurate but prevalent perception that HT use at any time and at any age is associated with cardiovascular harm, a perception that has unfortunately prevailed since the initial publication of the WHI E+P trial findings in 2002.

The WHI randomized trials and combined analysis

The WHI trials enrolled 27,347 postmenopausal women aged 50 to 79 years at baseline; almost two-thirds of the women enrolled were 60 years of age or older, and the majority of women were more than 10 years past menopause. WHI actually comprised two parallel randomized trials:

• One among 16,608 women who had not undergone hysterectomy (ie, with uterus intact), who were randomized to EPT or placebo (ie, WHI E+P trial) ¹
• One among 10,739 women who had undergone hysterectomy, who were randomized to ET or placebo.¹¹

Recent analyses from the WHI, published in 2007, assessed the cardiovascular effects of ET and EPT independently and combined, both overall and according to subject age and years since menopause when randomized.¹² Other analyses following the initial WHI E+P trial publication have analyzed the effects of HT according to duration of HT use and according to secondary end points. Many of these analyses have presented risks and benefits in terms of both nominal and adjusted confidence intervals (CIs). Nominal 95% CIs describe the variability in risk estimates that would arise from a simple trial for a single end point. Although nominal CIs are traditionally used, they do not take into account the multiple statistical testing issues (across time and across outcome categories) that occur in a trial. In contrast, adjusted 95% CIs correct for these stastical testing issues. From a clinical perspective, it is most appropriate to look at the adjusted CIs.

WHI: EPT vs placebo

CHD. Although the point estimate for CHD is increased, the 95% CI indicates that EPT has a non-significant effect on CHD outcome relative to placebo among all women randomized in the WHI E+P trial (mean age, 63 years).¹² This is a very important point for cardiologists and primary care physicians to note.

In a 2008 analysis of the WHI E+P trial that included a 2.4-year open-label follow-up subsequent to the randomized trial,¹⁹ the randomized trial data reported were again different than in previous reports, but remained nonsignificant. The hazard ratio (HR) reported for CHD in the 2008 analysis for the randomized portion of the trial was 1.22 (95% CI, 0.99 to 1.51), as compared with 1.23 (95% CI, 0.99 to 1.53) reported in 2007¹² (Table 1), 1.24 (95% CI, 1.00 to 1.54) reported in 2003,²⁰ and 1.29 (95% CI, 1.02 to 1.63) reported in 2002.¹ In the 2.4-year open-label follow-up period in which women were no longer on their randomized regimens (EPT or placebo), the HR for CHD between the original randomized treatment groups was nonsignificant at 0.95 (95% CI, 0.73 to 1.26) and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Stroke. The risk of stroke was increased significantly (by an additional 8 events per 10,000 women treated per year) in the EPT arm versus the placebo arm in the nominal analysis,¹² but this difference was nonsignificant after adjustment.¹³

In the 2008 WHI E+P analysis, the HR reported for stroke during the randomized trial phase was different than in previous reports—1.34 (95% CI, 1.05 to 1.71)¹⁹ versus 1.31 (95% CI, 1.03 to 1.68) reported in 2007¹² (Table 1)—and the adjusted analysis was not reported. In the 2.4-year open-label follow-up period in which women were no longer on their randomized regimens, the HR for stroke between the original randomized treatment groups was nonsignifi-cant at 1.16 (95% CI, 0.83 to 1.61) and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Breast cancer. Breast cancer risk was originally reported to be increased significantly (by an additional 8 cases per 10,000 women per year) in the EPT arm versus the placebo arm with the nominal statistic, but this increase was nonsignificant after adjustment.¹ This risk estimate was revised in a follow-up publication 1 year after the original data were reported, and the increase in risk in the EPT arm was still no longer significant in the adjusted analysis.²¹ Importantly, another subsequent analysis that adjusted for baseline risk factors for breast cancer resulted in a further revision of the risk estimate, which again showed a nonsignificant increase in the EPT arm relative to the placebo arm.¹⁴ This is very important since it is commonly accepted that EPT increases the risk of breast cancer when this has not been definitively proven in any randomized controlled trial.

Unfortunately, the most recent breast cancer data¹⁴ (Table 1) were not used in the 2008 WHI E+P analysis.¹⁹ However, even using the unadjusted data in the 2.4-year open-label follow-up in which women were no longer on their randomized regimens, the HR for breast cancer between the original randomized treatment groups was nonsignificant and the change in the HR over time from the randomized phase to the open-label phase was not significant.¹⁹

Venous thromboembolism (VTE). EPT was associated with a doubling of the risk of VTE (ie, deep vein thrombosis and pulmonary embolism) compared with placebo, resulting in an excess of 18 VTE events per 10,000 women per year of therapy.¹⁵ The risk of VTE was significant across the entire cohort of women (mean age, 63 years).

In the 2008 WHI E+P analysis, the HR reported for VTE during the randomized phase was different than in previous reports—1.98 (95% CI, 1.52 to 2.59)¹⁹ versus 2.06 (95% CI, 1.57 to 2.70) reported in 2004¹⁵ (Table 1)—and the HR during the 2.4-year open-label follow-up, in which women were no longer on their randomized regimens, was no longer significant (HR = 0.95; 95% CI, 0.63 to 1.44). This change in the HR over time from the randomized phase to the open-label phase was statistically significant.¹⁹

Fracture. The risk of hip fracture was reduced by 33% with EPT relative to placebo, which was statistically significant in the nominal analysis but not in the adjusted analysis.¹⁷ The risk of any fracture was reduced by 24% in women randomized to EPT compared with placebo, which was statistically significant and translated to 47 fewer fractures per 10,000 women per year of therapy.¹⁷ Clinically, these results are most impressive given that women randomized in the WHI were not selected on the basis of risk for osteoporosis or fracture. This claim cannot be made for any other therapy.

In the 2008 WHI E+P analysis of the 2.4-year open-label follow-up in which women were no longer on their randomized regimens, the HR between the original randomized treatment groups was nonsignificant for hip fracture and any fracture—0.92 (95% CI, 0.64 to 1.34) and 0.91 (95% CI, 0.78 to 1.06), respectively—and the change in the HR over time from the randomized phase to the open-label phase was not significant for either fracture outcome.¹⁹

Diabetes. The risk of new-onset diabetes was reduced by a statistically significant 21% in women randomized to EPT compared with placebo.¹⁸

CHD. Importantly, no significant difference was found between the ET and placebo arms with respect to CHD events in the overall cohort of women, whose average age was 64 years.¹²

Stroke. The risk of stroke was greater with ET than with placebo in the nominal analysis, but importantly, the difference in event rates (11 per 10,000 women per year of therapy) failed to reach significance in the adjusted analysis.^11,12

Breast cancer. A strong but nonsignificant trend toward a reduction in breast cancer risk was apparent in the ET arm (8 fewer breast cancer cases per 10,000 women per year of therapy). Among women who actually were adherent to their study regimen (ie, consuming ≥ 80% of their study medication), there was a statistically significant 33% reduction in breast cancer risk with ET relative to placebo.²² Importantly, the reduction in breast cancer risk relative to placebo was found across all the age ranges studied.¹¹

VTE. The excess risk of VTE with ET versus placebo (32%) was less than the excess risk of VTE associated with EPT (Table 1). Importantly, the risk of VTE associated with ET was not statistically significant.^11,23

Fracture. The risk of any fracture (hip or vertebral) was reduced significantly in the ET arm compared with the placebo arm.¹¹

Diabetes. In a nominal analysis, there was a trend toward a reduction in the risk of new-onset diabetes in women randomized to ET relative to placebo, which nearly achieved statistical significance.²⁴

WHAT EXPLAINS THE DISCORDANCE BETWEEN OBSERVATIONAL AND RANDOMIZED TRIALS OF HT?

How can the perceived discordance between the results from observational studies and those from randomized controlled trials be explained? There are currently three hypotheses:

• The populations differ in the two types of study designs (observational studies and randomized controlled trials)
• The duration of HT use differs
• The timing of HT initiation differs in relation to age, time since menopause, and stage of atherosclerosis.

Population characteristics

One obvious difference between randomized trials and observational studies of HT is the presence of menopausal symptoms. To maintain blinding, women with hot flashes were predominantly excluded from randomized trials of HT, whereas the presence of hot flashes is the predominant menopausal symptom of women included in observational studies and the main reason women seek HT from their providers.

Other consistent and possibly explanatory differences between clinical trials and observational studies of HT are patient age at enrollment, years since menopause, and body mass index (BMI). Comparing randomized controlled trials with observational studies, age at enrollment was much higher in the clinical trials (mean age ≥ 63 years) than in the observational studies (range of 30 to 55 years). Similarly, women enrolled in randomized trials were more than 10 years beyond menopause, whereas those in observational studies were less than 5 years beyond menopause. In fact, more than 80% of HT users in observational studies initiated HT within 1 or 2 years of menopause.

Additionally, women in randomized trials of HT tend to have higher BMIs than their counterparts in observational studies. For example, mean BMI was considerably higher in the WHI randomized trials (28.5 kg/m² and 30.1 kg/m²)^1,11 than in the observational Nurses’ Health Study (25.8 kg/m²),²⁵ and a full third (34%) of women in the WHI randomized trials were severely obese (BMI ≥ 30 kg/m²).

This point about BMI is noteworthy in light of findings on the effect of BMI on breast cancer risk from an analysis of the WHI observational study among 85,917 women aged 50 to 79 years old at enrollment.²⁶ This analysis found that BMI was unrelated to breast cancer risk among women who had used HT; however, among nonusers of HT, a baseline BMI greater than 31.1 kg/m² was associated with a 2.52 relative risk of breast cancer compared with a baseline BMI less than 22.6 kg/m². The risk of breast cancer with increasing BMI was most pronounced in younger postmenopausal women. One interpretation is that high endogenous estrogen levels in postmenopausal women with an elevated BMI serve to increase breast cancer risk to a level beyond which HT adds no further risk. Alternatively, conjugated equine estrogens may act through a selective estrogen receptor modulator mechanism to block any potential adverse breast tissue effects of elevated endogenous estrone and estradiol levels.

Another hypothesis for the discordant findings between observational and randomized studies of HT focuses on differences in duration of HT use between the two types of studies. Duration of HT use has been substantially longer in observational studies, ranging from 10 years to 40 years, compared with no more than 7 or 8 years in randomized trials to date. Moreover, the results from observational studies have suggested that the longer the duration of HT use, the greater the benefit in terms of CHD risk.

For example, a case-control study by Chilvers et al showed that HT protected against nonfatal myocardial infarction only when used for more than 60 months.²⁷ Analysis of data on EPT use from the Heart and Estrogen/progestin Replacement Study (HERS),²⁸ the randomized WHI E+P trial,²⁰ and the WHI observational study⁹ reveals an interesting and consistent trend: rates of CHD events increased during the first year of EPT therapy (compared with placebo or nonuse) but then declined over time, ending up below the rates of CHD events in placebo recipients or nonusers of HT after approximately 5 years of therapy. This trend was not as pronounced with ET use in either the WHI randomized trial²⁹ or the WHI observational study,¹⁰ but the incidence of CHD events did decline over time with ET use in both studies, and in the WHI observational study, greater than 5 years of ET use was associated with a significant 27% reduction (HR, 0.73; 95% CI, 0.61 to 0.84) in CHD events compared with nonuse.¹⁰

Timing of HT initiation

A third hypothesis for the discordance between randomized and observational studies of HT concerns the timing of HT initiation relative to patient age, time since menopause, and stage of atherosclerosis. As noted above, women enrolled in randomized trials have tended to be considerably older and further from menopause compared with their counterparts in observational studies. The effects of differences in timing of HT initiation on specific clinical end points, particularly in relation to cardiovascular health, are reviewed below.

Reprinted, with permission, from JAMA (Rossouw JE, et al. JAMA 2007; 297: 1465–1477). Copyright © 2007 American Medical Association. All rights reserved.

Early and continued HT use may interrupt the pathogenic sequence of vascular aging, potentially preventing progression of atherosclerosis to the late stage at which plaque rupture and clinical events occur. In contrast, late intervention with HT may have little effect on established plaque and, in the case of EPT, may actually predispose to plaque rupture. This concept has been demonstrated in a pair of sister studies, the Women’s Estrogen-Progestin Lipid-Lowering Hormone Atherosclerosis Regression Trial (WELLHART)³¹ and the Estrogen Prevention Atherosclerosis Trial (EPAT).³²

Mortality. The same Stanford University researchers who performed the above meta-analysis of CHD events also assessed odd ratios for overall mortality in a meta-analysis of 30 randomized trials of HT versus placebo that included a total of 26,708 postmenopausal women (representing 119,118 patient-years).³³ They found the timing of HT initiation to have an effect on mortality similar to its effect on CHD events. In the overall cohort of women, the odds of death were not different between the HT and placebo groups, but among women younger than 60 years (mean, 54 years), those randomized to HT had a significant 39% reduction in the risk of death (HR = 0.61; 95% CI, 0.39 to 0.95) compared with those randomized to placebo. HT had no effect on mortality among women older than 60 years (mean, 66 years) in this analysis.

These mortality data are consistent with those from the WHI randomized trials, in which both EPT and ET were associated with a 30% reduction in overall mortality relative to placebo among women 50 to 59 years old.¹² When both the EPT and ET portions of the WHI were combined to increase the sample size, HT was associated with a significant 30% reduction in mortality (HR = 0.70; 95% CI, 0.51 to 0.96) compared with placebo among women 50 to 59 years old.¹²

Stroke. The most recent WHI data indicate that stroke is not increased with ET in women 50 to 59 years of age, as there were 2 fewer events per 10,000 women per year of ET relative to placebo.¹² In this same age group, the risk of stroke from EPT was increased by 5 events per 10,000 women per year of therapy relative to placebo.¹² Among women randomized within 5 years of menopause, stroke risk was increased by 3 events per 10,000 women per year of EPT relative to placebo.13

VTE. Although age was not a significant contributing factor to the risk of VTE from HT in the WHI, the absolute risk of VTE was lower in younger versus older women. The additional absolute risk for VTE events per 10,000 women per year of EPT use was 11 events for women 50 to 59 years old at randomization, 16 events for women 60 to 69 years old at randomization, and 35 events for women 70 to 79 years old at randomization.¹⁵ The additional absolute risk for VTE events per 10,000 women per year of ET use was 4 events for women 50 to 59 years old at randomization, 7 events for women 60 to 69 years old at randomization, and 11 events for women 70 to 79 years old at randomization.²³ A history of a prior thromboembolic event increases the risk of VTE with postmenopausal HT use, which should be considered before HT is initiated.

Comparative effects of lipid-lowering therapy and HT

Examination of the evidence regarding lipid-lowering therapy for prevention of CHD, as well as its effects on breast cancer risk and coronary artery calcium scores in women, can add some much-needed perspective and context to the HT data reviewed above.

CHD prevention. Walsh and Pignone examined six randomized controlled trials (N = 11,435) of primary prevention with lipid-lowering medication in women and found no significant effect on CHD events, nonfatal myocardial infarction, CHD mortality, or total mortality.³⁴ In eight randomized controlled trials of secondary prevention (N = 8,272), lipid-lowering therapy in women resulted in significant reductions in CHD end points but had no effect on total mortality.³⁴

Atherosclerosis progression. In three randomized controlled trials, 1 to 4 years of statin therapy had no effect on the progression of coronary artery calcium compared with placebo.^41–43 Among 1,064 women 50 to 59 years old who participated in a WHI substudy called the WHI Coronary Artery Calcium Study (WHI-CACS), those who were randomized to ET had significantly less coronary artery calcium at year 7 compared with those assigned to placebo.⁴⁴ The mean Agatston coronary artery calcium scores were 83.1 with ET versus 123.1 with placebo (P = .02).

Comparing risk of HT with risk of other therapies

Other comparisons yield similar conclusions.³⁵ For instance, a comparison of the ET arm of the WHI randomized trial with raloxifene in the Raloxifene Use for the Heart (RUTH) trial⁵² reveals similar effects on CHD, stroke, VTE, pulmonary embolism, deep vein thrombosis, and breast cancer, with the only large difference being a greater reduction in bone fracture risk with ET compared with raloxifene.³⁵ Finally, in putting the risk of VTE with HT into perspective, consider that the risk of thromboembolic events with selective estrogen receptor modulators⁵² and with the fibric acid derivative fenofibrate in diabetics⁵⁰ is of similar magnitude as the risk with HT.

Risk must be viewed in light of age at HT initiation

CONCLUSIONS FROM RANDOMIZED TRIALS

HT’s effects on CHD and mortality: Timing is everything

Cumulative data from randomized trials indicate that in the overall population of women studied, HT, aspirin, and lipid-lowering therapy each have a null effect on the incidence of CHD and mortality. However, within this overall null effect, early initiation of HT (in terms of time from menopause [< 10 years] and age at initiation [< 60 years]) is associated with reductions in total mortality and CHD incidence. Additionally, a duration of HT use beyond 5 years is associated with a reduction in the incidence of CHD.^{9,10,20,28,29} These effects of the timing and duration of therapy are unique to HT. Unopposed ET may have an advantageous profile relative to EPT for reducing the incidence of CHD and mortality in postmenopausal women.

Gaining perspective on risks

The risks of stroke and VTE associated with HT are low for women overall and lower still for women who are within 10 years of menopause or younger than age 60 when they start HT. With respect to stroke, fewer cases of stroke developed in users of ET compared with placebo in women who started HT before age 60. The risks of EPT are comparable to those of other medications commonly used in this population of women. More broadly, the magnitude and types of risk associated with HT are similar to those associated with other commonly used therapies.

In addition, the underappreciated benefits of HT, such as potential prevention of diabetes mellitus (15 fewer cases of incident diabetes per 10,000 women per year with EPT and 14 fewer cases with ET), need to be recognized and discussed with our patients. These data are consistent in both observational studies and randomized trials.

The bottom line

As data from randomized trials of HT accumulate, the results are clearly similar to those from the more than 20 observational studies indicating that young, symptomatic postmenopausal women who use HT for long periods have lower rates of CHD and total mortality compared with postmenopausal women who do not use HT. Consistent with these data, the American Association of Clinical Endocrinologists issued a position statement in 2008 concluding that for symptomatic menopausal women under the age of 60, the benefits of HT exceed the risks.⁵³

Nevertheless, the bottom line remains that the estrogen-cardioprotective hypothesis has yet to be studied, since randomized trials have not been conducted in the same population of women from which the hypothesis was generated. This hypothesis will be directly evaluated, however, in the ongoing Early versus Late Intervention Trial with Estradiol (ELITE). This randomized trial, funded by the National Institute on Aging, is designed to determine the effects of 17β-estradiol on the progression of atherosclerosis, cognition, and other postmenopausal health issues in recently menopausal (< 6 years) and remotely menopausal (≥ 10 years) women with no history of cardiovascular disease or diabetes. Until data from trials like ELITE emerge, guidelines such as those from the North American Menopause Society⁵⁴ (reviewed in the next article in this supplement) are reasonable for clinical practice.

In March 2007, the North American Menopause Society (NAMS) issued an updated evidence-based position statement on the risks and benefits of hormone therapy (HT) in peri- and post-menopausal women.¹ This article will briefly review the major conclusions of that position statement and review three new reports from the Women’s Health Initiative (WHI) published after the NAMS position statement.^2–4 The objective is to update clinicians on current recommendations on the use of HT and to assess, together with the preceding article in this supplement by Hodis, emerging data that will inform future recommendations.

TAKE-HOME POINTS FROM THE UPDATED NAMS POSITION STATEMENT

The 2007 NAMS position statement on HT was developed by 14 expert clinicians and researchers who used previous NAMS position statements on the topic from 2002, 2003, and 2004 as a basis. The experts then reviewed all relevant subsequent evidence from a comprehensive literature search to determine areas of consensus and nonconsensus. Twenty-four areas of consensus and two areas of nonconsensus were identified, which represented a clear increase in consensus relative to the prior NAMS position statements. Thirty-two areas were identified as requiring further research.

Key recommendations and observations from the 2007 NAMS position statement are cited below, in many cases verbatim or near verbatim to preserve the intent.¹

Highlights of the NAMS position statement

Terminology

NAMS proposes adoption of the following terminology:

• Estrogen therapy (ET) for use of unopposed estrogen
• Estrogen-progestogen therapy (EPT) for combined use of estrogen and progestogen
• Hormone therapy (HT) to include both ET and EPT
• Progestogen to include both progesterone and progestins.

Indications for HT

• Treatment of moderate to severe vasomotor symptoms
• Treatment of moderate to severe vulvovaginal symptoms. When ET is being used only for this symptom, NAMS recommends local (vaginal) delivery.

Use of a progestogen

• Because the primary purpose of progestogen use is to prevent the endometrial cancer associated with unopposed estrogen, only women with a uterus should take a progestogen along with estrogen.
• Lack of endometrial safety data prevents NAMS from recommending long-cycle progestogen (eg, 12 to 14 days every 3 to 6 months), progestogen intrauterine systems, or low-dose estrogen without progestogen.
• A progestogen is usually not necessary with use of low-dose vaginal estrogen.⁵ However, separate from the NAMS statement is a Cochrane review of the use of vaginally administered estrogens that recommends further investigation of long-term endometrial safety with use of vaginal estrogen beyond 6 months.⁶

Cardiac and cerebrovascular disease

• HT is not recommended for prevention of coronary heart disease (CHD) at any age, pending new data to the contrary.
• HT should not be used for prevention of stroke and should be discouraged in women who have an increased risk of stroke.

Venous thromboembolism

• HT increases the risk of venous thrombosis and venous thromboembolism (VTE).

Diabetes mellitus

• Both ET and EPT reduced the risk of incident diabetes mellitus requiring treatment (by 12% and 21%, respectively, relative to placebo in the WHI).
• Evidence is insufficient to recommend HT solely for the prevention of diabetes mellitus.

Breast cancer

• EPT increased the risk of breast cancer in the WHI, but ET did not.
• Both ET and EPT increase breast cell proliferation, breast pain, and mammographic density. Diagnosis of cancer may be delayed.

Osteoporosis

• Both ET and EPT reduce the risk of postmenopausal fractures.
• HT is an option for reducing the risk of osteoporosis after the risks and benefits are weighed against those of other therapies.

Premature menopause and premature ovarian failure

• The absolute risks posed by ET and EPT may be lower in women with premature menopause or premature ovarian failure because of the lower incidence of CHD, stroke, and VTE in younger women. The risk-benefit ratio of HT is likely to be more favorable in this younger age group, but this has not yet been demonstrated.

Extended use of HT

• Extended use of HT may be considered in women who decide that menopausal symptom relief out­weighs the risks of HT, particularly after an attempt to stop HT has failed.
• Extended use may be appropriate for women with vasomotor symptoms at high risk of osteoporosis-related fracture.
• Extended use may be considered for the prevention of further bone loss in women with established low bone mass when other therapies are contraindicated or are not well tolerated.

Caution on use of “bioidentical” hormones

• In the absence of further data, compounded “bioidentical” hormone preparations should be presumed to carry the known risks and benefits of HT.
• The lack of regulatory oversight with regard to purity and consistency of bioidenticals prompts caution in their use.

Areas of nonconsensus

The NAMS panel did not reach consensus on the best way to discontinue HT or on the relative safety of continuous versus sequential use of progestogen along with estrogen. Lack of data and conflicting data prevented consensus in these two areas.

Since the publication of the 2007 NAMS position statement, three additional important analyses have emerged from the WHI randomized trial. The first report, by Rossouw et al, examined the effects of HT on the risk of cardiovascular disease (CVD) and other outcomes according to age and time since menopause.² The second analysis, by Manson et al, was a post hoc study of the extent of coronary artery calcification in the 50- to 59-year-old group in the ET arm of the WHI.³ The third analysis, by Heiss et al, reported health outcomes at 2.4 years after treatment was stopped in the EPT arm of the WHI.⁴

Cardiovascular results

The analysis by Rossouw et al of the CVD effects of HT by age and years since menopause has been reported in detail in the preceding paper by Hodis. In brief, the authors concluded that the data confirm a very low risk for women in their 50s who use HT for menopausal symptoms. The authors cautioned that the low risk from ages 50 to 59 does not guarantee lack of harm with prolonged use into older ages. Stroke and thrombosis risk were not dependent on years since menopause.

The WHI Coronary Artery Calcium Study was conducted in the ET arm approximately 1.3 years after the intervention (ET or placebo) was discontinued.3 Participants had completed a mean 7.4 years of intervention. The 1,064 eligible and available participants were aged 50 to 59 at WHI baseline. They underwent computed tomography of the heart. More than half of the women had a coronary artery calcium (CAC) score of 0. Overall, the mean CAC score was 83.1 among those randomized to ET and 123.1 among those randomized to placebo (P = .02). Classic CVD risk factors such as smoking, diabetes, hypertension, and high cholesterol were also associated with increased CAC scores, but they did not significantly modify the effect of ET on CAC. The authors cautioned that because of the multiple and complex effects of estrogen in the cardiovascular system, further study should be completed before ET is used for prevention of CAC.

Postintervention assessment

The third key HT-related WHI paper⁴ published since the 2007 NAMS position statement is the first to report the health events that occurred in the EPT arm since discontinuation of the study drugs.

Design and end points. The intervention phase of the WHI trial of EPT included 16,608 postmenopausal women (with intact uterus) aged 50 to 79 years at baseline who were randomized to treatment with EPT or placebo for a mean of 5.6 years before the treatment intervention was discontinued in July 2002 because the overall health risks of EPT were found to exceed the health benefits. Of these original participants, 15,730 women (95%) completed a planned postintervention follow-up consisting of semi-annual monitoring for adjudicated outcomes from July 2002 through March 2005. The mean duration of postintervention follow-up was 2.4 years.⁴

The primary outcomes of this postintervention analysis were CVD events and invasive breast cancer. These end points, together with endometrial cancer, colorectal cancer, stroke, pulmonary embolism, hip fracture, and death, were also factored into a global index of risks versus benefits with EPT.

Results. CVD. There was neither an elevated risk nor a decreased risk of CHD after discontinuation of EPT (hazard ratio [HR] = 0.95; 95% CI, 0.73 to 1.26). Risks that were elevated in the intervention phase, such as deep vein thrombosis (DVT) and stroke, disappeared after EPT was stopped. Women originally randomized to EPT had a similar rate of all CVD events compared with those who had been randomized to placebo (HR = 1.04; 95% CI, 0.89 to 1.21).

Breast and other cancers. The annualized incidence of invasive breast cancer in the postintervention period was 0.42% in the group that had been randomized to EPT versus 0.33% in the group randomized to placebo. This translated to a nonsignificant HR of 1.27 (95% CI, 0.91 to 1.78) for the postintervention period. In contrast, the annualized rate of all cancers (endometrial, colorectal, and breast combined) in the postintervention period was significantly higher in the EPT group (1.56%) than in the placebo group (1.26%) (HR = 1.24; 95% CI, 1.04 to 1.48). More extensive analysis of this finding is under way.

The cancer findings from this 2.4-year postintervention phase of the WHI parallel those from the 2.7-year postintervention phase of the Heart and Estrogen/progestin Replacement Study (HERS), in which between-group differences in the rates of breast and colon cancers approached null as the incidences of lung and other cancers increased in the group that had been randomized to EPT.⁷

Fractures. During the intervention phase, EPT was associated with a significant reduction in fractures; however, the EPT fracture benefit disappeared within the 2.4-year follow-up period.

Mortality. There was little difference in all-cause mortality between the treatment groups in the intervention phase. Although the difference was not statistically significant, there was a 15% higher mortality rate in the EPT group during the postintervention phase (HR = 1.15; 95% CI, 0.95 to 1.39). Contributing in part to this difference was an increased risk of mortality from lung cancer that requires further exploration.

Global index. The global index of risks versus benefits from enrollment to the present analysis remained significantly elevated (HR = 1.12; 95% CI, 1.03 to 1.21), suggesting more risk than benefit from use of EPT. The increase in the global index loses significance when the postintervention phase is considered alone (HR = 1.11; 95% CI, 0.99 to 1.27).

Conclusions. The researchers concluded that a number of outcome patterns observed with EPT in the intervention period of the WHI randomized trial did not persist during the 3-year postintervention follow-up:

• CHD, DVT, and stroke risks disappeared
• Hip fracture and total fracture benefits disappeared
• The composite risk of all cancers increased and was statistically significant in the postintervention phase, although the elevated risk of breast cancer was no longer statistically significant in the postintervention phase.

Summary of the new WHI reports

These three recent papers from the WHI suggest lower risks with short-term use of EPT in women ages 50 to 59 compared with older women. A delay in atherosclerosis and a decrease in all-cause mortality were also noted in this age group. The postintervention follow-up findings of a rapid disappearance of most risks and benefits of EPT will be of interest to patients who want to know what to expect when they discontinue HT. The late development (in years 5 to 8) of an increase in the composite risk of all cancers merits further investigation.

In March 2007, the North American Menopause Society (NAMS) issued an updated evidence-based position statement on the risks and benefits of hormone therapy (HT) in peri- and post-menopausal women.¹ This article will briefly review the major conclusions of that position statement and review three new reports from the Women’s Health Initiative (WHI) published after the NAMS position statement.^2–4 The objective is to update clinicians on current recommendations on the use of HT and to assess, together with the preceding article in this supplement by Hodis, emerging data that will inform future recommendations.

TAKE-HOME POINTS FROM THE UPDATED NAMS POSITION STATEMENT

The 2007 NAMS position statement on HT was developed by 14 expert clinicians and researchers who used previous NAMS position statements on the topic from 2002, 2003, and 2004 as a basis. The experts then reviewed all relevant subsequent evidence from a comprehensive literature search to determine areas of consensus and nonconsensus. Twenty-four areas of consensus and two areas of nonconsensus were identified, which represented a clear increase in consensus relative to the prior NAMS position statements. Thirty-two areas were identified as requiring further research.

Key recommendations and observations from the 2007 NAMS position statement are cited below, in many cases verbatim or near verbatim to preserve the intent.¹

Highlights of the NAMS position statement

Terminology

NAMS proposes adoption of the following terminology:

• Estrogen therapy (ET) for use of unopposed estrogen
• Estrogen-progestogen therapy (EPT) for combined use of estrogen and progestogen
• Hormone therapy (HT) to include both ET and EPT
• Progestogen to include both progesterone and progestins.

Indications for HT

• Treatment of moderate to severe vasomotor symptoms
• Treatment of moderate to severe vulvovaginal symptoms. When ET is being used only for this symptom, NAMS recommends local (vaginal) delivery.

Use of a progestogen

• Because the primary purpose of progestogen use is to prevent the endometrial cancer associated with unopposed estrogen, only women with a uterus should take a progestogen along with estrogen.
• Lack of endometrial safety data prevents NAMS from recommending long-cycle progestogen (eg, 12 to 14 days every 3 to 6 months), progestogen intrauterine systems, or low-dose estrogen without progestogen.
• A progestogen is usually not necessary with use of low-dose vaginal estrogen.⁵ However, separate from the NAMS statement is a Cochrane review of the use of vaginally administered estrogens that recommends further investigation of long-term endometrial safety with use of vaginal estrogen beyond 6 months.⁶

Cardiac and cerebrovascular disease

• HT is not recommended for prevention of coronary heart disease (CHD) at any age, pending new data to the contrary.
• HT should not be used for prevention of stroke and should be discouraged in women who have an increased risk of stroke.

Venous thromboembolism

• HT increases the risk of venous thrombosis and venous thromboembolism (VTE).

Diabetes mellitus

• Both ET and EPT reduced the risk of incident diabetes mellitus requiring treatment (by 12% and 21%, respectively, relative to placebo in the WHI).
• Evidence is insufficient to recommend HT solely for the prevention of diabetes mellitus.

Breast cancer

• EPT increased the risk of breast cancer in the WHI, but ET did not.
• Both ET and EPT increase breast cell proliferation, breast pain, and mammographic density. Diagnosis of cancer may be delayed.

Osteoporosis

• Both ET and EPT reduce the risk of postmenopausal fractures.
• HT is an option for reducing the risk of osteoporosis after the risks and benefits are weighed against those of other therapies.

Premature menopause and premature ovarian failure

• The absolute risks posed by ET and EPT may be lower in women with premature menopause or premature ovarian failure because of the lower incidence of CHD, stroke, and VTE in younger women. The risk-benefit ratio of HT is likely to be more favorable in this younger age group, but this has not yet been demonstrated.