Steven H. Woolf, MD, MPH

In January 2000 an article in The Lancet drew attention when it questioned the supporting evidence for screening mammography.¹ Danish investigators Peter Gøtzsche and Ole Olsen presented a series of apparent flaws in the 8 randomized trials of mammography, ultimately concluding that screening is unjustified. Their cogent arguments and the press coverage they received left many physicians wondering whether they should continue to order mammograms. The story led the CBS Evening News² and was featured in the Washington Post,³ Time,⁴ and Reuters.⁵

A Patient-Oriented Evidence that Matters (POEM) review in the April 2000 issue of The Journal of Family Practice⁶ that addressed The Lancet study lent apparent support to these concerns. The POEM related the arguments in The Lancet article without challenging them and concluded that “mammography screening has never been shown to help women to live longer.” The authors of this POEM suggested that the only reasons for screening to continue are “politics, patients’ preconceptions, and the fear of litigation.” Unlike most POEMs, this one included no critical appraisal of the methods or assumptions of the reviewed study. This lack of comment, combined with the authors’ negative remarks about mammography, may have convinced family physicians that the criticisms of Gøtzsche and Olsen were beyond dispute.

However, controversy does surround their arguments, as the many letters to the editor published in The Lancet attest.⁷ For example, The Lancet critique made much of inconsistent sample sizes and baseline dissimilarities between screened and unscreened women. The authors asserted that such age and socioeconomic differences were “incompatible with adequate randomization.” That premise is contestable. It is normal and predictable that a proportion of population variables will differ between groups for statistical reasons, no matter how perfect the randomization. Also, the observed age difference (1 to 6 months) would not explain the 21% reduction in mortality observed in the trials.⁸

For Gøtzsche and Olsen the discrepant age patterns and sample sizes were less a cause of the results than a warning sign that randomization had been subverted (because of failure to conceal allocation). Since mortality in the screened and unscreened groups differed by only a relatively small number of deaths, they reasoned that very little bias would be necessary to tip the scales in favor of mammography.

Several arguments weaken their case, however. First, they offered no evidence that subversion or unconcealed allocation actually occurred. They equated inexplicit documentation of procedures (and dissimilar group characteristics) with improper randomization. Second, even if unconcealed allocation occurred, it does not in itself thwart randomization. Investigators who know to which group a patient will be assigned can still follow the rules and make the correct assignment. Anecdotal reports of subversion (by deciphering assignment sequences to divert or target patients for allocation) do not offer denominator data to assess how often this occurs.⁹ It would have had to occur in every trial that favored mammography to uphold the authors’ allegations. Third, even if the trials were subverted there is no indication that case mix differed enough to skew outcomes. Age differences were minor; the authors speculated that sizable imbalances in unmeasured factors could have altered results, but they gave no evidence. They cited reports that poorly concealed allocation is associated with a 37% to 41% exaggeration in odds ratios,^10,11 but these reports concerned other trials and made arguable assumptions. Finally, their confirmatory finding—that only the 6 “flawed” trials reported a benefit for mammography and that the 2 acceptable trials showed no effect—was based on recalculated relative risk rates. The original trial data show no such pattern.⁸

This is not to suggest that weaknesses in the mammography trials do not merit scrutiny. Others have also voiced criticisms.¹² But the alarm raised by Gøtzsche and Olsen goes further, compelling us to rethink the purpose of critical appraisal and the extremes at which it might cause more harm than good.

Excessive critical appraisal

We seek perfection in evidence to safeguard patients. Prematurely adopting (or abandoning) interventions through uncritical acceptance of findings risks overlooking potential harms or more effective alternatives. But critical appraisal can do harm if valid evidence is rejected. Deciding whether to accept evidence counterbalances the risks of acceptance against the risks of rejection, which are inversely related. At one extreme of the spectrum, where data are accepted on face value (no appraisal), the risk of a type I error (accepting evidence of efficacy when the intervention does not work or causes harm) is high, and that of a type II error (discarding evidence when the intervention actually works) is low. At the other extreme (excessive scrutiny) the risk of a type II error is great; such errors harm patients because knowledge is rejected that can save (or improve) lives. Obviously, patients are best served somewhere in the middle, striking an optimal balance between the risks of type I and type II errors.

Enthusiasts for critical appraisal sometimes forget this, assuming that more is better. Gøtzsche and Olsen, fearing that the medical community had committed a type I error (promoting an ineffective or harmful screening test), set a high standard for judging validity (eg, dismissing trials with any baseline differences). But when data from 8 trials demonstrate a large effect size (21% reduction in mortality) with narrow confidence intervals (13%-29%),⁸ the risk of a type II error overwhelms that of a type I error. The implications of the error are stark for a disease that claims 40,000 lives per year.¹³ Rejecting evidence under these conditions is more likely to cause death than accepting it.

The pivotal question in rejecting evidence should not be whether there is a design flaw but something more precise: the probability that the observed outcomes are due to factors other than the intervention under consideration (eg, chance, confounding). This can be just as likely in the absence of design flaws (eg, a perfectly conducted uncontrolled case series) as when a study is performed poorly, and it can be low even when studies are imperfect. It is a mistake to reject evidence reflexively because of a design flaw without studying these probabilities.

For example, what worried Gøtzsche and Olsen about flawed randomization was that factors other than mammography might account for lower mortality. But how likely was that (compared with the probability that early detection was efficacious)? Suppose a trial with imbalanced randomization carries a 50% probability of producing spurious results due to chance or confounding. The probability that the same phenomenon would occur in 8 trials (conducted independently in 4 countries in different decades, using different technologies, views, and screening intervals¹⁴) would be in the neighborhood of (0.50)⁸ or 0.39%. The probability would be higher if one believed that subversion is systematic among researchers, but without data such speculation is an exercise in cynicism rather than science.

Suppose that the criticisms of Gøtzsche and Olsen justify the termination of mammography. Are we applying the same standard for other screening and clinical practices, or do we move the goal posts? Physicians screen for prostate cancer without one well-designed controlled trial showing that it lowers mortality.¹⁵ We say this is not evidence-based,¹⁶ but what kind of study would change our minds? The conventional answer is a randomized controlled trial showing a reduction in prostate cancer mortality is required,¹⁷ but The Lancet article finds that even 8 such trials are unconvincing. Indeed, flawless trials lose influence if the end points or setting lack generalizability.¹⁸ If the consequence of such high standards is that 30 years of trials involving a total of 482,000 women (and untold cost) cannot establish efficacy, the prospects for making the rest of medicine evidence-based are slim indeed.

The search for perfect data also has epistemologic flaws. No study can provide the absolute certainty that extremism in critical appraisal seeks. The willingness to reject studies based on improbable theories of confounding or research misbehavior may have less to do with good science than with discomfort with uncertainty, that is, unease with any possibility that the inferences of the investigators are wrong. The wait for better evidence is in vain, however, because science can only guess about reality. Even in a flawless trial a P value of .05 means that claims of significance will be wrong 5% of the time. Good studies have better odds in predicting reality, but they do not define it. It is legitimate to reject poorly designed studies because the probability of being wrong is too high. But to reject studies because there is any probability of being wrong is to wait futilely for a class of evidence that does not exist.

Inadequate critical appraisal

The POEM about The Lancet article highlights the other extreme of critical appraisal, accepting studies at face value. The review mentioned none of the limitations in The Lancet analysis, thus giving readers little reason to doubt the conclusion that mammography lacks scientific support and potentially convincing them to stop screening. Physicians should decide whether this is the right choice only after having heard all the issues. That this study reported a null effect and used meta-analysis does not lesson the need for critical appraisal. Like removing a counterweight from a scale, the omission of critical appraisal unduly elevates study findings (positive or negative), thus fomenting overreaction by not putting the information in context.

Several new resources, POEMs among them, have become available to alert physicians to important evidence. Some features (eg, the “Abstracts” section in the Journal of the American Medical Association) simply reprint abstracts. Others associated with the evidence-based medicine (EBM) movement offer critical appraisals. In family practice, these include POEMs and Evidence-Based Practice.¹⁹ In other specialties, they include the American College of Physicians’ ACP Journal Club and the EBM journals from the BMJ Publishing Group (eg, Evidence-Based Medicine, Evidence-Based Nursing). These efforts try to approach critical appraisal systematically. ACP Journal Club and the BMJ journals apply a quality filter (excluding studies failing certain criteria²⁰) and append a commentary that mentions design limitations. POEMs go further, devoting a section to study design and validity and giving the authors explicit criteria for assessing quality.²¹

But a closer look reveals inconsistencies in how these criteria are applied. The heterogeneity is apparent in any random set of POEMs: Some authors list strengths and weaknesses by name with no elaboration, some give more details, some address only one criterion (eg, allocation concealment), while others state simply that the study was well designed. Some, like The Lancet review, say nothing about quality, reporting only the design and results. Eight (17%) of the 48 POEMs published in the first 6 months of 2000 included no critical appraisal (or a vague remark).

Similar omissions plague ACP Journal Club and the BMJ journals. Although the reviews describe concealment of allocation and blinding, and the commentary sections sometimes address design flaws at length, the degree to which this occurs, if at all, is variable. An ACP Journal Club review²² of the United Kingdom Prospective Diabetes Study, the landmark trial of intensive glycemic control, mentioned no concerns about its external validity (for contrast see the report by the American Academy of Family Physicians and American Diabetes Association²³). Calling such synopses critical appraisals obfuscates the meaning of the term.

Some say that even brief remarks are critical appraisals. But good appraisals consistently and objectively rate studies using uniform criteria.²⁴ POEMs and the EBM journals do not currently meet this standard; more systematic procedures are needed to ensure that narratives routinely discuss core elements of internal and external validity and that manuscripts lacking these elements are returned. The current conditions for preparing reviews make this difficult, however. With their modest budgets³ journals rely on hundreds of volunteer contributors, each with a different writing style and level of expertise. Onerous procedures for critical appraisal might discourage participation. Because journal space is limited, narratives are kept short (700 words for POEMs,²⁵ 425 words for the BMJ journals and ACP Journal Club²⁰) to review more studies. Each issue of JFP contains 8 POEMs, and Evidence-Based Medicine has 24 reviews. Longer appraisals would reduce the number and currency of reviews and would be less concise for busy physicians.

The disadvantage of short reviews and rapid turnover is a greater risk of inaccuracies and imbalance. Careful analysis of a study requires more time to do research and more space to explain the results than journals currently provide. Authors have only weeks to prepare manuscripts, which leaves little time to verify that descriptions are accurate and give proportionate emphasis to the issues that matter most. For many studies a few hundred words provide inadequate room to fully explain the design, results, and limitations. The risk of mistakes is heightened with less time, analysis, expert review, and space.

This tradeoff between quantity and quality begs the question of what is more important to readers and to patient care: the number of studies that physicians know about or the accuracy with which they are described. POEMs, which are designed to change practice,²⁶ can do harm if physicians acting on inaccurate or incomplete information make choices that compromise outcomes. The Lancet review is a shot across the bow. The 40,000 annual deaths from breast cancer¹³ remind us that for certain topics a mistaken inference can cost thousands of lives. If inconsistencies in appraisal make this happen often enough, efforts to synopsize evidence can do more harm than good. Also, it is incongruent for programs espousing EBM, discourages a discipline that accepts evidence on face value, to report studies with little or no discussion of validity.

Setting policy in critical appraisals

It is also antithetical for EBM to support evidence-based practice guidelines²⁷ and to publish clinical advice that is not derived from these methods. Critical appraisals that conclude by suggesting how physicians should modify patient care cross the line from science to policy. In the first half of 2000 73% of the POEMs advised physicians (with varied explicitness) to use tests (6), drugs (14), or other treatments (5) and to withhold others (10). Such advice is common fare in the medical literature, but EBM ascribes to a higher standard. Because imprudent practice policies can do harm or compromise effectiveness, EBM holds that guidelines should be drafted with care using evidence-based methods.²⁸ This entails reviewing not one study but all relevant evidence, with systematic grading of studies and explicit linkage between the recommendations and the quality of the data.^27-30 This process typically involves an expert panel and months or years of deliberations.

In contrast, practice recommendations in EBM journals and POEMs reflect what individual authors think of a study. They lack the time, funding, and journal space for a systematic literature review. Thus, the authors and their readers cannot be sure that the conclusions reflect the evidence as a whole without undue influence from the reviewed study. Rules of evidence and grades for recommendations are rarely provided. Unlike guideline panels, authors seldom vet their recommendations with experts, societies, and agencies, which often uncover flawed inferences. The thinking process behind recommendations is necessarily telescoped. While the United States Preventive Services Task Force spent 2 years deciding whether pregnant women should be screened for bacterial vaginosis, a POEM³¹ produced its recommendations within weeks.

To some such pronouncements are not guidelines, only the bottom line of a review. But in policy terms it matters little whether physicians prescribe a drug because of a guideline or because of the advice they read in ACP Journal Club or a POEM. The outcome for the patient is the same. Reviews need a bottom line, but summarizing the results of a study (eg, drug A worked better than drug B) differs from advising physicians what to do (eg, prescribe drug A). The latter is a statement of policy rather than science and should be based on broader considerations than one study.²⁸

EBM faults guidelines that omit evidence-based methods, such as those issued by advocacy groups that reflect personal opinions and selective use of studies more than systematic reviews.^32,33 Yet the recommendations in EBM journals and POEMs differ little in appearance: They provide little documentation of how conclusions were reached, feature select evidence (the study under review), rely on authors’ opinions, and provide few details on rationale. EBM journals should extract themselves from this inconsistency by sharpening the distinction between summarizing evidence and setting policy and eschewing the latter unless it emanates from evidence-based methods.

How this is handled in POEMs will reflect on family medicine. The prominence the specialty has given POEMs (promotion in JFP, family practice literature,^34-35 the Internet,^36-37 and newsletters¹⁹) signals the way family physicians think studies should be reviewed. It is important to get this right. If POEMs are meant to be critical appraisals and 17% contain no critique, calling them critical appraisals casts doubts on the specialty’s understanding of the term and perpetuates confusion about definitions. Conversely, by instituting greater scrutiny—defining the core criteria that must be discussed to qualify a study commentary as a critical appraisal and systematizing their use in POEMs—the specialty would set a new standard for EBM. If POEMs are not meant to be critical appraisals, it is important to clarify the distinction in terms, especially for family physicians who have grown accustomed to POEMs, know little about alternatives, and have come to believe that POEMs, critical appraisals, and EBM are essentially the same.

Conclusions

Advocates of EBM should be systematic in their application of critical appraisal. Critical appraisals do not deserve the name if they accept studies on face value. The criteria for determining which studies are rated good or bad should be explicit and consistent. But the scrutiny of evidence should not be taken to extremes, to the point that studies are rejected for being imperfect when there is little likelihood that the findings are wrong. By making the perfect the enemy of the good, excesses in critical appraisal do injustice to the goal of helping patients and imply existence of a level of certainty that science cannot provide.

In January 2000 an article in The Lancet drew attention when it questioned the supporting evidence for screening mammography.¹ Danish investigators Peter Gøtzsche and Ole Olsen presented a series of apparent flaws in the 8 randomized trials of mammography, ultimately concluding that screening is unjustified. Their cogent arguments and the press coverage they received left many physicians wondering whether they should continue to order mammograms. The story led the CBS Evening News² and was featured in the Washington Post,³ Time,⁴ and Reuters.⁵

A Patient-Oriented Evidence that Matters (POEM) review in the April 2000 issue of The Journal of Family Practice⁶ that addressed The Lancet study lent apparent support to these concerns. The POEM related the arguments in The Lancet article without challenging them and concluded that “mammography screening has never been shown to help women to live longer.” The authors of this POEM suggested that the only reasons for screening to continue are “politics, patients’ preconceptions, and the fear of litigation.” Unlike most POEMs, this one included no critical appraisal of the methods or assumptions of the reviewed study. This lack of comment, combined with the authors’ negative remarks about mammography, may have convinced family physicians that the criticisms of Gøtzsche and Olsen were beyond dispute.

However, controversy does surround their arguments, as the many letters to the editor published in The Lancet attest.⁷ For example, The Lancet critique made much of inconsistent sample sizes and baseline dissimilarities between screened and unscreened women. The authors asserted that such age and socioeconomic differences were “incompatible with adequate randomization.” That premise is contestable. It is normal and predictable that a proportion of population variables will differ between groups for statistical reasons, no matter how perfect the randomization. Also, the observed age difference (1 to 6 months) would not explain the 21% reduction in mortality observed in the trials.⁸

For Gøtzsche and Olsen the discrepant age patterns and sample sizes were less a cause of the results than a warning sign that randomization had been subverted (because of failure to conceal allocation). Since mortality in the screened and unscreened groups differed by only a relatively small number of deaths, they reasoned that very little bias would be necessary to tip the scales in favor of mammography.

Several arguments weaken their case, however. First, they offered no evidence that subversion or unconcealed allocation actually occurred. They equated inexplicit documentation of procedures (and dissimilar group characteristics) with improper randomization. Second, even if unconcealed allocation occurred, it does not in itself thwart randomization. Investigators who know to which group a patient will be assigned can still follow the rules and make the correct assignment. Anecdotal reports of subversion (by deciphering assignment sequences to divert or target patients for allocation) do not offer denominator data to assess how often this occurs.⁹ It would have had to occur in every trial that favored mammography to uphold the authors’ allegations. Third, even if the trials were subverted there is no indication that case mix differed enough to skew outcomes. Age differences were minor; the authors speculated that sizable imbalances in unmeasured factors could have altered results, but they gave no evidence. They cited reports that poorly concealed allocation is associated with a 37% to 41% exaggeration in odds ratios,^10,11 but these reports concerned other trials and made arguable assumptions. Finally, their confirmatory finding—that only the 6 “flawed” trials reported a benefit for mammography and that the 2 acceptable trials showed no effect—was based on recalculated relative risk rates. The original trial data show no such pattern.⁸

This is not to suggest that weaknesses in the mammography trials do not merit scrutiny. Others have also voiced criticisms.¹² But the alarm raised by Gøtzsche and Olsen goes further, compelling us to rethink the purpose of critical appraisal and the extremes at which it might cause more harm than good.

Excessive critical appraisal

We seek perfection in evidence to safeguard patients. Prematurely adopting (or abandoning) interventions through uncritical acceptance of findings risks overlooking potential harms or more effective alternatives. But critical appraisal can do harm if valid evidence is rejected. Deciding whether to accept evidence counterbalances the risks of acceptance against the risks of rejection, which are inversely related. At one extreme of the spectrum, where data are accepted on face value (no appraisal), the risk of a type I error (accepting evidence of efficacy when the intervention does not work or causes harm) is high, and that of a type II error (discarding evidence when the intervention actually works) is low. At the other extreme (excessive scrutiny) the risk of a type II error is great; such errors harm patients because knowledge is rejected that can save (or improve) lives. Obviously, patients are best served somewhere in the middle, striking an optimal balance between the risks of type I and type II errors.

Enthusiasts for critical appraisal sometimes forget this, assuming that more is better. Gøtzsche and Olsen, fearing that the medical community had committed a type I error (promoting an ineffective or harmful screening test), set a high standard for judging validity (eg, dismissing trials with any baseline differences). But when data from 8 trials demonstrate a large effect size (21% reduction in mortality) with narrow confidence intervals (13%-29%),⁸ the risk of a type II error overwhelms that of a type I error. The implications of the error are stark for a disease that claims 40,000 lives per year.¹³ Rejecting evidence under these conditions is more likely to cause death than accepting it.

The pivotal question in rejecting evidence should not be whether there is a design flaw but something more precise: the probability that the observed outcomes are due to factors other than the intervention under consideration (eg, chance, confounding). This can be just as likely in the absence of design flaws (eg, a perfectly conducted uncontrolled case series) as when a study is performed poorly, and it can be low even when studies are imperfect. It is a mistake to reject evidence reflexively because of a design flaw without studying these probabilities.

For example, what worried Gøtzsche and Olsen about flawed randomization was that factors other than mammography might account for lower mortality. But how likely was that (compared with the probability that early detection was efficacious)? Suppose a trial with imbalanced randomization carries a 50% probability of producing spurious results due to chance or confounding. The probability that the same phenomenon would occur in 8 trials (conducted independently in 4 countries in different decades, using different technologies, views, and screening intervals¹⁴) would be in the neighborhood of (0.50)⁸ or 0.39%. The probability would be higher if one believed that subversion is systematic among researchers, but without data such speculation is an exercise in cynicism rather than science.

Suppose that the criticisms of Gøtzsche and Olsen justify the termination of mammography. Are we applying the same standard for other screening and clinical practices, or do we move the goal posts? Physicians screen for prostate cancer without one well-designed controlled trial showing that it lowers mortality.¹⁵ We say this is not evidence-based,¹⁶ but what kind of study would change our minds? The conventional answer is a randomized controlled trial showing a reduction in prostate cancer mortality is required,¹⁷ but The Lancet article finds that even 8 such trials are unconvincing. Indeed, flawless trials lose influence if the end points or setting lack generalizability.¹⁸ If the consequence of such high standards is that 30 years of trials involving a total of 482,000 women (and untold cost) cannot establish efficacy, the prospects for making the rest of medicine evidence-based are slim indeed.

The search for perfect data also has epistemologic flaws. No study can provide the absolute certainty that extremism in critical appraisal seeks. The willingness to reject studies based on improbable theories of confounding or research misbehavior may have less to do with good science than with discomfort with uncertainty, that is, unease with any possibility that the inferences of the investigators are wrong. The wait for better evidence is in vain, however, because science can only guess about reality. Even in a flawless trial a P value of .05 means that claims of significance will be wrong 5% of the time. Good studies have better odds in predicting reality, but they do not define it. It is legitimate to reject poorly designed studies because the probability of being wrong is too high. But to reject studies because there is any probability of being wrong is to wait futilely for a class of evidence that does not exist.

Inadequate critical appraisal

The POEM about The Lancet article highlights the other extreme of critical appraisal, accepting studies at face value. The review mentioned none of the limitations in The Lancet analysis, thus giving readers little reason to doubt the conclusion that mammography lacks scientific support and potentially convincing them to stop screening. Physicians should decide whether this is the right choice only after having heard all the issues. That this study reported a null effect and used meta-analysis does not lesson the need for critical appraisal. Like removing a counterweight from a scale, the omission of critical appraisal unduly elevates study findings (positive or negative), thus fomenting overreaction by not putting the information in context.

Several new resources, POEMs among them, have become available to alert physicians to important evidence. Some features (eg, the “Abstracts” section in the Journal of the American Medical Association) simply reprint abstracts. Others associated with the evidence-based medicine (EBM) movement offer critical appraisals. In family practice, these include POEMs and Evidence-Based Practice.¹⁹ In other specialties, they include the American College of Physicians’ ACP Journal Club and the EBM journals from the BMJ Publishing Group (eg, Evidence-Based Medicine, Evidence-Based Nursing). These efforts try to approach critical appraisal systematically. ACP Journal Club and the BMJ journals apply a quality filter (excluding studies failing certain criteria²⁰) and append a commentary that mentions design limitations. POEMs go further, devoting a section to study design and validity and giving the authors explicit criteria for assessing quality.²¹

But a closer look reveals inconsistencies in how these criteria are applied. The heterogeneity is apparent in any random set of POEMs: Some authors list strengths and weaknesses by name with no elaboration, some give more details, some address only one criterion (eg, allocation concealment), while others state simply that the study was well designed. Some, like The Lancet review, say nothing about quality, reporting only the design and results. Eight (17%) of the 48 POEMs published in the first 6 months of 2000 included no critical appraisal (or a vague remark).

Similar omissions plague ACP Journal Club and the BMJ journals. Although the reviews describe concealment of allocation and blinding, and the commentary sections sometimes address design flaws at length, the degree to which this occurs, if at all, is variable. An ACP Journal Club review²² of the United Kingdom Prospective Diabetes Study, the landmark trial of intensive glycemic control, mentioned no concerns about its external validity (for contrast see the report by the American Academy of Family Physicians and American Diabetes Association²³). Calling such synopses critical appraisals obfuscates the meaning of the term.

Some say that even brief remarks are critical appraisals. But good appraisals consistently and objectively rate studies using uniform criteria.²⁴ POEMs and the EBM journals do not currently meet this standard; more systematic procedures are needed to ensure that narratives routinely discuss core elements of internal and external validity and that manuscripts lacking these elements are returned. The current conditions for preparing reviews make this difficult, however. With their modest budgets³ journals rely on hundreds of volunteer contributors, each with a different writing style and level of expertise. Onerous procedures for critical appraisal might discourage participation. Because journal space is limited, narratives are kept short (700 words for POEMs,²⁵ 425 words for the BMJ journals and ACP Journal Club²⁰) to review more studies. Each issue of JFP contains 8 POEMs, and Evidence-Based Medicine has 24 reviews. Longer appraisals would reduce the number and currency of reviews and would be less concise for busy physicians.

The disadvantage of short reviews and rapid turnover is a greater risk of inaccuracies and imbalance. Careful analysis of a study requires more time to do research and more space to explain the results than journals currently provide. Authors have only weeks to prepare manuscripts, which leaves little time to verify that descriptions are accurate and give proportionate emphasis to the issues that matter most. For many studies a few hundred words provide inadequate room to fully explain the design, results, and limitations. The risk of mistakes is heightened with less time, analysis, expert review, and space.

This tradeoff between quantity and quality begs the question of what is more important to readers and to patient care: the number of studies that physicians know about or the accuracy with which they are described. POEMs, which are designed to change practice,²⁶ can do harm if physicians acting on inaccurate or incomplete information make choices that compromise outcomes. The Lancet review is a shot across the bow. The 40,000 annual deaths from breast cancer¹³ remind us that for certain topics a mistaken inference can cost thousands of lives. If inconsistencies in appraisal make this happen often enough, efforts to synopsize evidence can do more harm than good. Also, it is incongruent for programs espousing EBM, discourages a discipline that accepts evidence on face value, to report studies with little or no discussion of validity.

Setting policy in critical appraisals

It is also antithetical for EBM to support evidence-based practice guidelines²⁷ and to publish clinical advice that is not derived from these methods. Critical appraisals that conclude by suggesting how physicians should modify patient care cross the line from science to policy. In the first half of 2000 73% of the POEMs advised physicians (with varied explicitness) to use tests (6), drugs (14), or other treatments (5) and to withhold others (10). Such advice is common fare in the medical literature, but EBM ascribes to a higher standard. Because imprudent practice policies can do harm or compromise effectiveness, EBM holds that guidelines should be drafted with care using evidence-based methods.²⁸ This entails reviewing not one study but all relevant evidence, with systematic grading of studies and explicit linkage between the recommendations and the quality of the data.^27-30 This process typically involves an expert panel and months or years of deliberations.

In contrast, practice recommendations in EBM journals and POEMs reflect what individual authors think of a study. They lack the time, funding, and journal space for a systematic literature review. Thus, the authors and their readers cannot be sure that the conclusions reflect the evidence as a whole without undue influence from the reviewed study. Rules of evidence and grades for recommendations are rarely provided. Unlike guideline panels, authors seldom vet their recommendations with experts, societies, and agencies, which often uncover flawed inferences. The thinking process behind recommendations is necessarily telescoped. While the United States Preventive Services Task Force spent 2 years deciding whether pregnant women should be screened for bacterial vaginosis, a POEM³¹ produced its recommendations within weeks.

To some such pronouncements are not guidelines, only the bottom line of a review. But in policy terms it matters little whether physicians prescribe a drug because of a guideline or because of the advice they read in ACP Journal Club or a POEM. The outcome for the patient is the same. Reviews need a bottom line, but summarizing the results of a study (eg, drug A worked better than drug B) differs from advising physicians what to do (eg, prescribe drug A). The latter is a statement of policy rather than science and should be based on broader considerations than one study.²⁸

EBM faults guidelines that omit evidence-based methods, such as those issued by advocacy groups that reflect personal opinions and selective use of studies more than systematic reviews.^32,33 Yet the recommendations in EBM journals and POEMs differ little in appearance: They provide little documentation of how conclusions were reached, feature select evidence (the study under review), rely on authors’ opinions, and provide few details on rationale. EBM journals should extract themselves from this inconsistency by sharpening the distinction between summarizing evidence and setting policy and eschewing the latter unless it emanates from evidence-based methods.

How this is handled in POEMs will reflect on family medicine. The prominence the specialty has given POEMs (promotion in JFP, family practice literature,^34-35 the Internet,^36-37 and newsletters¹⁹) signals the way family physicians think studies should be reviewed. It is important to get this right. If POEMs are meant to be critical appraisals and 17% contain no critique, calling them critical appraisals casts doubts on the specialty’s understanding of the term and perpetuates confusion about definitions. Conversely, by instituting greater scrutiny—defining the core criteria that must be discussed to qualify a study commentary as a critical appraisal and systematizing their use in POEMs—the specialty would set a new standard for EBM. If POEMs are not meant to be critical appraisals, it is important to clarify the distinction in terms, especially for family physicians who have grown accustomed to POEMs, know little about alternatives, and have come to believe that POEMs, critical appraisals, and EBM are essentially the same.

Conclusions

Advocates of EBM should be systematic in their application of critical appraisal. Critical appraisals do not deserve the name if they accept studies on face value. The criteria for determining which studies are rated good or bad should be explicit and consistent. But the scrutiny of evidence should not be taken to extremes, to the point that studies are rejected for being imperfect when there is little likelihood that the findings are wrong. By making the perfect the enemy of the good, excesses in critical appraisal do injustice to the goal of helping patients and imply existence of a level of certainty that science cannot provide.

In January 2000 an article in The Lancet drew attention when it questioned the supporting evidence for screening mammography.¹ Danish investigators Peter Gøtzsche and Ole Olsen presented a series of apparent flaws in the 8 randomized trials of mammography, ultimately concluding that screening is unjustified. Their cogent arguments and the press coverage they received left many physicians wondering whether they should continue to order mammograms. The story led the CBS Evening News² and was featured in the Washington Post,³ Time,⁴ and Reuters.⁵

A Patient-Oriented Evidence that Matters (POEM) review in the April 2000 issue of The Journal of Family Practice⁶ that addressed The Lancet study lent apparent support to these concerns. The POEM related the arguments in The Lancet article without challenging them and concluded that “mammography screening has never been shown to help women to live longer.” The authors of this POEM suggested that the only reasons for screening to continue are “politics, patients’ preconceptions, and the fear of litigation.” Unlike most POEMs, this one included no critical appraisal of the methods or assumptions of the reviewed study. This lack of comment, combined with the authors’ negative remarks about mammography, may have convinced family physicians that the criticisms of Gøtzsche and Olsen were beyond dispute.

However, controversy does surround their arguments, as the many letters to the editor published in The Lancet attest.⁷ For example, The Lancet critique made much of inconsistent sample sizes and baseline dissimilarities between screened and unscreened women. The authors asserted that such age and socioeconomic differences were “incompatible with adequate randomization.” That premise is contestable. It is normal and predictable that a proportion of population variables will differ between groups for statistical reasons, no matter how perfect the randomization. Also, the observed age difference (1 to 6 months) would not explain the 21% reduction in mortality observed in the trials.⁸

For Gøtzsche and Olsen the discrepant age patterns and sample sizes were less a cause of the results than a warning sign that randomization had been subverted (because of failure to conceal allocation). Since mortality in the screened and unscreened groups differed by only a relatively small number of deaths, they reasoned that very little bias would be necessary to tip the scales in favor of mammography.

Several arguments weaken their case, however. First, they offered no evidence that subversion or unconcealed allocation actually occurred. They equated inexplicit documentation of procedures (and dissimilar group characteristics) with improper randomization. Second, even if unconcealed allocation occurred, it does not in itself thwart randomization. Investigators who know to which group a patient will be assigned can still follow the rules and make the correct assignment. Anecdotal reports of subversion (by deciphering assignment sequences to divert or target patients for allocation) do not offer denominator data to assess how often this occurs.⁹ It would have had to occur in every trial that favored mammography to uphold the authors’ allegations. Third, even if the trials were subverted there is no indication that case mix differed enough to skew outcomes. Age differences were minor; the authors speculated that sizable imbalances in unmeasured factors could have altered results, but they gave no evidence. They cited reports that poorly concealed allocation is associated with a 37% to 41% exaggeration in odds ratios,^10,11 but these reports concerned other trials and made arguable assumptions. Finally, their confirmatory finding—that only the 6 “flawed” trials reported a benefit for mammography and that the 2 acceptable trials showed no effect—was based on recalculated relative risk rates. The original trial data show no such pattern.⁸

This is not to suggest that weaknesses in the mammography trials do not merit scrutiny. Others have also voiced criticisms.¹² But the alarm raised by Gøtzsche and Olsen goes further, compelling us to rethink the purpose of critical appraisal and the extremes at which it might cause more harm than good.

Excessive critical appraisal

We seek perfection in evidence to safeguard patients. Prematurely adopting (or abandoning) interventions through uncritical acceptance of findings risks overlooking potential harms or more effective alternatives. But critical appraisal can do harm if valid evidence is rejected. Deciding whether to accept evidence counterbalances the risks of acceptance against the risks of rejection, which are inversely related. At one extreme of the spectrum, where data are accepted on face value (no appraisal), the risk of a type I error (accepting evidence of efficacy when the intervention does not work or causes harm) is high, and that of a type II error (discarding evidence when the intervention actually works) is low. At the other extreme (excessive scrutiny) the risk of a type II error is great; such errors harm patients because knowledge is rejected that can save (or improve) lives. Obviously, patients are best served somewhere in the middle, striking an optimal balance between the risks of type I and type II errors.

Enthusiasts for critical appraisal sometimes forget this, assuming that more is better. Gøtzsche and Olsen, fearing that the medical community had committed a type I error (promoting an ineffective or harmful screening test), set a high standard for judging validity (eg, dismissing trials with any baseline differences). But when data from 8 trials demonstrate a large effect size (21% reduction in mortality) with narrow confidence intervals (13%-29%),⁸ the risk of a type II error overwhelms that of a type I error. The implications of the error are stark for a disease that claims 40,000 lives per year.¹³ Rejecting evidence under these conditions is more likely to cause death than accepting it.

The pivotal question in rejecting evidence should not be whether there is a design flaw but something more precise: the probability that the observed outcomes are due to factors other than the intervention under consideration (eg, chance, confounding). This can be just as likely in the absence of design flaws (eg, a perfectly conducted uncontrolled case series) as when a study is performed poorly, and it can be low even when studies are imperfect. It is a mistake to reject evidence reflexively because of a design flaw without studying these probabilities.

For example, what worried Gøtzsche and Olsen about flawed randomization was that factors other than mammography might account for lower mortality. But how likely was that (compared with the probability that early detection was efficacious)? Suppose a trial with imbalanced randomization carries a 50% probability of producing spurious results due to chance or confounding. The probability that the same phenomenon would occur in 8 trials (conducted independently in 4 countries in different decades, using different technologies, views, and screening intervals¹⁴) would be in the neighborhood of (0.50)⁸ or 0.39%. The probability would be higher if one believed that subversion is systematic among researchers, but without data such speculation is an exercise in cynicism rather than science.

Suppose that the criticisms of Gøtzsche and Olsen justify the termination of mammography. Are we applying the same standard for other screening and clinical practices, or do we move the goal posts? Physicians screen for prostate cancer without one well-designed controlled trial showing that it lowers mortality.¹⁵ We say this is not evidence-based,¹⁶ but what kind of study would change our minds? The conventional answer is a randomized controlled trial showing a reduction in prostate cancer mortality is required,¹⁷ but The Lancet article finds that even 8 such trials are unconvincing. Indeed, flawless trials lose influence if the end points or setting lack generalizability.¹⁸ If the consequence of such high standards is that 30 years of trials involving a total of 482,000 women (and untold cost) cannot establish efficacy, the prospects for making the rest of medicine evidence-based are slim indeed.

The search for perfect data also has epistemologic flaws. No study can provide the absolute certainty that extremism in critical appraisal seeks. The willingness to reject studies based on improbable theories of confounding or research misbehavior may have less to do with good science than with discomfort with uncertainty, that is, unease with any possibility that the inferences of the investigators are wrong. The wait for better evidence is in vain, however, because science can only guess about reality. Even in a flawless trial a P value of .05 means that claims of significance will be wrong 5% of the time. Good studies have better odds in predicting reality, but they do not define it. It is legitimate to reject poorly designed studies because the probability of being wrong is too high. But to reject studies because there is any probability of being wrong is to wait futilely for a class of evidence that does not exist.

Inadequate critical appraisal

The POEM about The Lancet article highlights the other extreme of critical appraisal, accepting studies at face value. The review mentioned none of the limitations in The Lancet analysis, thus giving readers little reason to doubt the conclusion that mammography lacks scientific support and potentially convincing them to stop screening. Physicians should decide whether this is the right choice only after having heard all the issues. That this study reported a null effect and used meta-analysis does not lesson the need for critical appraisal. Like removing a counterweight from a scale, the omission of critical appraisal unduly elevates study findings (positive or negative), thus fomenting overreaction by not putting the information in context.

Several new resources, POEMs among them, have become available to alert physicians to important evidence. Some features (eg, the “Abstracts” section in the Journal of the American Medical Association) simply reprint abstracts. Others associated with the evidence-based medicine (EBM) movement offer critical appraisals. In family practice, these include POEMs and Evidence-Based Practice.¹⁹ In other specialties, they include the American College of Physicians’ ACP Journal Club and the EBM journals from the BMJ Publishing Group (eg, Evidence-Based Medicine, Evidence-Based Nursing). These efforts try to approach critical appraisal systematically. ACP Journal Club and the BMJ journals apply a quality filter (excluding studies failing certain criteria²⁰) and append a commentary that mentions design limitations. POEMs go further, devoting a section to study design and validity and giving the authors explicit criteria for assessing quality.²¹

But a closer look reveals inconsistencies in how these criteria are applied. The heterogeneity is apparent in any random set of POEMs: Some authors list strengths and weaknesses by name with no elaboration, some give more details, some address only one criterion (eg, allocation concealment), while others state simply that the study was well designed. Some, like The Lancet review, say nothing about quality, reporting only the design and results. Eight (17%) of the 48 POEMs published in the first 6 months of 2000 included no critical appraisal (or a vague remark).

Similar omissions plague ACP Journal Club and the BMJ journals. Although the reviews describe concealment of allocation and blinding, and the commentary sections sometimes address design flaws at length, the degree to which this occurs, if at all, is variable. An ACP Journal Club review²² of the United Kingdom Prospective Diabetes Study, the landmark trial of intensive glycemic control, mentioned no concerns about its external validity (for contrast see the report by the American Academy of Family Physicians and American Diabetes Association²³). Calling such synopses critical appraisals obfuscates the meaning of the term.

Some say that even brief remarks are critical appraisals. But good appraisals consistently and objectively rate studies using uniform criteria.²⁴ POEMs and the EBM journals do not currently meet this standard; more systematic procedures are needed to ensure that narratives routinely discuss core elements of internal and external validity and that manuscripts lacking these elements are returned. The current conditions for preparing reviews make this difficult, however. With their modest budgets³ journals rely on hundreds of volunteer contributors, each with a different writing style and level of expertise. Onerous procedures for critical appraisal might discourage participation. Because journal space is limited, narratives are kept short (700 words for POEMs,²⁵ 425 words for the BMJ journals and ACP Journal Club²⁰) to review more studies. Each issue of JFP contains 8 POEMs, and Evidence-Based Medicine has 24 reviews. Longer appraisals would reduce the number and currency of reviews and would be less concise for busy physicians.

The disadvantage of short reviews and rapid turnover is a greater risk of inaccuracies and imbalance. Careful analysis of a study requires more time to do research and more space to explain the results than journals currently provide. Authors have only weeks to prepare manuscripts, which leaves little time to verify that descriptions are accurate and give proportionate emphasis to the issues that matter most. For many studies a few hundred words provide inadequate room to fully explain the design, results, and limitations. The risk of mistakes is heightened with less time, analysis, expert review, and space.

This tradeoff between quantity and quality begs the question of what is more important to readers and to patient care: the number of studies that physicians know about or the accuracy with which they are described. POEMs, which are designed to change practice,²⁶ can do harm if physicians acting on inaccurate or incomplete information make choices that compromise outcomes. The Lancet review is a shot across the bow. The 40,000 annual deaths from breast cancer¹³ remind us that for certain topics a mistaken inference can cost thousands of lives. If inconsistencies in appraisal make this happen often enough, efforts to synopsize evidence can do more harm than good. Also, it is incongruent for programs espousing EBM, discourages a discipline that accepts evidence on face value, to report studies with little or no discussion of validity.

Setting policy in critical appraisals

It is also antithetical for EBM to support evidence-based practice guidelines²⁷ and to publish clinical advice that is not derived from these methods. Critical appraisals that conclude by suggesting how physicians should modify patient care cross the line from science to policy. In the first half of 2000 73% of the POEMs advised physicians (with varied explicitness) to use tests (6), drugs (14), or other treatments (5) and to withhold others (10). Such advice is common fare in the medical literature, but EBM ascribes to a higher standard. Because imprudent practice policies can do harm or compromise effectiveness, EBM holds that guidelines should be drafted with care using evidence-based methods.²⁸ This entails reviewing not one study but all relevant evidence, with systematic grading of studies and explicit linkage between the recommendations and the quality of the data.^27-30 This process typically involves an expert panel and months or years of deliberations.

In contrast, practice recommendations in EBM journals and POEMs reflect what individual authors think of a study. They lack the time, funding, and journal space for a systematic literature review. Thus, the authors and their readers cannot be sure that the conclusions reflect the evidence as a whole without undue influence from the reviewed study. Rules of evidence and grades for recommendations are rarely provided. Unlike guideline panels, authors seldom vet their recommendations with experts, societies, and agencies, which often uncover flawed inferences. The thinking process behind recommendations is necessarily telescoped. While the United States Preventive Services Task Force spent 2 years deciding whether pregnant women should be screened for bacterial vaginosis, a POEM³¹ produced its recommendations within weeks.

To some such pronouncements are not guidelines, only the bottom line of a review. But in policy terms it matters little whether physicians prescribe a drug because of a guideline or because of the advice they read in ACP Journal Club or a POEM. The outcome for the patient is the same. Reviews need a bottom line, but summarizing the results of a study (eg, drug A worked better than drug B) differs from advising physicians what to do (eg, prescribe drug A). The latter is a statement of policy rather than science and should be based on broader considerations than one study.²⁸

EBM faults guidelines that omit evidence-based methods, such as those issued by advocacy groups that reflect personal opinions and selective use of studies more than systematic reviews.^32,33 Yet the recommendations in EBM journals and POEMs differ little in appearance: They provide little documentation of how conclusions were reached, feature select evidence (the study under review), rely on authors’ opinions, and provide few details on rationale. EBM journals should extract themselves from this inconsistency by sharpening the distinction between summarizing evidence and setting policy and eschewing the latter unless it emanates from evidence-based methods.

How this is handled in POEMs will reflect on family medicine. The prominence the specialty has given POEMs (promotion in JFP, family practice literature,^34-35 the Internet,^36-37 and newsletters¹⁹) signals the way family physicians think studies should be reviewed. It is important to get this right. If POEMs are meant to be critical appraisals and 17% contain no critique, calling them critical appraisals casts doubts on the specialty’s understanding of the term and perpetuates confusion about definitions. Conversely, by instituting greater scrutiny—defining the core criteria that must be discussed to qualify a study commentary as a critical appraisal and systematizing their use in POEMs—the specialty would set a new standard for EBM. If POEMs are not meant to be critical appraisals, it is important to clarify the distinction in terms, especially for family physicians who have grown accustomed to POEMs, know little about alternatives, and have come to believe that POEMs, critical appraisals, and EBM are essentially the same.

Conclusions

Advocates of EBM should be systematic in their application of critical appraisal. Critical appraisals do not deserve the name if they accept studies on face value. The criteria for determining which studies are rated good or bad should be explicit and consistent. But the scrutiny of evidence should not be taken to extremes, to the point that studies are rejected for being imperfect when there is little likelihood that the findings are wrong. By making the perfect the enemy of the good, excesses in critical appraisal do injustice to the goal of helping patients and imply existence of a level of certainty that science cannot provide.

An estimated 16 million people in the United States have diabetes.¹ Its microvascular complications (retinopathy, nephropathy, neuropathy) are a leading cause of blindness among adults,² end-stage renal disease,³ and lower-extremity amputations.⁴ Its macrovascular complications pose an even greater public health burden, increasing the risk of coronary artery disease, stroke, and peripheral vascular disease.⁵ Each year diabetes costs the country an estimated $90 billion to $99 billion.^6,7

Type 2 diabetes, which accounts for 90% to 95% of diabetes cases,⁸ differs from type 1 disease in average age of onset and etiology. In both forms, however, the underlying cause of microvascular (and possibly macrovascular) complications appears to be chronic elevations in blood glucose concentrations. The overriding factors in predicting microvascular pathogenesis have less to do with the type of diabetes than with the number of years the patient has had hyperglycemia and the magnitude of the glucose elevation.

In recent years, 2 major randomized controlled trials (RCTs), the Diabetes Control and Complications Trial⁹ (DCCT) in patients with type 1 disease and the United Kingdom Prospective Diabetes Study¹⁰ (UKPDS) in patients with type 2 diabetes, have shown that microvascular complications can be reduced significantly in patients who achieve normal or near-normal blood glucose levels. There is now agreement in the medical community about the importance of lowering markedly elevated blood glucose levels.

The incremental benefit of tight glycemic control (as opposed to less intensive therapy) varies across patient groups, however. Years are required for microvascular complications to progress to symptomatic disease. Patients with type 1 diabetes, who are generally younger, are more likely to live long enough to benefit from tight glycemic control than patients with type 2 disease, who face a shorter life expectancy because of their age and risk of cardiovascular disease. For patients with coexistent diseases, the delayed benefits of glycemic control may be offset by the more immediate inconvenience, complications, and costs of intensive treatment and by the health effects of comorbid conditions.

These generalizations do not apply to all patients. The older age of onset of type 2 diabetes is a population average with a wide distribution. Many patients with type 2 diabetes live long enough to experience significant microvascular disease and may benefit from glycemic control. Also, interventions that extend life expectancy (eg, smoking cessation, blood pressure control, and lipid control) give patients more time to encounter advanced microvascular disease and an opportunity to benefit from treatment.

In 1996, the American Academy of Family Physicians convened a panel to conduct a systematic review of the evidence of the benefits and harms of glycemic control in type 2 diabetes and to develop evidence-based recommendations. The 9-member panel included family physicians, general internists, endocrinologists, and a practice guidelines methodologist. Four members were appointed by the American Diabetes Association and the American College of Physicians. We summarize the panel’s findings, which are available in a full report.*

Methods

Systematic Review

The review methods are provided in detail in the full report. The literature search retrieved published evidence on the effects of glycemic control on microvascular and macrovascular complications in type 1 and type 2 diabetes and on adverse effects. RCT evidence for type 1 diabetes was considered relevant in evaluating the effects of glycemic control in type 2 disease. A total of 798 citations met initial inclusion criteria. All articles underwent structured abstraction. We closed the search with the publication of the UKPDS results in September 1998.

In reviewing the evidence, the panel gave greater weight to RCTs than to observational studies and emphasized data on health outcomes perceptible to patients (eg, visual acuity) over those for intermediate or surrogate end points (eg, retinopathy) that precede or are associated with such outcomes. Most trials did not designate health outcomes as primary end points and therefore lacked the statistical power and duration to prove an effect. Panel recommendations were evidence-based and did not reflect expert opinion. Fourteen outside diabetologists and family physicians externally reviewed the full report, and revisions consistent with the evidence were adopted. The American Academy of Family Physicians and the American Diabetes Association endorsed the full report in March 1999 and this policy statement in October 1999.

We focused the review on the benefits of glycemic control in general and not on specific agents (eg, sulfonylureas, metformin). Interventions not associated with glycemic control (eg, laser phototherapy, angiotensin-converting enzyme inhibitors), which also mitigate the effects of microvascular disease, were not examined in our review.

Results

Microvascular Outcomes

Evidence from Observational Studies. Many cross-sectional studies indicate that people with type 2 diabetes who have higher plasma glucose or glycated hemoglobin (e.g., hemoglobin A1c) levels are more likely to have evidence of retinopathy, neuropathy, or albuminuria.¹¹ Numerous prospective longitudinal studies also show that an elevated fasting plasma glucose (FPG) concentration or glycated hemoglobin level at baseline or over time increases the chances that type 2 patients will develop new or worsened retinopathy, abnormal electrophysiologic findings, or renal dysfunction.^12-15 However, observational data, unlike evidence from RCTs, does not prove that lowering blood glucose levels reduces the incidence of these complications.

Evidence from RCTs. Ten RCTs do provide this evidence.^9,10,16-24 Three trials involved patients with type 2 diabetes: the very large UKPDS¹⁰ (approximately 4200 patients) and 2 small Japanese studies.^16,19 The largest of the 7 trials of patients with type 1 diabetes was the DCCT (1441 patients).⁹ In most trials, patients were randomly allocated to an intensive treatment group that received multiple or continuous insulin administrations or to a control group that received conventional insulin therapy. Most studies confirmed (through mean glycated hemoglobin levels) better glycemic control with intensive treatment. For example, mean hemoglobin A1c levels in the intensive/conservative treatment groups of the DCCT and UKPDS were 7.2%/9.1% and 7.0%/7.9%, respectively.^9,10 An old RCT (University Group Diabetes Program) that did not produce significant differences in glucose control in some treatment arms and lacked statistical power was excluded from our review.²⁵ Average lengths of follow-up in the RCTs ranged from 2 to 12.5 years (6.5 and 10 years, respectively, in the DCCT and UKPDS).

Retinopathy in Patients with Type 2 Diabetes. RCTs provide good evidence that glycemic control reduces the incidence of retinopathy. In a Japanese trial, among patients with no retinopathy at baseline, the 6-year incidence of new disease (progression Ž2 steps on the Early Treatment Diabetic Retinopathy Study [ETDRS] scale²⁶) in the intensive and conservative treatment groups was 6% and 36%, respectively, a relative reduction of 83%. In patients with retinopathy at baseline (secondary prevention group), the incidence rates for intensive and conservative treatment groups were 17% and 44%, respectively.¹⁶

The authors of the UKPDS results, who reported a 25% relative reduction in the incidence of microvascular complications (11.4 vs 8.6 events/1000 patient-years) attributed much of the benefit to reduced retinopathy.¹⁰ The need for laser therapy was lowered from 11.0 to 7.9 events/1000-patient years, a 29% relative reduction. Within 6 years, the incidence of a 2-step progression on the ETDRS scale was lowered from 28% to 23%. The relative reduction in cataract extraction was 24%. The incidence of decreased visual acuity, blindness, and vitreous hemorrhage was not lowered significantly. The extent to which the latter resulted from treatment for early complications is not known.

Retinopathy in Patients with Type 1 Diabetes. Among patients with no baseline retinopathy in the DCCT (primary prevention group), the 6.5-year incidence of a sustained 3-step change on the ETDRS scale was reduced by 76% (from 4.7 to 1.2/100 patient-years).⁹ In the secondary prevention group, the rate of progression was lowered by 54% (from 7.8 to 3.7/100 patient-years). In this group intensive treatment was also associated with a lower incidence of severe retinopathy, need for laser treatment, and sustained progression worsening for at least 6 months (adjusted relative risk reduction=47%, 56%, and 65%, respectively).^9,27 A Swedish trial reported improved ETDRS scales and a lower prevalence of visual impairment (14% vs 35%) at 7.8 years median follow-up.¹⁷

Peripheral Neuropathy in Patients with Type 2 Diabetes. Some trials suggest that lowering blood glucose improves isolated electrophysiologic measures.^10,16,19 A Japanese RCT¹⁶ reported an increase in median nerve conduction velocity and a reduction in arm vibration threshold, but other physiologic measures were unaffected. Neurologic symptoms were not measured. The UKPDS¹⁰ showed no effect on the incidence of absent ankle and knee reflexes, but abnormal biothesiometery data (for toes) occurred less frequently with intensive treatment. Impotence and heart rate responses to deep breathing and standing occurred with equal frequency.

Peripheral Neuropathy in Patients with Type 1 Diabetes. The DCCT showed a 69% reduction (9.8 vs 3.1/100 patient-years) in newly “confirmed clinical neuropathy” (abnormal neurologic history or physical examination combined with abnormal nerve conduction or autonomic nervous system studies), but the incidence of neurologic symptoms was not reported.^9,28 Progression of clinical neuropathy in patients with preexisting disease was reduced by 57%, from 16.1 to 7.0/100 patient-years. A Swedish RCT reported a lower incidence of neuropathic symptoms at 10-year follow-up (14% vs 32%) and higher pin-prick sensitivity.²⁹

Nephropathy in Patients with Type 2 Diabetes. Intensive insulin treatment does appear to reduce the incidence of albuminuria.^10,16,19 The UKPDS observed a lower incidence of microalbuminuria within 3 years and a lower incidence of gross proteinuria and increased plasma creatinine within 9 years of follow-up (relative risk reduction=17%, 33%, and 60%, respectively).¹⁰ The incidence rates of renal failure and death from renal disease did not differ significantly between the groups, but the absolute number of cases was small.

Nephropathy in Patients with Type 1 Diabetes. In the DCCT, among patients who lacked microalbuminuria at baseline, the incidence of new cases over 6.5 years was reduced by 34% (3.4 vs 2.2/100,000 patient-years), but the incidence of sustained microalbuminuria, macroalbuminuria, or abnormal creatinine clearance did not differ. In the secondary prevention group, the incidence of microalbuminuria was reduced from 5.7 to 3.6/100,000 patient-years, a 43% relative reduction, and the incidence of sustained microalbuminuria and of macroalbuminuria was also reduced.^9,30

Macrovascular Outcomes

Observational Studies. Some cross-sectional studies in type 2 diabetes report that elevated FPG concentrations or glycated hemoglobin levels are more common in people with coronary artery disease, an abnormal electrocardiogram, or cardiovascular disease.^31,32 Longitudinal studies show that patients with elevated blood glucose, glycated hemoglobin, or postprandial glucose levels at baseline are more likely to develop coronary artery disease or an abnormal electrocardiogram or to die of coronary artery or cardiovascular disease.^33-36 An association between blood glucose concentration and stroke or peripheral vascular disease (eg, incidence of amputation and foot ulcers) has also been demonstrated in such studies^{31,33,34,37,38} but less consistently.

Clinical Trials of Patients with Type 2 Diabetes. The UKPDS showed a 16% reduction in the 10-year incidence of myocardial infarction with intensive treatment, a difference of borderline statistical significance (P=.05, 95% confidence interval for relative risk=0.71-1.00).¹⁰ Statistically significant differences were noted in certain subgroups.³⁹ Sudden death was less common (relative risk reduction=46%, P=.05), but the incidence of fatal myocardial infarction, heart failure, angina, stroke, amputation, and death from peripheral vascular disease was unchanged.¹⁰ The authors noted that the study lacked statistical power to exclude an effect on fatal outcomes.

Another RCT reported no significant effect on cardiovascular events or mortality with intensive treatment, but the mean follow-up period was only 27 months.⁴⁰ A British trial involving patients with moderate hyperglycemia reported that cardiovascular events occurred less frequently in a group given high-dose tolbutamide and a recommended diet than in the control group, but the patient population, type of diabetes, and outcome measures were defined imprecisely.⁴¹

Clinical Trials of Patients with Type 1 Diabetes. The incidence of major cardiovascular and peripheral vascular events in most trials did not differ significantly with intensive treatment,^9,42 but the number of cases and length of follow-up were generally too small to detect a difference.

All-Cause Mortality

Observational Studies. Some observational studies report an association between poor glycemic control and all-cause mortality or overall survival rates in type 2 diabetes.^35,36,43-45 However, other cohort studies report that death rates are not reliably predicted by FPG concentrations or glycated hemoglobin levels.^46,47

Clinical Trials of Patients with Type 2 Diabetes. A Swedish RCT found that patients with diabetes admitted to coronary care units for recent myocardial infarction achieved better glycemic control and experienced significantly lower all-cause mortality (33% vs 44%) if they received intensive insulin therapy (insulin-glucose infusion for the first 24 hours and subcutaneous insulin 4 times daily for 3 months).⁴⁸ The UKPDS showed no significant effect on either all-cause or diabetes-related mortality but lacked statistical power to exclude an effect.¹⁰

Clinical Trials of Patients with Type 1 Diabetes. There are few data on all-cause mortality in patients with type 1 diabetes because of the low event rate.^9,42

Discussion

Potential Harms of Intensive Glycemic Control

Specific complications can occur with each of the agents used to treat type 2 diabetes: Insulin has potential adverse effects, and oral glucose-lowering drugs carry some risk of undesirable side effects and uncommon but serious complications (eg, lactic acidosis, hepatotoxicity).

Attempts to achieve euglycemia can increase the risk of hypoglycemia, and some medications are associated with weight gain. The risk of severe hypoglycemia is greatest for patients with type 1 diabetes. In the subjects with type 2 diabetes in the UKPDS, the incidence of major hypoglycemic episodes was higher among the intensively treated than conventionally treated patients, but the rate was low (1% to 2%).¹⁰ More typically in type 2 disease, a more substantial risk exists for minor hypoglycemic episodes, which are usually inconsequential.^10,40 An association between intensive treatment and weight gain has been reported (mean=3.1 kg and 4.6 kg in the UKPDS and DCCT, respectively),^9,10 but there is no evidence that this amount of weight gain affects outcomes.

Intensive treatment requires that patients perform home glucose monitoring; follow diet and physical activity regimens; tolerate minor side effects and the risk of more serious complications from medications; regularly attend physician visits for testing and examinations; and absorb costs not covered by insurance for physicians and medical supplies, lost work (or school), and transportation. In many cases these inconveniences, discomforts, and costs are borne over a number of years, often a lifetime. RCTs have shown no adverse association between these efforts and quality of life,⁴⁹ however, and one study suggested that glycemic control might improve quality of life and work productivity.⁵⁰

Modeling Estimates

Mathematical models, largely based on the DCCT, have attempted to estimate the magnitude of benefits and harms from glycemic control. One model estimated that patients with type 2 diabetes who maintained a glycated hemoglobin level of 7.2% would reduce their cumulative lifetime risk of blindness, end-stage renal disease, and lower-extremity amputation by 72% (from 19% to 5%), 87% (from 17% to 2%), and 67% (from 15% to 5%), respectively.⁵¹ Life expectancy would increase by 1.39 years. A Markov model estimated that reducing glycated hemoglobin from 9% to 7% in a patient in whom diabetes developed at age 45 years would lower the lifetime risk of blindness from 2.6% to 0.3%.⁵² The same change in a patient who developed diabetes at age 65 years would decrease the risk of blindness from only 0.5% to <0.1%.

In theory, such projections could be useful to clinicians and patients to estimate the benefits and harms of different levels of glycemic control in individual situations. Since there are different designs and assumptions, however, available models offer discrepant predictions about the same types of patients. For example, the lifetime risk of blindness in a white patient aged 55 years who lowers his glycated hemoglobin level from 9% to 7% would, according to one model, be reduced by 5.6% (from 9% to 3.4%)⁵¹ and, according to another model by 1.1% (from 1.2% to 0.1%).⁵² These discrepancies must be reconciled before reliable outcome estimates can be introduced with confidence in practice.

Weighing the Magnitude of Benefit

The evidence demonstrates a continuous and curvilinear relationship between hyperglycemia and the microvascular and neuropathic complications of diabetes, with risk rising progressively as mean blood glucose concentrations increase. RCTs confirm that for both type 1 and type 2 diabetes glycemic control significantly reduces the incidence of microvascular complications. The following points should be considered when applying this evidence to routine practice:

1. The intensity of treatment in RCTs may be difficult to replicate to the same degree in community practice. In the DCCT, for example, patients received insulin by injection 3 times daily or by external pump, self-monitored blood glucose at least 4 times per day, underwent weekly nocturnal blood glucose measurements, visited their study center monthly, and received frequent telephone calls. The target glycated hemoglobin value was less than 6.1%.⁹ More typical treatment practices were followed in the UKPDS. Although some practices and health care systems have successfully achieved satisfactory blood glucose levels through aggressive programs that assist clinicians and patients, other constraints and the inability or reluctance of patients to adhere to treatment protocols remain problems in other settings.⁵³
2. The microvascular end points in most RCTs were primarily intermediate (eg, ETDRS scales, nerve conduction velocity, urinary albumin excretion) or surrogate (eg, laser phototherapy) outcomes rather than health outcomes. Few trials were designed to measure health outcomes, providing limited data on the extent to which the symptoms that patients experience (eg, visual impairment, paresthesias, complications of renal failure) are reduced by intensive treatment. Such complaints generally do not occur until the patient has end-stage disease and are often forestalled by early treatment (eg, laser phototherapy). It is reasonable to infer that long-term benefits result from glycemic control-the intermediate end points affected are known risk factors for clinical disease-but one cannot assume that the observed magnitude of risk reduction for intermediate outcomes applies also to symptomatic disease.
3. Relative risk reductions are greater than absolute risk reductions. The 25% relative reduction in microvascular complications reported by the UKPDS represents an absolute reduction of only 2%: 8%, rather than 10%, of patients had complications during 10 years of treatment.¹⁰ Also, relative risk reductions generally refer to intermediate outcomes. The 76% reduction in the risk of retinopathy reported by the DCCT refers to a 3-step change on the ETDRS scale, not to improved vision.⁹ The number needed to treat to affect outcomes perceptible to patients is necessarily higher than that for retinopathy, delayed nerve conduction, or elevated urinary albumin excretion, because only a subset of patients with these intermediate outcomes go on to develop symptomatic disease.¹⁵ According to the UKPDS, 37 patients would require intensive treatment for 10 years to prevent one patient from undergoing laser treatment; 208 would require treatment to prevent one case of blindness.¹⁰ The observed 16% difference in the incidence of blindness in the UKPDS was not statistically significant.On the other hand, when examined at the population level, even modest absolute risk reductions can translate into large numbers of persons in society for whom clinical benefit is achievable. Given the millions of people in the United States with type 2 diabetes, even a 2% absolute reduction in the risk of microvascular complications represents many thousands of people who would benefit from glycemic control.
4. Because of the average time required for glycemic control to affect outcomes, some patients with diabetes may not live long enough to benefit, because of the competing risks of death of macrovascular complications and other comorbid diseases. Although elevated blood glucose levels are a likely risk factor for cardiovascular disease, glycemic control has not been shown to enhance life expectancy or prevent heart disease. The 16% reduction in myocardial infarction reported by the UKPDS was of borderline statistical significance.¹⁰ Another RCT did find that improved glycemic control reduced the incidence of ischemic cardiac events, stroke, and cardiovascular deaths in patients with acute myocardial infarction.⁴⁸ Two other trials that failed to show a benefit may have lacked adequate duration and sample size.^16,40
5. For any given patient, the absolute magnitude of risk reduction is a continuous variable that is a function of the patient’s current glycated hemoglobin level, the duration and magnitude of previous hyperglycemia, and the extent of preexisting microvascular complications. The probability that the patient will live long enough to experience the benefits of reduced complications depends on cardiovascular risk factors other than blood glucose (eg, smoking, hypertension, lipid levels, physical inactivity, obesity, preexisting coronary artery disease) and other determinants of life expectancy (eg, age, coexisting diseases, health status). Of these, the most critical variable is the patient’s current glycated hemoglobin level. Because of their increased risk of complications, individuals with marked elevations generally benefit more (in absolute terms) from the same absolute reduction in glycated hemoglobin levels than do individuals with mild to moderate elevations.⁵³ Although it is obviously important for clinicians to keep patients from progressing from mild (eg, hemoglobin A1c levels of 6% to 8%) to marked hyperglycemia (eg, hemoglobin A1c levels >9.5%), in those patients who have already developed marked hyperglycemia, efforts directed at achieving even moderate control (eg, hemoglobin A1c levels of 8% to 9.5%) will yield greater health benefits than pursuing euglycemia in patients with moderate elevations.

Recommendations for clinical practice

For any patient with type 2 diabetes, the better the glycemic control, the lower the probability of chronic microvascular, neuropathic, and possibly cardiovascular complications. However, because of differences in patients’ life expectancies, comorbidities, and preferences, it is inappropriate to set a uniform target glycated hemoglobin level for all patients. Individuals with long life expectancies and few comorbidities may wish to pursue euglycemia, but less vigorous goals may be appropriate for others, such as patients with multiple comorbid conditions or with limited life expectancies.

Whether the magnitude of benefit of a given treatment goal justifies the potential inconvenience, harms, and costs involves value judgments that must be tailored to the individual patient. Patients’ personal risk profiles and capabilities and the relative importance they assign to the potential outcomes and supporting evidence are integral in determining how intensively to treat.

Cardiovascular disease is the most likely cause of death in patients with type 2 diabetes, and attention to glycemic control should not distract clinicians and patients from other interventions that may be more effective in preventing coronary artery disease and stroke. These include smoking cessation, serum lipid management, control of blood pressure, diet, physical activity, and weight management. Guidelines for the control of these risk factors appear elsewhere.^54-56 Clinicians should also pursue treatments other than glycemic control for preventing microvascular complications (eg, blood pressure control, angiotensin-converting enzyme inhibitors for diabetic nephropathy, laser treatment for diabetic retinopathy).

Whatever the desired goals and intensity of treatment, patients face considerable barriers in implementing recommendations. Modifying diet and other personal habits; complying with self-monitoring, medication, and home care; and returning for follow-up visits are difficult. Physicians should work with patients to overcome remediable barriers and should use recommended techniques for patient education and counseling to offer the necessary information and motivation for meaningful change.⁵⁷

Acknowledgments

The systematic review on which this guideline is based was supported in part by funding from the Health Care Financing Administration. We thank Richard D. Kahn, PhD, (American Diabetes Association), and Herbert F. Young, MD, and Bellinda Schoof (American Academy of Family Physicians) for their assistance, as well as the expert panel that externally reviewed the full report: Eugene Barrett, MD; John A. Colwell, MD; Richard C. Eastman, MD; Saul Genuth, MD; Ronald Klein, MD, MPH; Martin Mahoney, MD; James W. Mold, MD; David M. Nathan, MD; Jonathan E. Rodnick, MD; Jeffrey L. Susman, MD; Sandeep Vijan, MD, MS; and Bruce Zimmerman, MD. Their participation in the review process does not necessarily imply endorsement of the report or its recommendations.

An estimated 16 million people in the United States have diabetes.¹ Its microvascular complications (retinopathy, nephropathy, neuropathy) are a leading cause of blindness among adults,² end-stage renal disease,³ and lower-extremity amputations.⁴ Its macrovascular complications pose an even greater public health burden, increasing the risk of coronary artery disease, stroke, and peripheral vascular disease.⁵ Each year diabetes costs the country an estimated $90 billion to $99 billion.^6,7

Type 2 diabetes, which accounts for 90% to 95% of diabetes cases,⁸ differs from type 1 disease in average age of onset and etiology. In both forms, however, the underlying cause of microvascular (and possibly macrovascular) complications appears to be chronic elevations in blood glucose concentrations. The overriding factors in predicting microvascular pathogenesis have less to do with the type of diabetes than with the number of years the patient has had hyperglycemia and the magnitude of the glucose elevation.

In recent years, 2 major randomized controlled trials (RCTs), the Diabetes Control and Complications Trial⁹ (DCCT) in patients with type 1 disease and the United Kingdom Prospective Diabetes Study¹⁰ (UKPDS) in patients with type 2 diabetes, have shown that microvascular complications can be reduced significantly in patients who achieve normal or near-normal blood glucose levels. There is now agreement in the medical community about the importance of lowering markedly elevated blood glucose levels.

The incremental benefit of tight glycemic control (as opposed to less intensive therapy) varies across patient groups, however. Years are required for microvascular complications to progress to symptomatic disease. Patients with type 1 diabetes, who are generally younger, are more likely to live long enough to benefit from tight glycemic control than patients with type 2 disease, who face a shorter life expectancy because of their age and risk of cardiovascular disease. For patients with coexistent diseases, the delayed benefits of glycemic control may be offset by the more immediate inconvenience, complications, and costs of intensive treatment and by the health effects of comorbid conditions.

These generalizations do not apply to all patients. The older age of onset of type 2 diabetes is a population average with a wide distribution. Many patients with type 2 diabetes live long enough to experience significant microvascular disease and may benefit from glycemic control. Also, interventions that extend life expectancy (eg, smoking cessation, blood pressure control, and lipid control) give patients more time to encounter advanced microvascular disease and an opportunity to benefit from treatment.

In 1996, the American Academy of Family Physicians convened a panel to conduct a systematic review of the evidence of the benefits and harms of glycemic control in type 2 diabetes and to develop evidence-based recommendations. The 9-member panel included family physicians, general internists, endocrinologists, and a practice guidelines methodologist. Four members were appointed by the American Diabetes Association and the American College of Physicians. We summarize the panel’s findings, which are available in a full report.*

Methods

Systematic Review

The review methods are provided in detail in the full report. The literature search retrieved published evidence on the effects of glycemic control on microvascular and macrovascular complications in type 1 and type 2 diabetes and on adverse effects. RCT evidence for type 1 diabetes was considered relevant in evaluating the effects of glycemic control in type 2 disease. A total of 798 citations met initial inclusion criteria. All articles underwent structured abstraction. We closed the search with the publication of the UKPDS results in September 1998.

In reviewing the evidence, the panel gave greater weight to RCTs than to observational studies and emphasized data on health outcomes perceptible to patients (eg, visual acuity) over those for intermediate or surrogate end points (eg, retinopathy) that precede or are associated with such outcomes. Most trials did not designate health outcomes as primary end points and therefore lacked the statistical power and duration to prove an effect. Panel recommendations were evidence-based and did not reflect expert opinion. Fourteen outside diabetologists and family physicians externally reviewed the full report, and revisions consistent with the evidence were adopted. The American Academy of Family Physicians and the American Diabetes Association endorsed the full report in March 1999 and this policy statement in October 1999.

We focused the review on the benefits of glycemic control in general and not on specific agents (eg, sulfonylureas, metformin). Interventions not associated with glycemic control (eg, laser phototherapy, angiotensin-converting enzyme inhibitors), which also mitigate the effects of microvascular disease, were not examined in our review.

Results

Microvascular Outcomes

Evidence from Observational Studies. Many cross-sectional studies indicate that people with type 2 diabetes who have higher plasma glucose or glycated hemoglobin (e.g., hemoglobin A1c) levels are more likely to have evidence of retinopathy, neuropathy, or albuminuria.¹¹ Numerous prospective longitudinal studies also show that an elevated fasting plasma glucose (FPG) concentration or glycated hemoglobin level at baseline or over time increases the chances that type 2 patients will develop new or worsened retinopathy, abnormal electrophysiologic findings, or renal dysfunction.^12-15 However, observational data, unlike evidence from RCTs, does not prove that lowering blood glucose levels reduces the incidence of these complications.

Evidence from RCTs. Ten RCTs do provide this evidence.^9,10,16-24 Three trials involved patients with type 2 diabetes: the very large UKPDS¹⁰ (approximately 4200 patients) and 2 small Japanese studies.^16,19 The largest of the 7 trials of patients with type 1 diabetes was the DCCT (1441 patients).⁹ In most trials, patients were randomly allocated to an intensive treatment group that received multiple or continuous insulin administrations or to a control group that received conventional insulin therapy. Most studies confirmed (through mean glycated hemoglobin levels) better glycemic control with intensive treatment. For example, mean hemoglobin A1c levels in the intensive/conservative treatment groups of the DCCT and UKPDS were 7.2%/9.1% and 7.0%/7.9%, respectively.^9,10 An old RCT (University Group Diabetes Program) that did not produce significant differences in glucose control in some treatment arms and lacked statistical power was excluded from our review.²⁵ Average lengths of follow-up in the RCTs ranged from 2 to 12.5 years (6.5 and 10 years, respectively, in the DCCT and UKPDS).

Retinopathy in Patients with Type 2 Diabetes. RCTs provide good evidence that glycemic control reduces the incidence of retinopathy. In a Japanese trial, among patients with no retinopathy at baseline, the 6-year incidence of new disease (progression Ž2 steps on the Early Treatment Diabetic Retinopathy Study [ETDRS] scale²⁶) in the intensive and conservative treatment groups was 6% and 36%, respectively, a relative reduction of 83%. In patients with retinopathy at baseline (secondary prevention group), the incidence rates for intensive and conservative treatment groups were 17% and 44%, respectively.¹⁶

The authors of the UKPDS results, who reported a 25% relative reduction in the incidence of microvascular complications (11.4 vs 8.6 events/1000 patient-years) attributed much of the benefit to reduced retinopathy.¹⁰ The need for laser therapy was lowered from 11.0 to 7.9 events/1000-patient years, a 29% relative reduction. Within 6 years, the incidence of a 2-step progression on the ETDRS scale was lowered from 28% to 23%. The relative reduction in cataract extraction was 24%. The incidence of decreased visual acuity, blindness, and vitreous hemorrhage was not lowered significantly. The extent to which the latter resulted from treatment for early complications is not known.

Retinopathy in Patients with Type 1 Diabetes. Among patients with no baseline retinopathy in the DCCT (primary prevention group), the 6.5-year incidence of a sustained 3-step change on the ETDRS scale was reduced by 76% (from 4.7 to 1.2/100 patient-years).⁹ In the secondary prevention group, the rate of progression was lowered by 54% (from 7.8 to 3.7/100 patient-years). In this group intensive treatment was also associated with a lower incidence of severe retinopathy, need for laser treatment, and sustained progression worsening for at least 6 months (adjusted relative risk reduction=47%, 56%, and 65%, respectively).^9,27 A Swedish trial reported improved ETDRS scales and a lower prevalence of visual impairment (14% vs 35%) at 7.8 years median follow-up.¹⁷

Peripheral Neuropathy in Patients with Type 2 Diabetes. Some trials suggest that lowering blood glucose improves isolated electrophysiologic measures.^10,16,19 A Japanese RCT¹⁶ reported an increase in median nerve conduction velocity and a reduction in arm vibration threshold, but other physiologic measures were unaffected. Neurologic symptoms were not measured. The UKPDS¹⁰ showed no effect on the incidence of absent ankle and knee reflexes, but abnormal biothesiometery data (for toes) occurred less frequently with intensive treatment. Impotence and heart rate responses to deep breathing and standing occurred with equal frequency.

Peripheral Neuropathy in Patients with Type 1 Diabetes. The DCCT showed a 69% reduction (9.8 vs 3.1/100 patient-years) in newly “confirmed clinical neuropathy” (abnormal neurologic history or physical examination combined with abnormal nerve conduction or autonomic nervous system studies), but the incidence of neurologic symptoms was not reported.^9,28 Progression of clinical neuropathy in patients with preexisting disease was reduced by 57%, from 16.1 to 7.0/100 patient-years. A Swedish RCT reported a lower incidence of neuropathic symptoms at 10-year follow-up (14% vs 32%) and higher pin-prick sensitivity.²⁹

Nephropathy in Patients with Type 2 Diabetes. Intensive insulin treatment does appear to reduce the incidence of albuminuria.^10,16,19 The UKPDS observed a lower incidence of microalbuminuria within 3 years and a lower incidence of gross proteinuria and increased plasma creatinine within 9 years of follow-up (relative risk reduction=17%, 33%, and 60%, respectively).¹⁰ The incidence rates of renal failure and death from renal disease did not differ significantly between the groups, but the absolute number of cases was small.

Nephropathy in Patients with Type 1 Diabetes. In the DCCT, among patients who lacked microalbuminuria at baseline, the incidence of new cases over 6.5 years was reduced by 34% (3.4 vs 2.2/100,000 patient-years), but the incidence of sustained microalbuminuria, macroalbuminuria, or abnormal creatinine clearance did not differ. In the secondary prevention group, the incidence of microalbuminuria was reduced from 5.7 to 3.6/100,000 patient-years, a 43% relative reduction, and the incidence of sustained microalbuminuria and of macroalbuminuria was also reduced.^9,30

Macrovascular Outcomes

Observational Studies. Some cross-sectional studies in type 2 diabetes report that elevated FPG concentrations or glycated hemoglobin levels are more common in people with coronary artery disease, an abnormal electrocardiogram, or cardiovascular disease.^31,32 Longitudinal studies show that patients with elevated blood glucose, glycated hemoglobin, or postprandial glucose levels at baseline are more likely to develop coronary artery disease or an abnormal electrocardiogram or to die of coronary artery or cardiovascular disease.^33-36 An association between blood glucose concentration and stroke or peripheral vascular disease (eg, incidence of amputation and foot ulcers) has also been demonstrated in such studies^{31,33,34,37,38} but less consistently.

Clinical Trials of Patients with Type 2 Diabetes. The UKPDS showed a 16% reduction in the 10-year incidence of myocardial infarction with intensive treatment, a difference of borderline statistical significance (P=.05, 95% confidence interval for relative risk=0.71-1.00).¹⁰ Statistically significant differences were noted in certain subgroups.³⁹ Sudden death was less common (relative risk reduction=46%, P=.05), but the incidence of fatal myocardial infarction, heart failure, angina, stroke, amputation, and death from peripheral vascular disease was unchanged.¹⁰ The authors noted that the study lacked statistical power to exclude an effect on fatal outcomes.

Another RCT reported no significant effect on cardiovascular events or mortality with intensive treatment, but the mean follow-up period was only 27 months.⁴⁰ A British trial involving patients with moderate hyperglycemia reported that cardiovascular events occurred less frequently in a group given high-dose tolbutamide and a recommended diet than in the control group, but the patient population, type of diabetes, and outcome measures were defined imprecisely.⁴¹

Clinical Trials of Patients with Type 1 Diabetes. The incidence of major cardiovascular and peripheral vascular events in most trials did not differ significantly with intensive treatment,^9,42 but the number of cases and length of follow-up were generally too small to detect a difference.

All-Cause Mortality

Observational Studies. Some observational studies report an association between poor glycemic control and all-cause mortality or overall survival rates in type 2 diabetes.^35,36,43-45 However, other cohort studies report that death rates are not reliably predicted by FPG concentrations or glycated hemoglobin levels.^46,47

Clinical Trials of Patients with Type 2 Diabetes. A Swedish RCT found that patients with diabetes admitted to coronary care units for recent myocardial infarction achieved better glycemic control and experienced significantly lower all-cause mortality (33% vs 44%) if they received intensive insulin therapy (insulin-glucose infusion for the first 24 hours and subcutaneous insulin 4 times daily for 3 months).⁴⁸ The UKPDS showed no significant effect on either all-cause or diabetes-related mortality but lacked statistical power to exclude an effect.¹⁰

Clinical Trials of Patients with Type 1 Diabetes. There are few data on all-cause mortality in patients with type 1 diabetes because of the low event rate.^9,42

Discussion

Potential Harms of Intensive Glycemic Control

Specific complications can occur with each of the agents used to treat type 2 diabetes: Insulin has potential adverse effects, and oral glucose-lowering drugs carry some risk of undesirable side effects and uncommon but serious complications (eg, lactic acidosis, hepatotoxicity).

Attempts to achieve euglycemia can increase the risk of hypoglycemia, and some medications are associated with weight gain. The risk of severe hypoglycemia is greatest for patients with type 1 diabetes. In the subjects with type 2 diabetes in the UKPDS, the incidence of major hypoglycemic episodes was higher among the intensively treated than conventionally treated patients, but the rate was low (1% to 2%).¹⁰ More typically in type 2 disease, a more substantial risk exists for minor hypoglycemic episodes, which are usually inconsequential.^10,40 An association between intensive treatment and weight gain has been reported (mean=3.1 kg and 4.6 kg in the UKPDS and DCCT, respectively),^9,10 but there is no evidence that this amount of weight gain affects outcomes.

Intensive treatment requires that patients perform home glucose monitoring; follow diet and physical activity regimens; tolerate minor side effects and the risk of more serious complications from medications; regularly attend physician visits for testing and examinations; and absorb costs not covered by insurance for physicians and medical supplies, lost work (or school), and transportation. In many cases these inconveniences, discomforts, and costs are borne over a number of years, often a lifetime. RCTs have shown no adverse association between these efforts and quality of life,⁴⁹ however, and one study suggested that glycemic control might improve quality of life and work productivity.⁵⁰

Modeling Estimates

Mathematical models, largely based on the DCCT, have attempted to estimate the magnitude of benefits and harms from glycemic control. One model estimated that patients with type 2 diabetes who maintained a glycated hemoglobin level of 7.2% would reduce their cumulative lifetime risk of blindness, end-stage renal disease, and lower-extremity amputation by 72% (from 19% to 5%), 87% (from 17% to 2%), and 67% (from 15% to 5%), respectively.⁵¹ Life expectancy would increase by 1.39 years. A Markov model estimated that reducing glycated hemoglobin from 9% to 7% in a patient in whom diabetes developed at age 45 years would lower the lifetime risk of blindness from 2.6% to 0.3%.⁵² The same change in a patient who developed diabetes at age 65 years would decrease the risk of blindness from only 0.5% to <0.1%.

In theory, such projections could be useful to clinicians and patients to estimate the benefits and harms of different levels of glycemic control in individual situations. Since there are different designs and assumptions, however, available models offer discrepant predictions about the same types of patients. For example, the lifetime risk of blindness in a white patient aged 55 years who lowers his glycated hemoglobin level from 9% to 7% would, according to one model, be reduced by 5.6% (from 9% to 3.4%)⁵¹ and, according to another model by 1.1% (from 1.2% to 0.1%).⁵² These discrepancies must be reconciled before reliable outcome estimates can be introduced with confidence in practice.

Weighing the Magnitude of Benefit

The evidence demonstrates a continuous and curvilinear relationship between hyperglycemia and the microvascular and neuropathic complications of diabetes, with risk rising progressively as mean blood glucose concentrations increase. RCTs confirm that for both type 1 and type 2 diabetes glycemic control significantly reduces the incidence of microvascular complications. The following points should be considered when applying this evidence to routine practice:

1. The intensity of treatment in RCTs may be difficult to replicate to the same degree in community practice. In the DCCT, for example, patients received insulin by injection 3 times daily or by external pump, self-monitored blood glucose at least 4 times per day, underwent weekly nocturnal blood glucose measurements, visited their study center monthly, and received frequent telephone calls. The target glycated hemoglobin value was less than 6.1%.⁹ More typical treatment practices were followed in the UKPDS. Although some practices and health care systems have successfully achieved satisfactory blood glucose levels through aggressive programs that assist clinicians and patients, other constraints and the inability or reluctance of patients to adhere to treatment protocols remain problems in other settings.⁵³
2. The microvascular end points in most RCTs were primarily intermediate (eg, ETDRS scales, nerve conduction velocity, urinary albumin excretion) or surrogate (eg, laser phototherapy) outcomes rather than health outcomes. Few trials were designed to measure health outcomes, providing limited data on the extent to which the symptoms that patients experience (eg, visual impairment, paresthesias, complications of renal failure) are reduced by intensive treatment. Such complaints generally do not occur until the patient has end-stage disease and are often forestalled by early treatment (eg, laser phototherapy). It is reasonable to infer that long-term benefits result from glycemic control-the intermediate end points affected are known risk factors for clinical disease-but one cannot assume that the observed magnitude of risk reduction for intermediate outcomes applies also to symptomatic disease.
3. Relative risk reductions are greater than absolute risk reductions. The 25% relative reduction in microvascular complications reported by the UKPDS represents an absolute reduction of only 2%: 8%, rather than 10%, of patients had complications during 10 years of treatment.¹⁰ Also, relative risk reductions generally refer to intermediate outcomes. The 76% reduction in the risk of retinopathy reported by the DCCT refers to a 3-step change on the ETDRS scale, not to improved vision.⁹ The number needed to treat to affect outcomes perceptible to patients is necessarily higher than that for retinopathy, delayed nerve conduction, or elevated urinary albumin excretion, because only a subset of patients with these intermediate outcomes go on to develop symptomatic disease.¹⁵ According to the UKPDS, 37 patients would require intensive treatment for 10 years to prevent one patient from undergoing laser treatment; 208 would require treatment to prevent one case of blindness.¹⁰ The observed 16% difference in the incidence of blindness in the UKPDS was not statistically significant.On the other hand, when examined at the population level, even modest absolute risk reductions can translate into large numbers of persons in society for whom clinical benefit is achievable. Given the millions of people in the United States with type 2 diabetes, even a 2% absolute reduction in the risk of microvascular complications represents many thousands of people who would benefit from glycemic control.
4. Because of the average time required for glycemic control to affect outcomes, some patients with diabetes may not live long enough to benefit, because of the competing risks of death of macrovascular complications and other comorbid diseases. Although elevated blood glucose levels are a likely risk factor for cardiovascular disease, glycemic control has not been shown to enhance life expectancy or prevent heart disease. The 16% reduction in myocardial infarction reported by the UKPDS was of borderline statistical significance.¹⁰ Another RCT did find that improved glycemic control reduced the incidence of ischemic cardiac events, stroke, and cardiovascular deaths in patients with acute myocardial infarction.⁴⁸ Two other trials that failed to show a benefit may have lacked adequate duration and sample size.^16,40
5. For any given patient, the absolute magnitude of risk reduction is a continuous variable that is a function of the patient’s current glycated hemoglobin level, the duration and magnitude of previous hyperglycemia, and the extent of preexisting microvascular complications. The probability that the patient will live long enough to experience the benefits of reduced complications depends on cardiovascular risk factors other than blood glucose (eg, smoking, hypertension, lipid levels, physical inactivity, obesity, preexisting coronary artery disease) and other determinants of life expectancy (eg, age, coexisting diseases, health status). Of these, the most critical variable is the patient’s current glycated hemoglobin level. Because of their increased risk of complications, individuals with marked elevations generally benefit more (in absolute terms) from the same absolute reduction in glycated hemoglobin levels than do individuals with mild to moderate elevations.⁵³ Although it is obviously important for clinicians to keep patients from progressing from mild (eg, hemoglobin A1c levels of 6% to 8%) to marked hyperglycemia (eg, hemoglobin A1c levels >9.5%), in those patients who have already developed marked hyperglycemia, efforts directed at achieving even moderate control (eg, hemoglobin A1c levels of 8% to 9.5%) will yield greater health benefits than pursuing euglycemia in patients with moderate elevations.

Recommendations for clinical practice

For any patient with type 2 diabetes, the better the glycemic control, the lower the probability of chronic microvascular, neuropathic, and possibly cardiovascular complications. However, because of differences in patients’ life expectancies, comorbidities, and preferences, it is inappropriate to set a uniform target glycated hemoglobin level for all patients. Individuals with long life expectancies and few comorbidities may wish to pursue euglycemia, but less vigorous goals may be appropriate for others, such as patients with multiple comorbid conditions or with limited life expectancies.

Whether the magnitude of benefit of a given treatment goal justifies the potential inconvenience, harms, and costs involves value judgments that must be tailored to the individual patient. Patients’ personal risk profiles and capabilities and the relative importance they assign to the potential outcomes and supporting evidence are integral in determining how intensively to treat.

Cardiovascular disease is the most likely cause of death in patients with type 2 diabetes, and attention to glycemic control should not distract clinicians and patients from other interventions that may be more effective in preventing coronary artery disease and stroke. These include smoking cessation, serum lipid management, control of blood pressure, diet, physical activity, and weight management. Guidelines for the control of these risk factors appear elsewhere.^54-56 Clinicians should also pursue treatments other than glycemic control for preventing microvascular complications (eg, blood pressure control, angiotensin-converting enzyme inhibitors for diabetic nephropathy, laser treatment for diabetic retinopathy).

Whatever the desired goals and intensity of treatment, patients face considerable barriers in implementing recommendations. Modifying diet and other personal habits; complying with self-monitoring, medication, and home care; and returning for follow-up visits are difficult. Physicians should work with patients to overcome remediable barriers and should use recommended techniques for patient education and counseling to offer the necessary information and motivation for meaningful change.⁵⁷

Acknowledgments

The systematic review on which this guideline is based was supported in part by funding from the Health Care Financing Administration. We thank Richard D. Kahn, PhD, (American Diabetes Association), and Herbert F. Young, MD, and Bellinda Schoof (American Academy of Family Physicians) for their assistance, as well as the expert panel that externally reviewed the full report: Eugene Barrett, MD; John A. Colwell, MD; Richard C. Eastman, MD; Saul Genuth, MD; Ronald Klein, MD, MPH; Martin Mahoney, MD; James W. Mold, MD; David M. Nathan, MD; Jonathan E. Rodnick, MD; Jeffrey L. Susman, MD; Sandeep Vijan, MD, MS; and Bruce Zimmerman, MD. Their participation in the review process does not necessarily imply endorsement of the report or its recommendations.

An estimated 16 million people in the United States have diabetes.¹ Its microvascular complications (retinopathy, nephropathy, neuropathy) are a leading cause of blindness among adults,² end-stage renal disease,³ and lower-extremity amputations.⁴ Its macrovascular complications pose an even greater public health burden, increasing the risk of coronary artery disease, stroke, and peripheral vascular disease.⁵ Each year diabetes costs the country an estimated $90 billion to $99 billion.^6,7

Type 2 diabetes, which accounts for 90% to 95% of diabetes cases,⁸ differs from type 1 disease in average age of onset and etiology. In both forms, however, the underlying cause of microvascular (and possibly macrovascular) complications appears to be chronic elevations in blood glucose concentrations. The overriding factors in predicting microvascular pathogenesis have less to do with the type of diabetes than with the number of years the patient has had hyperglycemia and the magnitude of the glucose elevation.

In recent years, 2 major randomized controlled trials (RCTs), the Diabetes Control and Complications Trial⁹ (DCCT) in patients with type 1 disease and the United Kingdom Prospective Diabetes Study¹⁰ (UKPDS) in patients with type 2 diabetes, have shown that microvascular complications can be reduced significantly in patients who achieve normal or near-normal blood glucose levels. There is now agreement in the medical community about the importance of lowering markedly elevated blood glucose levels.

The incremental benefit of tight glycemic control (as opposed to less intensive therapy) varies across patient groups, however. Years are required for microvascular complications to progress to symptomatic disease. Patients with type 1 diabetes, who are generally younger, are more likely to live long enough to benefit from tight glycemic control than patients with type 2 disease, who face a shorter life expectancy because of their age and risk of cardiovascular disease. For patients with coexistent diseases, the delayed benefits of glycemic control may be offset by the more immediate inconvenience, complications, and costs of intensive treatment and by the health effects of comorbid conditions.

These generalizations do not apply to all patients. The older age of onset of type 2 diabetes is a population average with a wide distribution. Many patients with type 2 diabetes live long enough to experience significant microvascular disease and may benefit from glycemic control. Also, interventions that extend life expectancy (eg, smoking cessation, blood pressure control, and lipid control) give patients more time to encounter advanced microvascular disease and an opportunity to benefit from treatment.

In 1996, the American Academy of Family Physicians convened a panel to conduct a systematic review of the evidence of the benefits and harms of glycemic control in type 2 diabetes and to develop evidence-based recommendations. The 9-member panel included family physicians, general internists, endocrinologists, and a practice guidelines methodologist. Four members were appointed by the American Diabetes Association and the American College of Physicians. We summarize the panel’s findings, which are available in a full report.*

Methods

Systematic Review

The review methods are provided in detail in the full report. The literature search retrieved published evidence on the effects of glycemic control on microvascular and macrovascular complications in type 1 and type 2 diabetes and on adverse effects. RCT evidence for type 1 diabetes was considered relevant in evaluating the effects of glycemic control in type 2 disease. A total of 798 citations met initial inclusion criteria. All articles underwent structured abstraction. We closed the search with the publication of the UKPDS results in September 1998.

In reviewing the evidence, the panel gave greater weight to RCTs than to observational studies and emphasized data on health outcomes perceptible to patients (eg, visual acuity) over those for intermediate or surrogate end points (eg, retinopathy) that precede or are associated with such outcomes. Most trials did not designate health outcomes as primary end points and therefore lacked the statistical power and duration to prove an effect. Panel recommendations were evidence-based and did not reflect expert opinion. Fourteen outside diabetologists and family physicians externally reviewed the full report, and revisions consistent with the evidence were adopted. The American Academy of Family Physicians and the American Diabetes Association endorsed the full report in March 1999 and this policy statement in October 1999.

We focused the review on the benefits of glycemic control in general and not on specific agents (eg, sulfonylureas, metformin). Interventions not associated with glycemic control (eg, laser phototherapy, angiotensin-converting enzyme inhibitors), which also mitigate the effects of microvascular disease, were not examined in our review.

Results

Microvascular Outcomes

Evidence from Observational Studies. Many cross-sectional studies indicate that people with type 2 diabetes who have higher plasma glucose or glycated hemoglobin (e.g., hemoglobin A1c) levels are more likely to have evidence of retinopathy, neuropathy, or albuminuria.¹¹ Numerous prospective longitudinal studies also show that an elevated fasting plasma glucose (FPG) concentration or glycated hemoglobin level at baseline or over time increases the chances that type 2 patients will develop new or worsened retinopathy, abnormal electrophysiologic findings, or renal dysfunction.^12-15 However, observational data, unlike evidence from RCTs, does not prove that lowering blood glucose levels reduces the incidence of these complications.

Evidence from RCTs. Ten RCTs do provide this evidence.^9,10,16-24 Three trials involved patients with type 2 diabetes: the very large UKPDS¹⁰ (approximately 4200 patients) and 2 small Japanese studies.^16,19 The largest of the 7 trials of patients with type 1 diabetes was the DCCT (1441 patients).⁹ In most trials, patients were randomly allocated to an intensive treatment group that received multiple or continuous insulin administrations or to a control group that received conventional insulin therapy. Most studies confirmed (through mean glycated hemoglobin levels) better glycemic control with intensive treatment. For example, mean hemoglobin A1c levels in the intensive/conservative treatment groups of the DCCT and UKPDS were 7.2%/9.1% and 7.0%/7.9%, respectively.^9,10 An old RCT (University Group Diabetes Program) that did not produce significant differences in glucose control in some treatment arms and lacked statistical power was excluded from our review.²⁵ Average lengths of follow-up in the RCTs ranged from 2 to 12.5 years (6.5 and 10 years, respectively, in the DCCT and UKPDS).

Retinopathy in Patients with Type 2 Diabetes. RCTs provide good evidence that glycemic control reduces the incidence of retinopathy. In a Japanese trial, among patients with no retinopathy at baseline, the 6-year incidence of new disease (progression Ž2 steps on the Early Treatment Diabetic Retinopathy Study [ETDRS] scale²⁶) in the intensive and conservative treatment groups was 6% and 36%, respectively, a relative reduction of 83%. In patients with retinopathy at baseline (secondary prevention group), the incidence rates for intensive and conservative treatment groups were 17% and 44%, respectively.¹⁶

The authors of the UKPDS results, who reported a 25% relative reduction in the incidence of microvascular complications (11.4 vs 8.6 events/1000 patient-years) attributed much of the benefit to reduced retinopathy.¹⁰ The need for laser therapy was lowered from 11.0 to 7.9 events/1000-patient years, a 29% relative reduction. Within 6 years, the incidence of a 2-step progression on the ETDRS scale was lowered from 28% to 23%. The relative reduction in cataract extraction was 24%. The incidence of decreased visual acuity, blindness, and vitreous hemorrhage was not lowered significantly. The extent to which the latter resulted from treatment for early complications is not known.

Retinopathy in Patients with Type 1 Diabetes. Among patients with no baseline retinopathy in the DCCT (primary prevention group), the 6.5-year incidence of a sustained 3-step change on the ETDRS scale was reduced by 76% (from 4.7 to 1.2/100 patient-years).⁹ In the secondary prevention group, the rate of progression was lowered by 54% (from 7.8 to 3.7/100 patient-years). In this group intensive treatment was also associated with a lower incidence of severe retinopathy, need for laser treatment, and sustained progression worsening for at least 6 months (adjusted relative risk reduction=47%, 56%, and 65%, respectively).^9,27 A Swedish trial reported improved ETDRS scales and a lower prevalence of visual impairment (14% vs 35%) at 7.8 years median follow-up.¹⁷

Peripheral Neuropathy in Patients with Type 2 Diabetes. Some trials suggest that lowering blood glucose improves isolated electrophysiologic measures.^10,16,19 A Japanese RCT¹⁶ reported an increase in median nerve conduction velocity and a reduction in arm vibration threshold, but other physiologic measures were unaffected. Neurologic symptoms were not measured. The UKPDS¹⁰ showed no effect on the incidence of absent ankle and knee reflexes, but abnormal biothesiometery data (for toes) occurred less frequently with intensive treatment. Impotence and heart rate responses to deep breathing and standing occurred with equal frequency.

Peripheral Neuropathy in Patients with Type 1 Diabetes. The DCCT showed a 69% reduction (9.8 vs 3.1/100 patient-years) in newly “confirmed clinical neuropathy” (abnormal neurologic history or physical examination combined with abnormal nerve conduction or autonomic nervous system studies), but the incidence of neurologic symptoms was not reported.^9,28 Progression of clinical neuropathy in patients with preexisting disease was reduced by 57%, from 16.1 to 7.0/100 patient-years. A Swedish RCT reported a lower incidence of neuropathic symptoms at 10-year follow-up (14% vs 32%) and higher pin-prick sensitivity.²⁹

Nephropathy in Patients with Type 2 Diabetes. Intensive insulin treatment does appear to reduce the incidence of albuminuria.^10,16,19 The UKPDS observed a lower incidence of microalbuminuria within 3 years and a lower incidence of gross proteinuria and increased plasma creatinine within 9 years of follow-up (relative risk reduction=17%, 33%, and 60%, respectively).¹⁰ The incidence rates of renal failure and death from renal disease did not differ significantly between the groups, but the absolute number of cases was small.

Nephropathy in Patients with Type 1 Diabetes. In the DCCT, among patients who lacked microalbuminuria at baseline, the incidence of new cases over 6.5 years was reduced by 34% (3.4 vs 2.2/100,000 patient-years), but the incidence of sustained microalbuminuria, macroalbuminuria, or abnormal creatinine clearance did not differ. In the secondary prevention group, the incidence of microalbuminuria was reduced from 5.7 to 3.6/100,000 patient-years, a 43% relative reduction, and the incidence of sustained microalbuminuria and of macroalbuminuria was also reduced.^9,30

Macrovascular Outcomes

Observational Studies. Some cross-sectional studies in type 2 diabetes report that elevated FPG concentrations or glycated hemoglobin levels are more common in people with coronary artery disease, an abnormal electrocardiogram, or cardiovascular disease.^31,32 Longitudinal studies show that patients with elevated blood glucose, glycated hemoglobin, or postprandial glucose levels at baseline are more likely to develop coronary artery disease or an abnormal electrocardiogram or to die of coronary artery or cardiovascular disease.^33-36 An association between blood glucose concentration and stroke or peripheral vascular disease (eg, incidence of amputation and foot ulcers) has also been demonstrated in such studies^{31,33,34,37,38} but less consistently.

Clinical Trials of Patients with Type 2 Diabetes. The UKPDS showed a 16% reduction in the 10-year incidence of myocardial infarction with intensive treatment, a difference of borderline statistical significance (P=.05, 95% confidence interval for relative risk=0.71-1.00).¹⁰ Statistically significant differences were noted in certain subgroups.³⁹ Sudden death was less common (relative risk reduction=46%, P=.05), but the incidence of fatal myocardial infarction, heart failure, angina, stroke, amputation, and death from peripheral vascular disease was unchanged.¹⁰ The authors noted that the study lacked statistical power to exclude an effect on fatal outcomes.

Another RCT reported no significant effect on cardiovascular events or mortality with intensive treatment, but the mean follow-up period was only 27 months.⁴⁰ A British trial involving patients with moderate hyperglycemia reported that cardiovascular events occurred less frequently in a group given high-dose tolbutamide and a recommended diet than in the control group, but the patient population, type of diabetes, and outcome measures were defined imprecisely.⁴¹

Clinical Trials of Patients with Type 1 Diabetes. The incidence of major cardiovascular and peripheral vascular events in most trials did not differ significantly with intensive treatment,^9,42 but the number of cases and length of follow-up were generally too small to detect a difference.

All-Cause Mortality

Observational Studies. Some observational studies report an association between poor glycemic control and all-cause mortality or overall survival rates in type 2 diabetes.^35,36,43-45 However, other cohort studies report that death rates are not reliably predicted by FPG concentrations or glycated hemoglobin levels.^46,47

Clinical Trials of Patients with Type 2 Diabetes. A Swedish RCT found that patients with diabetes admitted to coronary care units for recent myocardial infarction achieved better glycemic control and experienced significantly lower all-cause mortality (33% vs 44%) if they received intensive insulin therapy (insulin-glucose infusion for the first 24 hours and subcutaneous insulin 4 times daily for 3 months).⁴⁸ The UKPDS showed no significant effect on either all-cause or diabetes-related mortality but lacked statistical power to exclude an effect.¹⁰

Clinical Trials of Patients with Type 1 Diabetes. There are few data on all-cause mortality in patients with type 1 diabetes because of the low event rate.^9,42

Discussion

Potential Harms of Intensive Glycemic Control

Specific complications can occur with each of the agents used to treat type 2 diabetes: Insulin has potential adverse effects, and oral glucose-lowering drugs carry some risk of undesirable side effects and uncommon but serious complications (eg, lactic acidosis, hepatotoxicity).

Attempts to achieve euglycemia can increase the risk of hypoglycemia, and some medications are associated with weight gain. The risk of severe hypoglycemia is greatest for patients with type 1 diabetes. In the subjects with type 2 diabetes in the UKPDS, the incidence of major hypoglycemic episodes was higher among the intensively treated than conventionally treated patients, but the rate was low (1% to 2%).¹⁰ More typically in type 2 disease, a more substantial risk exists for minor hypoglycemic episodes, which are usually inconsequential.^10,40 An association between intensive treatment and weight gain has been reported (mean=3.1 kg and 4.6 kg in the UKPDS and DCCT, respectively),^9,10 but there is no evidence that this amount of weight gain affects outcomes.

Intensive treatment requires that patients perform home glucose monitoring; follow diet and physical activity regimens; tolerate minor side effects and the risk of more serious complications from medications; regularly attend physician visits for testing and examinations; and absorb costs not covered by insurance for physicians and medical supplies, lost work (or school), and transportation. In many cases these inconveniences, discomforts, and costs are borne over a number of years, often a lifetime. RCTs have shown no adverse association between these efforts and quality of life,⁴⁹ however, and one study suggested that glycemic control might improve quality of life and work productivity.⁵⁰

Modeling Estimates

Mathematical models, largely based on the DCCT, have attempted to estimate the magnitude of benefits and harms from glycemic control. One model estimated that patients with type 2 diabetes who maintained a glycated hemoglobin level of 7.2% would reduce their cumulative lifetime risk of blindness, end-stage renal disease, and lower-extremity amputation by 72% (from 19% to 5%), 87% (from 17% to 2%), and 67% (from 15% to 5%), respectively.⁵¹ Life expectancy would increase by 1.39 years. A Markov model estimated that reducing glycated hemoglobin from 9% to 7% in a patient in whom diabetes developed at age 45 years would lower the lifetime risk of blindness from 2.6% to 0.3%.⁵² The same change in a patient who developed diabetes at age 65 years would decrease the risk of blindness from only 0.5% to <0.1%.

In theory, such projections could be useful to clinicians and patients to estimate the benefits and harms of different levels of glycemic control in individual situations. Since there are different designs and assumptions, however, available models offer discrepant predictions about the same types of patients. For example, the lifetime risk of blindness in a white patient aged 55 years who lowers his glycated hemoglobin level from 9% to 7% would, according to one model, be reduced by 5.6% (from 9% to 3.4%)⁵¹ and, according to another model by 1.1% (from 1.2% to 0.1%).⁵² These discrepancies must be reconciled before reliable outcome estimates can be introduced with confidence in practice.

Weighing the Magnitude of Benefit

The evidence demonstrates a continuous and curvilinear relationship between hyperglycemia and the microvascular and neuropathic complications of diabetes, with risk rising progressively as mean blood glucose concentrations increase. RCTs confirm that for both type 1 and type 2 diabetes glycemic control significantly reduces the incidence of microvascular complications. The following points should be considered when applying this evidence to routine practice:

1. The intensity of treatment in RCTs may be difficult to replicate to the same degree in community practice. In the DCCT, for example, patients received insulin by injection 3 times daily or by external pump, self-monitored blood glucose at least 4 times per day, underwent weekly nocturnal blood glucose measurements, visited their study center monthly, and received frequent telephone calls. The target glycated hemoglobin value was less than 6.1%.⁹ More typical treatment practices were followed in the UKPDS. Although some practices and health care systems have successfully achieved satisfactory blood glucose levels through aggressive programs that assist clinicians and patients, other constraints and the inability or reluctance of patients to adhere to treatment protocols remain problems in other settings.⁵³
2. The microvascular end points in most RCTs were primarily intermediate (eg, ETDRS scales, nerve conduction velocity, urinary albumin excretion) or surrogate (eg, laser phototherapy) outcomes rather than health outcomes. Few trials were designed to measure health outcomes, providing limited data on the extent to which the symptoms that patients experience (eg, visual impairment, paresthesias, complications of renal failure) are reduced by intensive treatment. Such complaints generally do not occur until the patient has end-stage disease and are often forestalled by early treatment (eg, laser phototherapy). It is reasonable to infer that long-term benefits result from glycemic control-the intermediate end points affected are known risk factors for clinical disease-but one cannot assume that the observed magnitude of risk reduction for intermediate outcomes applies also to symptomatic disease.
3. Relative risk reductions are greater than absolute risk reductions. The 25% relative reduction in microvascular complications reported by the UKPDS represents an absolute reduction of only 2%: 8%, rather than 10%, of patients had complications during 10 years of treatment.¹⁰ Also, relative risk reductions generally refer to intermediate outcomes. The 76% reduction in the risk of retinopathy reported by the DCCT refers to a 3-step change on the ETDRS scale, not to improved vision.⁹ The number needed to treat to affect outcomes perceptible to patients is necessarily higher than that for retinopathy, delayed nerve conduction, or elevated urinary albumin excretion, because only a subset of patients with these intermediate outcomes go on to develop symptomatic disease.¹⁵ According to the UKPDS, 37 patients would require intensive treatment for 10 years to prevent one patient from undergoing laser treatment; 208 would require treatment to prevent one case of blindness.¹⁰ The observed 16% difference in the incidence of blindness in the UKPDS was not statistically significant.On the other hand, when examined at the population level, even modest absolute risk reductions can translate into large numbers of persons in society for whom clinical benefit is achievable. Given the millions of people in the United States with type 2 diabetes, even a 2% absolute reduction in the risk of microvascular complications represents many thousands of people who would benefit from glycemic control.
4. Because of the average time required for glycemic control to affect outcomes, some patients with diabetes may not live long enough to benefit, because of the competing risks of death of macrovascular complications and other comorbid diseases. Although elevated blood glucose levels are a likely risk factor for cardiovascular disease, glycemic control has not been shown to enhance life expectancy or prevent heart disease. The 16% reduction in myocardial infarction reported by the UKPDS was of borderline statistical significance.¹⁰ Another RCT did find that improved glycemic control reduced the incidence of ischemic cardiac events, stroke, and cardiovascular deaths in patients with acute myocardial infarction.⁴⁸ Two other trials that failed to show a benefit may have lacked adequate duration and sample size.^16,40
5. For any given patient, the absolute magnitude of risk reduction is a continuous variable that is a function of the patient’s current glycated hemoglobin level, the duration and magnitude of previous hyperglycemia, and the extent of preexisting microvascular complications. The probability that the patient will live long enough to experience the benefits of reduced complications depends on cardiovascular risk factors other than blood glucose (eg, smoking, hypertension, lipid levels, physical inactivity, obesity, preexisting coronary artery disease) and other determinants of life expectancy (eg, age, coexisting diseases, health status). Of these, the most critical variable is the patient’s current glycated hemoglobin level. Because of their increased risk of complications, individuals with marked elevations generally benefit more (in absolute terms) from the same absolute reduction in glycated hemoglobin levels than do individuals with mild to moderate elevations.⁵³ Although it is obviously important for clinicians to keep patients from progressing from mild (eg, hemoglobin A1c levels of 6% to 8%) to marked hyperglycemia (eg, hemoglobin A1c levels >9.5%), in those patients who have already developed marked hyperglycemia, efforts directed at achieving even moderate control (eg, hemoglobin A1c levels of 8% to 9.5%) will yield greater health benefits than pursuing euglycemia in patients with moderate elevations.

Recommendations for clinical practice

For any patient with type 2 diabetes, the better the glycemic control, the lower the probability of chronic microvascular, neuropathic, and possibly cardiovascular complications. However, because of differences in patients’ life expectancies, comorbidities, and preferences, it is inappropriate to set a uniform target glycated hemoglobin level for all patients. Individuals with long life expectancies and few comorbidities may wish to pursue euglycemia, but less vigorous goals may be appropriate for others, such as patients with multiple comorbid conditions or with limited life expectancies.

Whether the magnitude of benefit of a given treatment goal justifies the potential inconvenience, harms, and costs involves value judgments that must be tailored to the individual patient. Patients’ personal risk profiles and capabilities and the relative importance they assign to the potential outcomes and supporting evidence are integral in determining how intensively to treat.

Cardiovascular disease is the most likely cause of death in patients with type 2 diabetes, and attention to glycemic control should not distract clinicians and patients from other interventions that may be more effective in preventing coronary artery disease and stroke. These include smoking cessation, serum lipid management, control of blood pressure, diet, physical activity, and weight management. Guidelines for the control of these risk factors appear elsewhere.^54-56 Clinicians should also pursue treatments other than glycemic control for preventing microvascular complications (eg, blood pressure control, angiotensin-converting enzyme inhibitors for diabetic nephropathy, laser treatment for diabetic retinopathy).

Whatever the desired goals and intensity of treatment, patients face considerable barriers in implementing recommendations. Modifying diet and other personal habits; complying with self-monitoring, medication, and home care; and returning for follow-up visits are difficult. Physicians should work with patients to overcome remediable barriers and should use recommended techniques for patient education and counseling to offer the necessary information and motivation for meaningful change.⁵⁷

Acknowledgments

The systematic review on which this guideline is based was supported in part by funding from the Health Care Financing Administration. We thank Richard D. Kahn, PhD, (American Diabetes Association), and Herbert F. Young, MD, and Bellinda Schoof (American Academy of Family Physicians) for their assistance, as well as the expert panel that externally reviewed the full report: Eugene Barrett, MD; John A. Colwell, MD; Richard C. Eastman, MD; Saul Genuth, MD; Ronald Klein, MD, MPH; Martin Mahoney, MD; James W. Mold, MD; David M. Nathan, MD; Jonathan E. Rodnick, MD; Jeffrey L. Susman, MD; Sandeep Vijan, MD, MS; and Bruce Zimmerman, MD. Their participation in the review process does not necessarily imply endorsement of the report or its recommendations.

Growing evidence that certain interventions can significantly lower morbidity and mortality (eg, mammography, immunizations, b-blockers for acute myocardial infarction) has focused attention on the challenges of implementation: translating this evidence into practice. Clinicians do not always offer recommended services to patients, and patients do not always readily accept them. Approximately 50% of smokers report that their physician has never advised them to quit.¹ One out of 4 patients with an acute myocardial infarction is discharged without a prescription for a b-blocker.² Only 40% of patients with atrial fibrillation receive warfarin.³ Conversely, tests that are known to be ineffective, such as routine chest radiographs, urinalyses, and preoperative blood work, are ordered routinely.⁴

Tools for behavior change

Various programs have been developed to bridge the gap between what should be practiced and what is actually done, but few have been uniformly successful. Passive education, such as conferences or the publication of clinical practice guidelines, has been shown consistently to be ineffective.⁵ More active strategies to implement guidelines, such as educational outreach, feedback, reminder systems, and continuous quality improvement, offer greater promise and have captured the interest of physicians, health systems, hospitals, managed care plans, and quality improvement organizations.⁶ To date, however, research on whether these methods produce meaningful change in practice patterns or patient outcomes has yielded mixed results.

One such study appears in this issue of the Journal. McBride and colleagues⁷ compared 4 strategies for improving preventive cardiology services at 45 Midwestern primary care practices. The control practices attended an educational conference and received a kit of materials. The other 3 groups attended a similar conference but also received a practice consultation, an on-site prevention coordinator, or both. The study used surrogate measures—provider behaviors rather than health outcomes such as lipid levels and blood pressure—to gauge effectiveness, and the results were positive. Patient history questionnaires, problem lists, and flow sheets were used more often by the combined intervention group than by the conference-only control group. Other behaviors, such as documentation of risk factor screening and management in the medical record, improved across all intervention groups. The authors apparently did not examine whether patients in the intervention groups had improved outcomes, such as better control of risk factors or a lower incidence of heart disease. With only 10 or 11 practices per group, the statistical power and duration of follow-up to make such comparisons was probably lacking.

The need for discretion in quality improvement

How should we apply these results? When there is evidence that a particular strategy improves outcomes—in this case, practice consultations and on-site coordinators—should physicians immediately adopt that approach in their own practices? Although McBride and colleagues found full participation in the project, it is doubtful that practices nationwide would have the necessary resources. The practice consultation included 3 meetings and 2 follow-up visits, and the on-site coordinator devoted 4.5 hours per week per physician. Moreover, this is only one of many studies showcasing a promising success in quality improvement. No clinician could adopt the full range of strategies that have been advocated by researchers and health systems. Even if that were possible for one disease, it would be impossible for all of the conditions encountered in primary care for which quality improvement is needed, such as preventive care, heart disease, diabetes, asthma, and depression.

Practices face trade-offs when considering quality improvement.⁸ Although there are exceptions— improved systems of care for one disease can have spillover benefits for other conditions—quality initiatives in one area tend to draw time, resources, and motivation away from others. Before reconfiguring practice operations, the astute clinician must judge not only whether available resources can support the effort, but whether the strategy offers the best use of resources. In the case of the study by McBride and colleagues, physicians might ask whether the proven benefit—improved chart records—justifies a change when data on patient outcomes are lacking. Even if improved health outcomes are likely, they should judge whether applying the same effort to another aspect of care, perhaps for another disease, would help patients even more.

In weighing these choices, physicians should not rely on the results of a single study. It is best to step back and examine the evidence as a whole, reviewing the results of multiple studies of the same strategy. For example, a Cochrane group analyzed 18 systematic reviews of various methods for disseminating and implementing evidence in practice. Although some interventions were consistently effective (eg, educational outreach, reminders, multifaceted interventions, interactive education), others were rarely or never effective (eg, educational materials, didactic teaching) or inconsistently effective (eg, audit, feedback, local opinion leaders, local adaptation, patient-mediated interventions).⁹ Similarly, a recent review of 58 studies of strategies for improving preventive care found that most interventions were effective in some studies but not others.¹⁰

An analytic framework for behavior change

It makes sense that a particular strategy would not succeed in all cases. The reason that clinicians do not adopt new behaviors, or abandon old ones, is often specific to the disease or procedure in question, local practice conditions, and the personal barriers that each physician faces.¹¹ A common solution would not be expected to work. Most physicians undergo stages of change in adopting new behaviors:

1. They must have knowledge (information). They must know about the new data or new practice guidelines that advocate a change in practice behavior. Keeping abreast of this knowledge, with its exponential expansion, is medicine’s great challenge. However, as so many studies have shown, information by itself is not enough.¹²
2. Knowledge must foster a change in attitudes. Clinicians must accept the validity of the evidence and its applicability to their practices and their patients. There must be “buy in” for new practice guidelines and acceptance that the recommendations represent good medical care; have been embraced by peers, local consultants, opinion leaders, or one’s specialty; and are acceptable to patients.
3. Even if physicians know about and accept the behavior, they must have the ability to implement it. Enthusiasm by itself is insufficient if there is a lack of time, resources, staff, training, or equipment. Physicians must have access to eligible patients, and those patients must be able and willing to do their part. (Like clinicians, patients may not comply because of barriers to knowledge, attitudes, ability, and reinforcement.) Finally, constraints imposed by office or clinic operations, practice leadership, information systems, regulations, and insurance coverage can impede change.
4. Like all people, physicians need reinforcement to maintain behaviors. It is human nature to forget, overlook, or lose interest over time. That 36% of physicians do not notify patients of abnormal test results is not because they doubt the importance of that type of communication.¹³ The most committed physician needs reminder systems to remember when to implement guidelines, tracking systems to identify patients who need follow-up, and encouragement from practice leaders, systems of care, and patients that their efforts are appreciated.

Putting the framework to use

This 4-part framework helps to organize the menu of implementation tools that are available to physicians. Some tools focus on providing knowledge, such as conferences, journal articles, practice guidelines, the Cochrane database,¹⁴ and information mastery programs to help clinicians access useful data.¹⁵ Some focus on attitudes, such as local adaptation of guidelines,¹⁶ academic detailing,¹⁷ endorsements by opinion leaders and specialty societies,¹⁸ and feedback from colleagues and patients.¹⁹ Some address ability, such as scheduling and staff changes,²⁰ revised delivery systems,²¹ skill building, teamwork,²² information technology,²³ comprehensive disease²⁴ or total quality²⁵ management, and community support. Some provide reinforcement, such as computerized or manual reminder systems, flow sheets, standing orders, provider incentives, and feedback reports.²⁶

Knowing that, in general, the 4 steps occur in sequence helps clarify why so many methods of changing behavior appear successful in some settings but not in others. An intervention that delivers information is not helpful if clinicians already know the facts but lack ability. If a family practice does poorly in administering polio vaccine, the problem is less likely to be solved by circulating a photocopy of the Advisory Committee on Immunization Practices’s immunization schedule—the physicians already know the guidelines and the importance of vaccination—than by implementing a tracking and reminder system to flag eligible patients, the most effective way to boost immunization rates.²⁷ Conversely, reinforcement tools, such as reminder systems and standing orders, are unlikely to succeed if clinicians are at an earlier stage of change (eg, they are unaware of or question the data). Adding a space for exercise counseling on flow sheets or preventive care forms accomplishes little if physicians are fundamentally resistant because they doubt this type of counseling helps patients. A knowledge intervention is precisely what is needed when physicians withhold warfarin for atrial fibrillation because of the mistaken belief that bleeding complications outweigh benefits.²⁸ Citing an earlier description of this model,²⁹ Cabana and colleagues³⁰ expanded its structure to better organize published evidence on barriers to physician adherence to practice guidelines. In their model, barriers to knowledge include lack of awareness and familiarity with guidelines. Barriers to attitudes include lack of agreement with guidelines, lack of outcome expectancy, lack of self-efficacy, and lack of motivation. Barriers to behavior include factors related to patients (eg, patient expectations), the practice environment (eg, lack of time, resources), and the guidelines themselves (eg, conflicting recommendations). Of the 120 studies on barriers covered in their review, 58% examined only one type of barrier.

A diagnostic approach

Too many advocates of quality improvement champion their method as the only way to improve care. Hospitals, practices, and health systems often seize on a particular approach for improving quality, perhaps because it is easier to organize programs around a single theme. But there are no “magic bullets.”³¹ Seasoned clinicians know this; they understand that proper treatment begins with a good diagnosis. The first step is to “find the lesion,” to determine precisely why the guideline is not followed. Knowing whether the barriers involve knowledge, attitude, ability, or reinforcement is the starting point for designing a targeted solution. The alternative is quality improvement by reflex. No physician can improve everything at once, and this is especially true for primary care physicians because of the spectrum of diseases for which they care. Their special need to set priorities makes a rational diagnostic approach to quality improvement essential.

Seeking outcomes that matter

The study by McBride and colleagues reminds us that the utility of outcomes research often depends on the outcome measures. An effect on surrogate or intermediate end points, such as better use of medical records, does not prove beneficial for patients unless data suggest that a change in such measures improves health.³² Smoking illustrates the ideal surrogate measure because of the strong evidence linking it to disease. Too many researchers rely on less-validated surrogate measures, either because more distal health outcomes are hard to quantify or because statistical power concerns demand too large or lengthy a study. Using surrogates is easier, but it has yielded a profusion of outcome studies that fail to tell us whether patients benefit in ways that matter. It would be better to do fewer studies and conserve the resources for a definitive investigation that gives patient-centered outcomes the attention they deserve.

Growing evidence that certain interventions can significantly lower morbidity and mortality (eg, mammography, immunizations, b-blockers for acute myocardial infarction) has focused attention on the challenges of implementation: translating this evidence into practice. Clinicians do not always offer recommended services to patients, and patients do not always readily accept them. Approximately 50% of smokers report that their physician has never advised them to quit.¹ One out of 4 patients with an acute myocardial infarction is discharged without a prescription for a b-blocker.² Only 40% of patients with atrial fibrillation receive warfarin.³ Conversely, tests that are known to be ineffective, such as routine chest radiographs, urinalyses, and preoperative blood work, are ordered routinely.⁴

Tools for behavior change

Various programs have been developed to bridge the gap between what should be practiced and what is actually done, but few have been uniformly successful. Passive education, such as conferences or the publication of clinical practice guidelines, has been shown consistently to be ineffective.⁵ More active strategies to implement guidelines, such as educational outreach, feedback, reminder systems, and continuous quality improvement, offer greater promise and have captured the interest of physicians, health systems, hospitals, managed care plans, and quality improvement organizations.⁶ To date, however, research on whether these methods produce meaningful change in practice patterns or patient outcomes has yielded mixed results.

One such study appears in this issue of the Journal. McBride and colleagues⁷ compared 4 strategies for improving preventive cardiology services at 45 Midwestern primary care practices. The control practices attended an educational conference and received a kit of materials. The other 3 groups attended a similar conference but also received a practice consultation, an on-site prevention coordinator, or both. The study used surrogate measures—provider behaviors rather than health outcomes such as lipid levels and blood pressure—to gauge effectiveness, and the results were positive. Patient history questionnaires, problem lists, and flow sheets were used more often by the combined intervention group than by the conference-only control group. Other behaviors, such as documentation of risk factor screening and management in the medical record, improved across all intervention groups. The authors apparently did not examine whether patients in the intervention groups had improved outcomes, such as better control of risk factors or a lower incidence of heart disease. With only 10 or 11 practices per group, the statistical power and duration of follow-up to make such comparisons was probably lacking.

The need for discretion in quality improvement

How should we apply these results? When there is evidence that a particular strategy improves outcomes—in this case, practice consultations and on-site coordinators—should physicians immediately adopt that approach in their own practices? Although McBride and colleagues found full participation in the project, it is doubtful that practices nationwide would have the necessary resources. The practice consultation included 3 meetings and 2 follow-up visits, and the on-site coordinator devoted 4.5 hours per week per physician. Moreover, this is only one of many studies showcasing a promising success in quality improvement. No clinician could adopt the full range of strategies that have been advocated by researchers and health systems. Even if that were possible for one disease, it would be impossible for all of the conditions encountered in primary care for which quality improvement is needed, such as preventive care, heart disease, diabetes, asthma, and depression.

Practices face trade-offs when considering quality improvement.⁸ Although there are exceptions— improved systems of care for one disease can have spillover benefits for other conditions—quality initiatives in one area tend to draw time, resources, and motivation away from others. Before reconfiguring practice operations, the astute clinician must judge not only whether available resources can support the effort, but whether the strategy offers the best use of resources. In the case of the study by McBride and colleagues, physicians might ask whether the proven benefit—improved chart records—justifies a change when data on patient outcomes are lacking. Even if improved health outcomes are likely, they should judge whether applying the same effort to another aspect of care, perhaps for another disease, would help patients even more.

In weighing these choices, physicians should not rely on the results of a single study. It is best to step back and examine the evidence as a whole, reviewing the results of multiple studies of the same strategy. For example, a Cochrane group analyzed 18 systematic reviews of various methods for disseminating and implementing evidence in practice. Although some interventions were consistently effective (eg, educational outreach, reminders, multifaceted interventions, interactive education), others were rarely or never effective (eg, educational materials, didactic teaching) or inconsistently effective (eg, audit, feedback, local opinion leaders, local adaptation, patient-mediated interventions).⁹ Similarly, a recent review of 58 studies of strategies for improving preventive care found that most interventions were effective in some studies but not others.¹⁰

An analytic framework for behavior change

It makes sense that a particular strategy would not succeed in all cases. The reason that clinicians do not adopt new behaviors, or abandon old ones, is often specific to the disease or procedure in question, local practice conditions, and the personal barriers that each physician faces.¹¹ A common solution would not be expected to work. Most physicians undergo stages of change in adopting new behaviors:

1. They must have knowledge (information). They must know about the new data or new practice guidelines that advocate a change in practice behavior. Keeping abreast of this knowledge, with its exponential expansion, is medicine’s great challenge. However, as so many studies have shown, information by itself is not enough.¹²
2. Knowledge must foster a change in attitudes. Clinicians must accept the validity of the evidence and its applicability to their practices and their patients. There must be “buy in” for new practice guidelines and acceptance that the recommendations represent good medical care; have been embraced by peers, local consultants, opinion leaders, or one’s specialty; and are acceptable to patients.
3. Even if physicians know about and accept the behavior, they must have the ability to implement it. Enthusiasm by itself is insufficient if there is a lack of time, resources, staff, training, or equipment. Physicians must have access to eligible patients, and those patients must be able and willing to do their part. (Like clinicians, patients may not comply because of barriers to knowledge, attitudes, ability, and reinforcement.) Finally, constraints imposed by office or clinic operations, practice leadership, information systems, regulations, and insurance coverage can impede change.
4. Like all people, physicians need reinforcement to maintain behaviors. It is human nature to forget, overlook, or lose interest over time. That 36% of physicians do not notify patients of abnormal test results is not because they doubt the importance of that type of communication.¹³ The most committed physician needs reminder systems to remember when to implement guidelines, tracking systems to identify patients who need follow-up, and encouragement from practice leaders, systems of care, and patients that their efforts are appreciated.

Putting the framework to use

This 4-part framework helps to organize the menu of implementation tools that are available to physicians. Some tools focus on providing knowledge, such as conferences, journal articles, practice guidelines, the Cochrane database,¹⁴ and information mastery programs to help clinicians access useful data.¹⁵ Some focus on attitudes, such as local adaptation of guidelines,¹⁶ academic detailing,¹⁷ endorsements by opinion leaders and specialty societies,¹⁸ and feedback from colleagues and patients.¹⁹ Some address ability, such as scheduling and staff changes,²⁰ revised delivery systems,²¹ skill building, teamwork,²² information technology,²³ comprehensive disease²⁴ or total quality²⁵ management, and community support. Some provide reinforcement, such as computerized or manual reminder systems, flow sheets, standing orders, provider incentives, and feedback reports.²⁶

Knowing that, in general, the 4 steps occur in sequence helps clarify why so many methods of changing behavior appear successful in some settings but not in others. An intervention that delivers information is not helpful if clinicians already know the facts but lack ability. If a family practice does poorly in administering polio vaccine, the problem is less likely to be solved by circulating a photocopy of the Advisory Committee on Immunization Practices’s immunization schedule—the physicians already know the guidelines and the importance of vaccination—than by implementing a tracking and reminder system to flag eligible patients, the most effective way to boost immunization rates.²⁷ Conversely, reinforcement tools, such as reminder systems and standing orders, are unlikely to succeed if clinicians are at an earlier stage of change (eg, they are unaware of or question the data). Adding a space for exercise counseling on flow sheets or preventive care forms accomplishes little if physicians are fundamentally resistant because they doubt this type of counseling helps patients. A knowledge intervention is precisely what is needed when physicians withhold warfarin for atrial fibrillation because of the mistaken belief that bleeding complications outweigh benefits.²⁸ Citing an earlier description of this model,²⁹ Cabana and colleagues³⁰ expanded its structure to better organize published evidence on barriers to physician adherence to practice guidelines. In their model, barriers to knowledge include lack of awareness and familiarity with guidelines. Barriers to attitudes include lack of agreement with guidelines, lack of outcome expectancy, lack of self-efficacy, and lack of motivation. Barriers to behavior include factors related to patients (eg, patient expectations), the practice environment (eg, lack of time, resources), and the guidelines themselves (eg, conflicting recommendations). Of the 120 studies on barriers covered in their review, 58% examined only one type of barrier.

A diagnostic approach

Too many advocates of quality improvement champion their method as the only way to improve care. Hospitals, practices, and health systems often seize on a particular approach for improving quality, perhaps because it is easier to organize programs around a single theme. But there are no “magic bullets.”³¹ Seasoned clinicians know this; they understand that proper treatment begins with a good diagnosis. The first step is to “find the lesion,” to determine precisely why the guideline is not followed. Knowing whether the barriers involve knowledge, attitude, ability, or reinforcement is the starting point for designing a targeted solution. The alternative is quality improvement by reflex. No physician can improve everything at once, and this is especially true for primary care physicians because of the spectrum of diseases for which they care. Their special need to set priorities makes a rational diagnostic approach to quality improvement essential.

Seeking outcomes that matter

The study by McBride and colleagues reminds us that the utility of outcomes research often depends on the outcome measures. An effect on surrogate or intermediate end points, such as better use of medical records, does not prove beneficial for patients unless data suggest that a change in such measures improves health.³² Smoking illustrates the ideal surrogate measure because of the strong evidence linking it to disease. Too many researchers rely on less-validated surrogate measures, either because more distal health outcomes are hard to quantify or because statistical power concerns demand too large or lengthy a study. Using surrogates is easier, but it has yielded a profusion of outcome studies that fail to tell us whether patients benefit in ways that matter. It would be better to do fewer studies and conserve the resources for a definitive investigation that gives patient-centered outcomes the attention they deserve.

Growing evidence that certain interventions can significantly lower morbidity and mortality (eg, mammography, immunizations, b-blockers for acute myocardial infarction) has focused attention on the challenges of implementation: translating this evidence into practice. Clinicians do not always offer recommended services to patients, and patients do not always readily accept them. Approximately 50% of smokers report that their physician has never advised them to quit.¹ One out of 4 patients with an acute myocardial infarction is discharged without a prescription for a b-blocker.² Only 40% of patients with atrial fibrillation receive warfarin.³ Conversely, tests that are known to be ineffective, such as routine chest radiographs, urinalyses, and preoperative blood work, are ordered routinely.⁴

Tools for behavior change

Various programs have been developed to bridge the gap between what should be practiced and what is actually done, but few have been uniformly successful. Passive education, such as conferences or the publication of clinical practice guidelines, has been shown consistently to be ineffective.⁵ More active strategies to implement guidelines, such as educational outreach, feedback, reminder systems, and continuous quality improvement, offer greater promise and have captured the interest of physicians, health systems, hospitals, managed care plans, and quality improvement organizations.⁶ To date, however, research on whether these methods produce meaningful change in practice patterns or patient outcomes has yielded mixed results.

One such study appears in this issue of the Journal. McBride and colleagues⁷ compared 4 strategies for improving preventive cardiology services at 45 Midwestern primary care practices. The control practices attended an educational conference and received a kit of materials. The other 3 groups attended a similar conference but also received a practice consultation, an on-site prevention coordinator, or both. The study used surrogate measures—provider behaviors rather than health outcomes such as lipid levels and blood pressure—to gauge effectiveness, and the results were positive. Patient history questionnaires, problem lists, and flow sheets were used more often by the combined intervention group than by the conference-only control group. Other behaviors, such as documentation of risk factor screening and management in the medical record, improved across all intervention groups. The authors apparently did not examine whether patients in the intervention groups had improved outcomes, such as better control of risk factors or a lower incidence of heart disease. With only 10 or 11 practices per group, the statistical power and duration of follow-up to make such comparisons was probably lacking.

The need for discretion in quality improvement

How should we apply these results? When there is evidence that a particular strategy improves outcomes—in this case, practice consultations and on-site coordinators—should physicians immediately adopt that approach in their own practices? Although McBride and colleagues found full participation in the project, it is doubtful that practices nationwide would have the necessary resources. The practice consultation included 3 meetings and 2 follow-up visits, and the on-site coordinator devoted 4.5 hours per week per physician. Moreover, this is only one of many studies showcasing a promising success in quality improvement. No clinician could adopt the full range of strategies that have been advocated by researchers and health systems. Even if that were possible for one disease, it would be impossible for all of the conditions encountered in primary care for which quality improvement is needed, such as preventive care, heart disease, diabetes, asthma, and depression.

Practices face trade-offs when considering quality improvement.⁸ Although there are exceptions— improved systems of care for one disease can have spillover benefits for other conditions—quality initiatives in one area tend to draw time, resources, and motivation away from others. Before reconfiguring practice operations, the astute clinician must judge not only whether available resources can support the effort, but whether the strategy offers the best use of resources. In the case of the study by McBride and colleagues, physicians might ask whether the proven benefit—improved chart records—justifies a change when data on patient outcomes are lacking. Even if improved health outcomes are likely, they should judge whether applying the same effort to another aspect of care, perhaps for another disease, would help patients even more.

In weighing these choices, physicians should not rely on the results of a single study. It is best to step back and examine the evidence as a whole, reviewing the results of multiple studies of the same strategy. For example, a Cochrane group analyzed 18 systematic reviews of various methods for disseminating and implementing evidence in practice. Although some interventions were consistently effective (eg, educational outreach, reminders, multifaceted interventions, interactive education), others were rarely or never effective (eg, educational materials, didactic teaching) or inconsistently effective (eg, audit, feedback, local opinion leaders, local adaptation, patient-mediated interventions).⁹ Similarly, a recent review of 58 studies of strategies for improving preventive care found that most interventions were effective in some studies but not others.¹⁰

An analytic framework for behavior change

It makes sense that a particular strategy would not succeed in all cases. The reason that clinicians do not adopt new behaviors, or abandon old ones, is often specific to the disease or procedure in question, local practice conditions, and the personal barriers that each physician faces.¹¹ A common solution would not be expected to work. Most physicians undergo stages of change in adopting new behaviors:

1. They must have knowledge (information). They must know about the new data or new practice guidelines that advocate a change in practice behavior. Keeping abreast of this knowledge, with its exponential expansion, is medicine’s great challenge. However, as so many studies have shown, information by itself is not enough.¹²
2. Knowledge must foster a change in attitudes. Clinicians must accept the validity of the evidence and its applicability to their practices and their patients. There must be “buy in” for new practice guidelines and acceptance that the recommendations represent good medical care; have been embraced by peers, local consultants, opinion leaders, or one’s specialty; and are acceptable to patients.
3. Even if physicians know about and accept the behavior, they must have the ability to implement it. Enthusiasm by itself is insufficient if there is a lack of time, resources, staff, training, or equipment. Physicians must have access to eligible patients, and those patients must be able and willing to do their part. (Like clinicians, patients may not comply because of barriers to knowledge, attitudes, ability, and reinforcement.) Finally, constraints imposed by office or clinic operations, practice leadership, information systems, regulations, and insurance coverage can impede change.
4. Like all people, physicians need reinforcement to maintain behaviors. It is human nature to forget, overlook, or lose interest over time. That 36% of physicians do not notify patients of abnormal test results is not because they doubt the importance of that type of communication.¹³ The most committed physician needs reminder systems to remember when to implement guidelines, tracking systems to identify patients who need follow-up, and encouragement from practice leaders, systems of care, and patients that their efforts are appreciated.

Putting the framework to use

This 4-part framework helps to organize the menu of implementation tools that are available to physicians. Some tools focus on providing knowledge, such as conferences, journal articles, practice guidelines, the Cochrane database,¹⁴ and information mastery programs to help clinicians access useful data.¹⁵ Some focus on attitudes, such as local adaptation of guidelines,¹⁶ academic detailing,¹⁷ endorsements by opinion leaders and specialty societies,¹⁸ and feedback from colleagues and patients.¹⁹ Some address ability, such as scheduling and staff changes,²⁰ revised delivery systems,²¹ skill building, teamwork,²² information technology,²³ comprehensive disease²⁴ or total quality²⁵ management, and community support. Some provide reinforcement, such as computerized or manual reminder systems, flow sheets, standing orders, provider incentives, and feedback reports.²⁶

Knowing that, in general, the 4 steps occur in sequence helps clarify why so many methods of changing behavior appear successful in some settings but not in others. An intervention that delivers information is not helpful if clinicians already know the facts but lack ability. If a family practice does poorly in administering polio vaccine, the problem is less likely to be solved by circulating a photocopy of the Advisory Committee on Immunization Practices’s immunization schedule—the physicians already know the guidelines and the importance of vaccination—than by implementing a tracking and reminder system to flag eligible patients, the most effective way to boost immunization rates.²⁷ Conversely, reinforcement tools, such as reminder systems and standing orders, are unlikely to succeed if clinicians are at an earlier stage of change (eg, they are unaware of or question the data). Adding a space for exercise counseling on flow sheets or preventive care forms accomplishes little if physicians are fundamentally resistant because they doubt this type of counseling helps patients. A knowledge intervention is precisely what is needed when physicians withhold warfarin for atrial fibrillation because of the mistaken belief that bleeding complications outweigh benefits.²⁸ Citing an earlier description of this model,²⁹ Cabana and colleagues³⁰ expanded its structure to better organize published evidence on barriers to physician adherence to practice guidelines. In their model, barriers to knowledge include lack of awareness and familiarity with guidelines. Barriers to attitudes include lack of agreement with guidelines, lack of outcome expectancy, lack of self-efficacy, and lack of motivation. Barriers to behavior include factors related to patients (eg, patient expectations), the practice environment (eg, lack of time, resources), and the guidelines themselves (eg, conflicting recommendations). Of the 120 studies on barriers covered in their review, 58% examined only one type of barrier.

A diagnostic approach

Too many advocates of quality improvement champion their method as the only way to improve care. Hospitals, practices, and health systems often seize on a particular approach for improving quality, perhaps because it is easier to organize programs around a single theme. But there are no “magic bullets.”³¹ Seasoned clinicians know this; they understand that proper treatment begins with a good diagnosis. The first step is to “find the lesion,” to determine precisely why the guideline is not followed. Knowing whether the barriers involve knowledge, attitude, ability, or reinforcement is the starting point for designing a targeted solution. The alternative is quality improvement by reflex. No physician can improve everything at once, and this is especially true for primary care physicians because of the spectrum of diseases for which they care. Their special need to set priorities makes a rational diagnostic approach to quality improvement essential.

Seeking outcomes that matter

The study by McBride and colleagues reminds us that the utility of outcomes research often depends on the outcome measures. An effect on surrogate or intermediate end points, such as better use of medical records, does not prove beneficial for patients unless data suggest that a change in such measures improves health.³² Smoking illustrates the ideal surrogate measure because of the strong evidence linking it to disease. Too many researchers rely on less-validated surrogate measures, either because more distal health outcomes are hard to quantify or because statistical power concerns demand too large or lengthy a study. Using surrogates is easier, but it has yielded a profusion of outcome studies that fail to tell us whether patients benefit in ways that matter. It would be better to do fewer studies and conserve the resources for a definitive investigation that gives patient-centered outcomes the attention they deserve.