An Evidence-Based Approach to Medical Care Raises Uncomfortable Questions

The analysis of the screening mammography data in The Lancet¹ and our subsequent Patient-Oriented Evidence that Matters (POEMs) review² have accomplished an important goal: They got people talking about the benefits versus the risks of this common medical intervention. The practice of evidence-based medicine always begins with a question. The editorial by Dr Woolf³ shows that a POEM review can have the effect of generating questions in the minds of readers.

The analysis of the data supporting screening mammography by Gøtzsche and Olsen¹ focused on 2 limitations of the available research. The first is the appearance of data loss in the trials indicated by differing reports of the number of patients in several of the trials. The databases in these studies are very large (more than 500,000 women have been studied) making them unwieldy and subject to mistakes.

The second limitation found in these trials is the one that causes the most concern: the inadequate concealment of allocation assignment. Researchers enrolling a patient into the trial knew or could have known whether a woman would be in the screened or unscreened group and could have chosen not to enroll a particular patient or type of patient. According to Gøtzsche and Olsen, this knowledge resulted in an imbalance within the 2 groups.

The importance of study design

Along with randomization and blinding, concealed allocation is a way to keep researchers from succumbing to those human traits that get in the way of the scientific method. The Cochrane Collaboration⁴ considers concealed allocation more important than the method of randomization for preventing the introduction of bias into a study, and others⁵ consider it essential for the prevention of selection bias.⁴ This lack of concealed allocation is a serious flaw—not a random error for which a meta-analysis of multiple studies can compensate—that can be responsible for the same bias occurring over and over in different studies, as was demonstrated in this analysis. Studies without concealed allocation may markedly overestimate a benefit of treatment.^5-8

The issues of study design seem to be technical nitpicking until one realizes that the difference in deaths of women screened for breast cancer compared with those who were not screened was only 65 out of a population of almost 0.5 million women. Although the screening may have benefited many women, this benefit has to be weighed against the lack of benefit or actual harm that might have been experienced by others. Ewart⁹ has discussed the ethics of screening programs such as this one and concluded that screening tests must be shown to be beneficial beyond a reasonable doubt before widespread use. The article by Gøtzsche and Olsen and the POEM review suggest that this criterion is not met with our current research.

The POEMs process

The primary goal of the POEMs section in JFP is to present relevant and accurate new research findings in a way that makes them accessible to the busy practicing clinician.* The studies are rigorously evaluated in a redundant process by the primary writer of the review, an editor, members of our editorial board at the University of Missouri, and a supervising editor. In recognition of the fact that the typical clinician has fewer than 30 minutes per week to devote to foraging for new information, each review is limited to 700 words or less. In this short space we try to achieve a balance of information that is clear and concise yet points out the limitations of the research. Reviews that painstakingly outline the complete critical appraisal for each article would be complete but essentially unreadable for all but the most fanatical reader.

The best available evidence

As stated by Dr Woolf, clinical practice guidelines that are based on and linked to clinical evidence are excellent tools for directing practice decisions. However, guidelines often take years to produce, may be influenced by politics more than an explicit evaluation of the best evidence,^10,11 and are not available for many clinical problems. Physicians must act now, using the best available evidence, and they must know today how to integrate a new research finding into their clinical practices. Some articles should change practice. We focus on articles that use patient-oriented outcomes, because we feel that these are much less susceptible to the ping-pong seen when the focus is on disease-oriented outcomes.

As our technological armamentarium increases in size and invasiveness, it is important that we remember the dictum primum non nocere (first do no harm). These days we can do a lot of harm if we are not careful. Absolute certainty is absolutely impossible, and we do not have to wait for that, of course. The results of the reanalysis of the mammography research data suggest no benefit. We evaluated this analysis and reported the results in the POEMs review. It caused several people to ask questions.,¹² That is a good thing.

The analysis of the screening mammography data in The Lancet¹ and our subsequent Patient-Oriented Evidence that Matters (POEMs) review² have accomplished an important goal: They got people talking about the benefits versus the risks of this common medical intervention. The practice of evidence-based medicine always begins with a question. The editorial by Dr Woolf³ shows that a POEM review can have the effect of generating questions in the minds of readers.

The analysis of the data supporting screening mammography by Gøtzsche and Olsen¹ focused on 2 limitations of the available research. The first is the appearance of data loss in the trials indicated by differing reports of the number of patients in several of the trials. The databases in these studies are very large (more than 500,000 women have been studied) making them unwieldy and subject to mistakes.

The second limitation found in these trials is the one that causes the most concern: the inadequate concealment of allocation assignment. Researchers enrolling a patient into the trial knew or could have known whether a woman would be in the screened or unscreened group and could have chosen not to enroll a particular patient or type of patient. According to Gøtzsche and Olsen, this knowledge resulted in an imbalance within the 2 groups.

The importance of study design

Along with randomization and blinding, concealed allocation is a way to keep researchers from succumbing to those human traits that get in the way of the scientific method. The Cochrane Collaboration⁴ considers concealed allocation more important than the method of randomization for preventing the introduction of bias into a study, and others⁵ consider it essential for the prevention of selection bias.⁴ This lack of concealed allocation is a serious flaw—not a random error for which a meta-analysis of multiple studies can compensate—that can be responsible for the same bias occurring over and over in different studies, as was demonstrated in this analysis. Studies without concealed allocation may markedly overestimate a benefit of treatment.^5-8

The issues of study design seem to be technical nitpicking until one realizes that the difference in deaths of women screened for breast cancer compared with those who were not screened was only 65 out of a population of almost 0.5 million women. Although the screening may have benefited many women, this benefit has to be weighed against the lack of benefit or actual harm that might have been experienced by others. Ewart⁹ has discussed the ethics of screening programs such as this one and concluded that screening tests must be shown to be beneficial beyond a reasonable doubt before widespread use. The article by Gøtzsche and Olsen and the POEM review suggest that this criterion is not met with our current research.

The POEMs process

The primary goal of the POEMs section in JFP is to present relevant and accurate new research findings in a way that makes them accessible to the busy practicing clinician.* The studies are rigorously evaluated in a redundant process by the primary writer of the review, an editor, members of our editorial board at the University of Missouri, and a supervising editor. In recognition of the fact that the typical clinician has fewer than 30 minutes per week to devote to foraging for new information, each review is limited to 700 words or less. In this short space we try to achieve a balance of information that is clear and concise yet points out the limitations of the research. Reviews that painstakingly outline the complete critical appraisal for each article would be complete but essentially unreadable for all but the most fanatical reader.

The best available evidence

As stated by Dr Woolf, clinical practice guidelines that are based on and linked to clinical evidence are excellent tools for directing practice decisions. However, guidelines often take years to produce, may be influenced by politics more than an explicit evaluation of the best evidence,^10,11 and are not available for many clinical problems. Physicians must act now, using the best available evidence, and they must know today how to integrate a new research finding into their clinical practices. Some articles should change practice. We focus on articles that use patient-oriented outcomes, because we feel that these are much less susceptible to the ping-pong seen when the focus is on disease-oriented outcomes.

As our technological armamentarium increases in size and invasiveness, it is important that we remember the dictum primum non nocere (first do no harm). These days we can do a lot of harm if we are not careful. Absolute certainty is absolutely impossible, and we do not have to wait for that, of course. The results of the reanalysis of the mammography research data suggest no benefit. We evaluated this analysis and reported the results in the POEMs review. It caused several people to ask questions.,¹² That is a good thing.

The analysis of the screening mammography data in The Lancet¹ and our subsequent Patient-Oriented Evidence that Matters (POEMs) review² have accomplished an important goal: They got people talking about the benefits versus the risks of this common medical intervention. The practice of evidence-based medicine always begins with a question. The editorial by Dr Woolf³ shows that a POEM review can have the effect of generating questions in the minds of readers.

The analysis of the data supporting screening mammography by Gøtzsche and Olsen¹ focused on 2 limitations of the available research. The first is the appearance of data loss in the trials indicated by differing reports of the number of patients in several of the trials. The databases in these studies are very large (more than 500,000 women have been studied) making them unwieldy and subject to mistakes.

The second limitation found in these trials is the one that causes the most concern: the inadequate concealment of allocation assignment. Researchers enrolling a patient into the trial knew or could have known whether a woman would be in the screened or unscreened group and could have chosen not to enroll a particular patient or type of patient. According to Gøtzsche and Olsen, this knowledge resulted in an imbalance within the 2 groups.

The importance of study design

Along with randomization and blinding, concealed allocation is a way to keep researchers from succumbing to those human traits that get in the way of the scientific method. The Cochrane Collaboration⁴ considers concealed allocation more important than the method of randomization for preventing the introduction of bias into a study, and others⁵ consider it essential for the prevention of selection bias.⁴ This lack of concealed allocation is a serious flaw—not a random error for which a meta-analysis of multiple studies can compensate—that can be responsible for the same bias occurring over and over in different studies, as was demonstrated in this analysis. Studies without concealed allocation may markedly overestimate a benefit of treatment.^5-8

The issues of study design seem to be technical nitpicking until one realizes that the difference in deaths of women screened for breast cancer compared with those who were not screened was only 65 out of a population of almost 0.5 million women. Although the screening may have benefited many women, this benefit has to be weighed against the lack of benefit or actual harm that might have been experienced by others. Ewart⁹ has discussed the ethics of screening programs such as this one and concluded that screening tests must be shown to be beneficial beyond a reasonable doubt before widespread use. The article by Gøtzsche and Olsen and the POEM review suggest that this criterion is not met with our current research.

The POEMs process

The primary goal of the POEMs section in JFP is to present relevant and accurate new research findings in a way that makes them accessible to the busy practicing clinician.* The studies are rigorously evaluated in a redundant process by the primary writer of the review, an editor, members of our editorial board at the University of Missouri, and a supervising editor. In recognition of the fact that the typical clinician has fewer than 30 minutes per week to devote to foraging for new information, each review is limited to 700 words or less. In this short space we try to achieve a balance of information that is clear and concise yet points out the limitations of the research. Reviews that painstakingly outline the complete critical appraisal for each article would be complete but essentially unreadable for all but the most fanatical reader.

The best available evidence

As stated by Dr Woolf, clinical practice guidelines that are based on and linked to clinical evidence are excellent tools for directing practice decisions. However, guidelines often take years to produce, may be influenced by politics more than an explicit evaluation of the best evidence,^10,11 and are not available for many clinical problems. Physicians must act now, using the best available evidence, and they must know today how to integrate a new research finding into their clinical practices. Some articles should change practice. We focus on articles that use patient-oriented outcomes, because we feel that these are much less susceptible to the ping-pong seen when the focus is on disease-oriented outcomes.

As our technological armamentarium increases in size and invasiveness, it is important that we remember the dictum primum non nocere (first do no harm). These days we can do a lot of harm if we are not careful. Absolute certainty is absolutely impossible, and we do not have to wait for that, of course. The results of the reanalysis of the mammography research data suggest no benefit. We evaluated this analysis and reported the results in the POEMs review. It caused several people to ask questions.,¹² That is a good thing.