User login
VIENNA – Janneke A. Bastiaansen, PhD, has some advice for clinicians and researchers as they peruse the published literature on clinical trials of medication or psychotherapy for major depressive disorder: Don’t believe everything you read.
“Be critical. Use your critical mind,” she urged at the annual congress of the European College of Neuropsychopharmacology.
The results of her analysis of 105 clinical trials of antidepressant drugs and 142 studies of psychotherapy indicated that the literature is rife with four types of bias: publication, outcome reporting, spin, and citation bias.
She took a closer look at 105 clinical trials of antidepressant drugs registered with the U.S. National Institutes of Health at clinicaltrials.gov. Fifty-three reported positive findings, and 52 were negative. Fifty-two of the 53 positive trials were ultimately published, as were only 25 of the 52 negative studies. That’s a sterling example of publication bias.
Upon careful scrutiny of the 25 negative trials that were published, 10 were misleadingly reported as positive studies. The investigators either switched out the prespecified primary outcome previously filed with NIH and promoted a positive secondary outcome to primary outcome status because the original primary outcome was negative, or they omitted the negative outcomes altogether. That’s outcome-reporting bias.
Of the 15 published negative drug trials that were free of outcome-reporting bias, the authors of 10 of the studies employed “spin,” using phrases such as “the treatment was numerically superior.”
Thus, only 5 of the 25 published negative clinical trials unambiguously reported that the studied treatment was not effective.
“Here the message is that, when you read a paper, look at the results, come up with your own conclusion, and then compare it with the conclusion of the authors, because sometimes they’ve colored it in a more positive way,” Dr. Bastiaansen said in an interview.
Citation bias is the phenomenon whereby positive clinical trials are cited more frequently than published negative trials.
“As a clinician, if you look at the literature and print out every paper that’s out there on a given antidepressant drug for major depression, and you look at that pile, you’ll think: ‘Ah, the literature is really strong about this treatment effect,’ because positive papers selectively cite other positive papers,” she continued.
The pharmaceutical industry takes a lot of heat for selectively burying company-sponsored negative trials, but the literature on psychotherapy for major depression is actually more opaque.
“A lot of people aim their arrows at the pharmaceutical industry and say: ‘Everything’s bad about pharma,’ but actually, you see bias in every field. You see it in the trials of psychotherapy. It’s very important to know that it’s ubiquitous. The positive side of the antidepressant drug trials is that there’s this standard database [clinicaltrials.gov], and you can use it to check out what trial is published and what’s not. It’s not the case for psychotherapy trials. I think we need a mandatory registry for clinical trials of psychotherapy as well,” Dr. Bastiaansen said.
Of the 142 psychotherapy studies, 49 were negative, but the abstracts of only 12 of those 49 concluded that psychotherapy was not more effective than a control.
Dr. Bastiaansen declared having no financial conflicts regarding her university-funded study.
VIENNA – Janneke A. Bastiaansen, PhD, has some advice for clinicians and researchers as they peruse the published literature on clinical trials of medication or psychotherapy for major depressive disorder: Don’t believe everything you read.
“Be critical. Use your critical mind,” she urged at the annual congress of the European College of Neuropsychopharmacology.
The results of her analysis of 105 clinical trials of antidepressant drugs and 142 studies of psychotherapy indicated that the literature is rife with four types of bias: publication, outcome reporting, spin, and citation bias.
She took a closer look at 105 clinical trials of antidepressant drugs registered with the U.S. National Institutes of Health at clinicaltrials.gov. Fifty-three reported positive findings, and 52 were negative. Fifty-two of the 53 positive trials were ultimately published, as were only 25 of the 52 negative studies. That’s a sterling example of publication bias.
Upon careful scrutiny of the 25 negative trials that were published, 10 were misleadingly reported as positive studies. The investigators either switched out the prespecified primary outcome previously filed with NIH and promoted a positive secondary outcome to primary outcome status because the original primary outcome was negative, or they omitted the negative outcomes altogether. That’s outcome-reporting bias.
Of the 15 published negative drug trials that were free of outcome-reporting bias, the authors of 10 of the studies employed “spin,” using phrases such as “the treatment was numerically superior.”
Thus, only 5 of the 25 published negative clinical trials unambiguously reported that the studied treatment was not effective.
“Here the message is that, when you read a paper, look at the results, come up with your own conclusion, and then compare it with the conclusion of the authors, because sometimes they’ve colored it in a more positive way,” Dr. Bastiaansen said in an interview.
Citation bias is the phenomenon whereby positive clinical trials are cited more frequently than published negative trials.
“As a clinician, if you look at the literature and print out every paper that’s out there on a given antidepressant drug for major depression, and you look at that pile, you’ll think: ‘Ah, the literature is really strong about this treatment effect,’ because positive papers selectively cite other positive papers,” she continued.
The pharmaceutical industry takes a lot of heat for selectively burying company-sponsored negative trials, but the literature on psychotherapy for major depression is actually more opaque.
“A lot of people aim their arrows at the pharmaceutical industry and say: ‘Everything’s bad about pharma,’ but actually, you see bias in every field. You see it in the trials of psychotherapy. It’s very important to know that it’s ubiquitous. The positive side of the antidepressant drug trials is that there’s this standard database [clinicaltrials.gov], and you can use it to check out what trial is published and what’s not. It’s not the case for psychotherapy trials. I think we need a mandatory registry for clinical trials of psychotherapy as well,” Dr. Bastiaansen said.
Of the 142 psychotherapy studies, 49 were negative, but the abstracts of only 12 of those 49 concluded that psychotherapy was not more effective than a control.
Dr. Bastiaansen declared having no financial conflicts regarding her university-funded study.
VIENNA – Janneke A. Bastiaansen, PhD, has some advice for clinicians and researchers as they peruse the published literature on clinical trials of medication or psychotherapy for major depressive disorder: Don’t believe everything you read.
“Be critical. Use your critical mind,” she urged at the annual congress of the European College of Neuropsychopharmacology.
The results of her analysis of 105 clinical trials of antidepressant drugs and 142 studies of psychotherapy indicated that the literature is rife with four types of bias: publication, outcome reporting, spin, and citation bias.
She took a closer look at 105 clinical trials of antidepressant drugs registered with the U.S. National Institutes of Health at clinicaltrials.gov. Fifty-three reported positive findings, and 52 were negative. Fifty-two of the 53 positive trials were ultimately published, as were only 25 of the 52 negative studies. That’s a sterling example of publication bias.
Upon careful scrutiny of the 25 negative trials that were published, 10 were misleadingly reported as positive studies. The investigators either switched out the prespecified primary outcome previously filed with NIH and promoted a positive secondary outcome to primary outcome status because the original primary outcome was negative, or they omitted the negative outcomes altogether. That’s outcome-reporting bias.
Of the 15 published negative drug trials that were free of outcome-reporting bias, the authors of 10 of the studies employed “spin,” using phrases such as “the treatment was numerically superior.”
Thus, only 5 of the 25 published negative clinical trials unambiguously reported that the studied treatment was not effective.
“Here the message is that, when you read a paper, look at the results, come up with your own conclusion, and then compare it with the conclusion of the authors, because sometimes they’ve colored it in a more positive way,” Dr. Bastiaansen said in an interview.
Citation bias is the phenomenon whereby positive clinical trials are cited more frequently than published negative trials.
“As a clinician, if you look at the literature and print out every paper that’s out there on a given antidepressant drug for major depression, and you look at that pile, you’ll think: ‘Ah, the literature is really strong about this treatment effect,’ because positive papers selectively cite other positive papers,” she continued.
The pharmaceutical industry takes a lot of heat for selectively burying company-sponsored negative trials, but the literature on psychotherapy for major depression is actually more opaque.
“A lot of people aim their arrows at the pharmaceutical industry and say: ‘Everything’s bad about pharma,’ but actually, you see bias in every field. You see it in the trials of psychotherapy. It’s very important to know that it’s ubiquitous. The positive side of the antidepressant drug trials is that there’s this standard database [clinicaltrials.gov], and you can use it to check out what trial is published and what’s not. It’s not the case for psychotherapy trials. I think we need a mandatory registry for clinical trials of psychotherapy as well,” Dr. Bastiaansen said.
Of the 142 psychotherapy studies, 49 were negative, but the abstracts of only 12 of those 49 concluded that psychotherapy was not more effective than a control.
Dr. Bastiaansen declared having no financial conflicts regarding her university-funded study.
Key clinical point:
Major finding: Only 5 of 25 published negative clinical trials of antidepressant drug therapy unambiguously reported that the studied treatment was not effective.
Data source: For this study, 105 clinical trials of antidepressant drugs and 142 on psychotherapy for major depressive disorder were examined systematically for evidence of four types of bias that damage the quality of the evidence base.
Disclosures: The presenter reported having no financial conflicts of interest regarding her university-funded study.