Studies reveal lack of transparency and poor reporting of research

Researcher in the lab

Photo by Bill Branson

Two new studies suggest biomedical research may be hindered by poor reporting and a lack of transparency.

In one study, researchers analyzed more than 400 biomedical science articles and found the papers rarely provided full protocol information, complete data, and the necessary level of transparency to verify or replicate the work.

In the other study, researchers analyzed more than 500 preclinical experiments and found that most didn’t contain sufficient information on the animals used.

Both studies were published in PLOS Biology.

For the first study, Shareen Iqbal, PhD, of Emory University in Atlanta, Georgia, and her colleagues analyzed papers published between 2000 and 2014.

The team set out to determine the extent to which researchers report key information necessary for properly evaluating and replicating published research, including availability of protocols, data, and the frequency of published novel or replication studies.

Out of 441 articles drawn from across the biomedical literature, only 1 paper provided a full protocol, and none of the papers made all the data available. The majority of studies didn’t state funding or conflicts of interest, and replication studies were very rare.

Dr Iqbal and her colleagues said they hope their study will further sensitize scientists, funders, journals, and other science-related stakeholders about the need to improve these indicators.

For the second study, Ulrich Dirnagl, MD, of Charité Universitätsmedizin in Berlin, Germany, and his colleagues examined 100 papers describing preclinical research on stroke and cancer. These papers contained accounts of 316 experiments on infarct volume and 206 experiments on tumor shrinkage.

The vast majority of the reports didn’t contain sufficient information on how many animals were used in the experiments. What’s more, in many papers, animals “vanished” over the course of the study.

Using a computer model, the researchers simulated the effects of such animal loss on the validity of the experiments. They found that the more animals lost or removed, the shakier or more biased the experimental conclusions.

“The study began with an attempt to look at the robustness of findings in a handful of preclinical papers, but the sheer number of missing animals stopped us in our tracks,” said author Constance Holman, a graduate student at Charité Universitätsmedizin.

Researchers from both studies believe their findings add to the list of concerns about bias and reporting in research, but the results also establish ways in which research can become more transparent and potentially more reproducible.

Researcher in the lab

Photo by Bill Branson

Two new studies suggest biomedical research may be hindered by poor reporting and a lack of transparency.

In one study, researchers analyzed more than 400 biomedical science articles and found the papers rarely provided full protocol information, complete data, and the necessary level of transparency to verify or replicate the work.

In the other study, researchers analyzed more than 500 preclinical experiments and found that most didn’t contain sufficient information on the animals used.

Both studies were published in PLOS Biology.

For the first study, Shareen Iqbal, PhD, of Emory University in Atlanta, Georgia, and her colleagues analyzed papers published between 2000 and 2014.

The team set out to determine the extent to which researchers report key information necessary for properly evaluating and replicating published research, including availability of protocols, data, and the frequency of published novel or replication studies.

Out of 441 articles drawn from across the biomedical literature, only 1 paper provided a full protocol, and none of the papers made all the data available. The majority of studies didn’t state funding or conflicts of interest, and replication studies were very rare.

Dr Iqbal and her colleagues said they hope their study will further sensitize scientists, funders, journals, and other science-related stakeholders about the need to improve these indicators.

For the second study, Ulrich Dirnagl, MD, of Charité Universitätsmedizin in Berlin, Germany, and his colleagues examined 100 papers describing preclinical research on stroke and cancer. These papers contained accounts of 316 experiments on infarct volume and 206 experiments on tumor shrinkage.

The vast majority of the reports didn’t contain sufficient information on how many animals were used in the experiments. What’s more, in many papers, animals “vanished” over the course of the study.

Using a computer model, the researchers simulated the effects of such animal loss on the validity of the experiments. They found that the more animals lost or removed, the shakier or more biased the experimental conclusions.

“The study began with an attempt to look at the robustness of findings in a handful of preclinical papers, but the sheer number of missing animals stopped us in our tracks,” said author Constance Holman, a graduate student at Charité Universitätsmedizin.

Researchers from both studies believe their findings add to the list of concerns about bias and reporting in research, but the results also establish ways in which research can become more transparent and potentially more reproducible.

Researcher in the lab

Photo by Bill Branson

Two new studies suggest biomedical research may be hindered by poor reporting and a lack of transparency.

In one study, researchers analyzed more than 400 biomedical science articles and found the papers rarely provided full protocol information, complete data, and the necessary level of transparency to verify or replicate the work.

In the other study, researchers analyzed more than 500 preclinical experiments and found that most didn’t contain sufficient information on the animals used.

Both studies were published in PLOS Biology.

For the first study, Shareen Iqbal, PhD, of Emory University in Atlanta, Georgia, and her colleagues analyzed papers published between 2000 and 2014.

The team set out to determine the extent to which researchers report key information necessary for properly evaluating and replicating published research, including availability of protocols, data, and the frequency of published novel or replication studies.

Out of 441 articles drawn from across the biomedical literature, only 1 paper provided a full protocol, and none of the papers made all the data available. The majority of studies didn’t state funding or conflicts of interest, and replication studies were very rare.

Dr Iqbal and her colleagues said they hope their study will further sensitize scientists, funders, journals, and other science-related stakeholders about the need to improve these indicators.

For the second study, Ulrich Dirnagl, MD, of Charité Universitätsmedizin in Berlin, Germany, and his colleagues examined 100 papers describing preclinical research on stroke and cancer. These papers contained accounts of 316 experiments on infarct volume and 206 experiments on tumor shrinkage.

The vast majority of the reports didn’t contain sufficient information on how many animals were used in the experiments. What’s more, in many papers, animals “vanished” over the course of the study.

Using a computer model, the researchers simulated the effects of such animal loss on the validity of the experiments. They found that the more animals lost or removed, the shakier or more biased the experimental conclusions.

“The study began with an attempt to look at the robustness of findings in a handful of preclinical papers, but the sheer number of missing animals stopped us in our tracks,” said author Constance Holman, a graduate student at Charité Universitätsmedizin.

Researchers from both studies believe their findings add to the list of concerns about bias and reporting in research, but the results also establish ways in which research can become more transparent and potentially more reproducible.