REPRODUCIBILITY

What is it?

The Brazilian Reproducibility Initiative is a multicenter initiative to estimate the reproducibility of Brazilian biomedical science. Supported by the Serrapilheira Institute, our goal is to reproduce a sample of 60 experiments of Brazilian articles, initially using 3 methods: MTT cell viability assay, elevated plus maze and RT-PCR.

 

Why focus on Brazilian articles?

Because knowing how much of our published research is reproducible is vital to improve the country’s science. As such efforts are not routinely performed by funding agencies, we believe that the scientific community must face this task as a collective duty.

 

What kinds of experiments are replicated?

The selected experiments include models and methods that are used by multiple laboratories in the country. These methods were selected on the basis of a wide survey of the most commonly used techniques in Brazilian publications.

 

How were experiments selected for replication?

We based our work on a random sample of Brazilian articles, selecting individual relevant experiments in 60 articles. To decrease the risk of bias, the article of origin of each experiment will only be revealed at the end of the study.

 

How are replications performed?

Each experiment is reproduced in 3 different laboratories, with preregistered protocols developed to be as close as possible to those of the original study. Measures to decrease bias such as randomization of samples, blinding of experimenters and sample size calculations are used to ensure rigor.

 

When will a replication be deemed successful?

As done by previous initiatives of this kind, we will use multiple criteria to assess the replications. For instance, one of them considers that an experiment has been successfully replicated when its effect size is within the 95% prediction interval defined by the effect sizes obtained in each of the three replications. Our analysis plan can be seen here.

 

If an experiment is not replicated, does it mean that the original result was wrong?

No. Many factors can lead to non-reproducibility, including uncontrollable factors related to individual laboratory conditions. That said, performing replications in different labs will allow us to estimate interlaboratory variation for each method, and thus judge whether the original effect is within the expected range of effects.

 

What will be done with the data?

Besides estimating the reproducibility of experiments in specific areas of Brazilian science, we will use the database to study if there are variables in the original studies that can predict successful replications, in order to identify factors that can increase the reproducibility of a finding.

 

Does the study plan to evaluate individual researchers?

No. On the contrary, we did not include more than one experiment per article or research group, and is thus insufficient to judge them individually. Our goal is to estimate the general reproducibility of experiments in Brazilian biomedical science and propose ways to improve it.

 

How can I participate?

The teams and laboratories that replicate the experiments have already been defined, although we might add more groups at some point, if needed. To hear about new opportunities for contributing to the Initiative, follow us on social media and here on the website. You can also sign up for our e-mail newsletter.

 

What do I gain by being a collaborator?

All participating laboratories will be co-authors in the Brazilian Reproducibility Initiative, which will sign articles derived from the project as a consortium. The costs of experiments will be financed by a grant from the Serrapilheira Institute.

 

What does Brazilian science gain with this effort?

At the end of the study, we will have performed the first systematic study of research reproducibility at a national level, at least to our knowledge. Independently of the results, the process of generating these data will allow us to reflect on our individual and collective practices, helping to place Brazil at the forefront of the international effort to develop more reliable scientific practices.