Submissions

Registered Reports

Registered Reports should follow everything above (e.g., preprint that is updated), but has additional instructions. 

A registered report is an article form in which theory, data collection and/or proposed analyses are pre-registered and reviewed prior to the data collection and/or proposed analyses. Registered reports are submitted and reviewed in two stages. Once before data collection and/or analyses, and once when the final paper is completed. A unique feature is that registered reports are given in-principle-acceptance (IPA) prior to data collection and/or analysis.

The registered report is on the one hand one of the most convincing ways of doing research, as it precludes p-hacking and HARKing. We therefore very much welcome submissions of the ‘registered report’ type. On the other hand, the registered report is also one of the most challenging ways of doing research, as many important decisions of the research need to be stipulated beforehand. We provide a checklist that authors and reviewers need to use when writing and evaluating a registered report, to assist them in creating a high-standard that prevents p-hacking and HARKing.

The registered report guidelines described below are based on empirical hypothesis testing research, primarily as a way to prevent p-hacking and HARKing. However, the format of planning research together with reviewers before data collection / and or analysis can be useful regardless of the type of research. Authors are strongly encouraged to submit other types of contributions in this format, but should be aware that the guidelines below do not fully apply then. Please contact the Editor-in-Chief about such submissions so that we can plan ahead for it.

Rarely, a registered report cannot be made public immediately because that would interfere with the project itself. For example, if the study is an audit of researchers who are likely to read Meta-Psychology. In that case, send an email to the Editor-in-Chief instead of submitting.

Author

The guidelines for both authors and reviewers follow those of the journal Cortex http://cdn.elsevier.com/promis_misc/PROMISpub_idt_Guidelines_cortex_RR_17_04_2013.pdf with a few notable exceptions described below.

Registered reports that plan to collect data in small samples and consequently may have low statistical power are considered for publication. The reason is that small-sample research without bias (i.e., no publication bias, no ­p-hacking, no HARKing) may be very useful for subsequent meta-analyses. However, this has to be made explicit and justified in the paper.

In stage 1, authors submit papers including a full introduction and methods section. The methods section must have a statistical analyses subsection that explains all analyses to test the hypotheses derived in the introduction. If possible, we recommend that the code for the proposed analysis are included as supplementary material and tested on simulated mock data.

The introduction and methods section may not be changed after the report is in-principle-accepted for publication. That is, these sections will not change while or after collecting data and/or doing the analyses.

Authors should use a checklist (https://osf.io/6bv27/) when writing their report in the first stage to preclude p-hacking and HARKing. To be more precise, we require researchers to be specific, precise, and exhaustive in their plan of the study (Wicherts et al., 2016). These three prerequisites are translated into the scoring of items of the checklist, with a score of 3 tantamount to a specific, precise and exhaustive description with respect to an item. Authors should aim at score “3” for each relevant item on the checklist.

In stage 2, the final report should consist of a “planned analyses” section, and possibly an “exploratory analyses” section. Authors are free to report any analyses in the “exploratory analyses” section, as long as the details of these analyses are provided in the article (or in supplements).

Note that the editor and reviewers will NOT evaluate the registered report on its novelty, statistical significance, or how ‘exciting’ it is. The main criteria for acceptance are

1. Soundness of theory, and hypotheses following from the theory
2. Soundness of methodology and statistics to test the hypotheses
3. Adherence to the checklist (aiming for ‘3’s on items of the checklist)
4. The report in part 1 being the same as the report in part 2 (wrt introduction and methods)
5. Results section being divided into one part corresponding to the “planned analyses” section, and possibly one “exploratory” part; does the first part correspond to the         planned analyses section, and does the exploratory part include the details of the analyses? Are statistical techniques used that are appropriate for exploratory data analysis?
6. Soundness of conclusion and discussion section

Reviewers

Reviewers should carefully read the author instructions as a registered-report is quite different. Reviewers see the paper at least twice, although it is possible that a reviewer is involved only in one stage of the review. Simply put, the “hard job” for reviewers is at the first stage of the research, when the reviewer judges the soundness of the proposed research and the adherence to the checklist. At the second stage, the reviewer’s most important task is to check the correspondence between the proposed research and the actual analyses.

References

Wicherts, J.M., Veldkamp, C.L.S., Augusteijn, H.E.M., Bakker, M., Van Aert, R.C.M., and Van Assen, M.A.L.M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers.  http://dx.doi.org/10.3389/fpsyg.2016.01832

Original articles

This section contains original articles on the topic of meta-psychology. All articles in this section have been peer-reviewed according to the journal policy.

Tutorials

Tutorials on methods relevant for psychology researchers.

Commentaries

This section contains commentaries on articles published in Meta-Psychology as well as commentaries on articles published in other journals. Re-analysis of own research are essentially a self-commentary and should also be part of this section.

Registered Replication Reports

Replication studies that are submitted as proposals that are given in-principle acceptance prior to data collection.

Replication Reports

Replications of previous empirical findings on any topic relevant for psychology. Has to be preregistered.

File-Drawer Reports

This section contains File-Drawer Reports. Unlike Original articles, these need not be on the topic of Meta-Psychology but can be on any topic of psychology. They require a complete emptying of a file-drawer on a topic, including - but not limited to - null results and failed studies. 

File-Drawer Reports is a reporting format where researchers have the opportunity to completely empty their file-drawer on a psychology topic. File-Drawer-Reports should not compete with journals on general or specialized psychology topics, and should be specifically for studies that would elsewhere remain in the file-drawer.

They should contain a transparency statement:

"This report is an exhaustive report on all data available from research project(s) relating to the topic, where at least one of the authors was principal investigator, or have otherwise the right to publish  the results. This includes not only null findings, or unexpected findings, but also studies that are suspected to have failed, with careful explanation of the circumstances of the failure (e.g., experimental error, failed manipulation check). The context surrounding how these data were collected, and if they are somehow connected to already published studies (e.g., dropped experiments) is carefully explained. "

The primary goal of a File-Drawer-Report is to reduce the impact of publication bias by making hard to publish data available. Another goal is to make researchers learn from failures and mistakes. Poorly designed studies that are part of the topic should thus be included, even if they are embarrassing for the authors,  but their limitations should be carefully explained.