Welcome to the Systematic Multiverse Analysis Registration Tool!
This app is designed to guide researchers through the transparent and systematic definition of pipelines to be included in multiverse analyses. By using this tool, you can ensure your research meets recently proposed guidelines (Short et al., 2025) of transparency and rigor.
Why Multiverse Analysis?
Multiverse analysis addresses critical issues in research replicability by reporting the uncertainty that arises in the reported outcome due to the multiplicity of defensible data processing and analysis decisions. By reporting a distribution of outcomes across defensible variations in data processing and analysis, researchers can:
- Communicate uncertainty: Illustrate how different analytical choices influence research findings.
- Foster robust scientific findings: Build confidence in results by demonstrating their consistency across defensible analytic variations.
- Enhance transparency: Clearly document and communicate the decision-making process.
In order to achieve these benefits, the multiverse should be principled (Del Giudice & Gangestad, 2021), including only defensible pipelines that are arbitrary in terms of a specific criterion, such as validity, testing for the same effect, and estimating the effect with comparable precision. This ensures that the included pipelines align with the research goals without introducing irrelevant noise.
Implementing this principled approach requires transparency and systematicity throughout the decision-making process. This is where the Multiverse Analysis Preregistration App helps, providing researchers with a structured, user-friendly platform to define, evaluate, and refine their multiverse analyses. By facilitating these critical steps, the app ensures that researchers can achieve transparency, rigor, and replicability in their multiverse studies.
Purpose and Use of the App
The Multiverse Analysis Preregistration App is designed as a practical solution for researchers across disciplines, providing a user-friendly interface for defining, evaluating, and refining analytic pipelines in a systematic and transparent way.
The app serves several key purposes:
- Supporting rigorous decision-making: Help users identify defensible pipelines and refine them based on clear criteria, reducing the risk of bias.
- Efficient and systematic implementation: Save time and effort by using a structured workflow.
- Facilitating transparency: Generate detailed documentation of the decision-making process that can be shared as a multiverse analysis preregistration of pipelines and as supplementary information with the reported results, to make your multiverse analysis traceable and reproducible.
- Encouraging interdisciplinary innovation: Bridge gaps between fields by providing a standardized platform for multiverse analysis that incorporates best practices and fosters collaboration.
The app is intended for use by researchers in fields that can benefit from multiverse analysis, such as Sociology, Psychology, Neurocognitive Science, Education, Economics, Epidemiology and beyond, helping them to define and refine their multiverse analyses with greater ease. Whether you are conducting a single study or collaborating on a large-scale interdisciplinary project, this tool is designed to make multiverse analysis more accessible and impactful.
App Features
This app walks you through two key stages:
- Multiverse 1.0: Define the defensible pipelines for your analysis.
- Multiverse 2.0: Refine these pipelines into a principled multiverse (Del Giudice and Gangestad, 2021).
Each step is designed to be intuitive and exportable. Overall this app is here to guide you through a transparent, systematic, and replicable research process.
You can watch the instruction video that demonstrates the use of app
Please enter a username in the text box below that will be used as a unique identifier for your multiverse in our repository.
All data (username and pipeline decisions) are processed and stored in compliance with GDPR regulations. The application is hosted by the Shiny App server of the META-REP project (Gollwitzer, 2020).
Continue Your Multiverse Analysis
If you have previously completed Multiverse 1.0 and saved your progress, you can upload your saved Construction Documentation zip folder to continue from where you left off. Please do not change the file structure within the zip folder. You will be able to upload this zip folder later to retrieve your progress.
Once uploaded, the app will automatically restore your previous selections and inputs, allowing you to continue directly with Multiverse 2.0.
References
Del Giudice, M., & Gangestad, S. W. (2021). A traveler’s guide to the multiverse: Promises, pitfalls, and a framework for the evaluation of analytic decisions. Advances in Methods and Practices in Psychological Science, 4(1), 2515245920954925. https://doi.org/10.1177/2515245920954925
Gollwitzer, M. (2020). DFG Priority Program SPP 2317 Proposal: A meta-scientific program to analyze and optimize replicability in the behavioral, social, and cognitive sciences (META-REP). PsychArchives. https://doi.org/10.23668/PSYCHARCHIVES.3010
Short, C. A., Breznau, N., Bruntsch, M., Burkhardt, M., Busch, N., Cesnaite, E., Frank, M., Gießing, C., Krähmer, D., Kristanto, D., Lonsdorf, T., Neuendorf, C., Nguyen, H. H. V., Rausch, M., Schmalz, X., Schneck, A., Tabakci, C., Hildebrandt, A. (2025a). Multi-curious: A multi-disciplinary guide to a multiverse analysis. MetaArXiV. https://doi.org/10.31222/osf.io/4yzeh_v1
In this step of the procedure, you should review your listed decision nodes and options and categorise them as defensible (even if the defensibility is conditional on other options, or on its place along the pipeline), or indefensible (regardless of other options along the pipeline, or where this would be placed along the pipeline sequence, this option at this decision node will never be defensible for your research question or dataset). You do not need to consider equivalence yet – only the defensibility of the decision nodes and options.
Using the radio buttons, please label each decision node and option as defensible or indefensible.
Using the text boxes below, write the justifications that will be used for your defensibility decisions. Then assign the relevant justifications to each defensible or indefensible label.
This process ensures that only valid and justified elements are included in your analysis, promoting rigor and transparency.
Here you can see all of the decision nodes and options that were categorized as defensible in the previous step.
By dragging and dropping the defensible decision nodes and options, please create sequences of defensible pipelines. Please construct all plausible combinations. You do not need to consider equivalence yet – only defensible combinations.
To start a new pipeline, click 'New Pipeline' below.
Below you can see a table of each saved pipeline and a path diagram of the defensible multiverse that you have created. You can export this by clicking the ‘Export’ button underneath each. Please continue the procedure to refine the Multiverse 1.0 into a principled multiverse based on equivalence.
To transition to a principled multiverse in a data-driven manner, please compute your multiverse 1.0 on a subset of your data (e.g., 10% of your sample of participants), plotting a quality metric and not an effect size of interest. Pipelines that fall within an interval threshold, which you will define à priori below, will be considered as equivalent and will be included in your principled multiverse.
In the text box below, please enter the criterion for equivalence (e.g., signal-to-noise ratio, standardised measurement error, intraclass correlation coefficient).
In the text box below, please set a threshold for equivalence (e.g., the interval values of proportion that will be used as a threshold for declaring equivalence in the aforementioned criterion).
In the text box below, specify the subsample for equivalence testing (e.g., a percentage of the final sample of participants).
Perform Equivalence Analysis
Please compute the multiverse 1.0 on the subset of data specified above, plotting the quality metric specified above.
Below you see the list of pipelines in your Multiverse 1.0. For each pipeline, please enter the value of the quality metric, and please select either ‘Type E’ and ‘Type N’ next to each pipeline to report whether the pipeline did or did not meet the threshold for equivalence based on your criteria.
When you have done this, you can proceed to the final tab to view and export your principled multiverse analysis.
Pipeline Structure
Visual Network
Below you can find a table and visual representation of the principled multiverse. This can now be computed on your full sample to observe the distribution of the effect of interest across these equivalence variations.
Export Your Work
Download the complete documentation, including multiverse 1.0, multiverse 2.0, and equivalence thresholds, in a .pdf format for preregistration or publication.
Principled Pipeline Structure
Visual Network
Download Your Principled Multiverse PDF
Thank you for using the Systematic Multiverse
Analysis Registration Tool
Our team
Hildebrandt, Andrea, Prof. Dr. rer. nat.
Short, Cassie, PhD.
Frank, Maximilian, MsC.
