Risk of Bias Assistant

The Risk of Bias Assistant aims to facilitate the rating of studies according to the Metapsy adaptation of Cochrane’s Risk of Bias Tool (Version 2) for psychological intervention trials. Using the assistant, you can address all relevant signaling questions step by step. Based on your current ratings, the assistant will automatically skip questions that are not required to complete the rating algorithm.

Once all required questions are answered, the assistant automatically performs the domain-specific and overall risk of bias rating. The results can be downloaded as an MS Excel (.xlsx) file or copied to the clipboard.

The Risk of Bias Assistant can be found at metapsy.org/rob/assistant.

User Interface

In a full-width browser window, the Risk of Bias Assistant consists of three columns. The left column displays the current progress in the rating. The five risk of bias domains (Randomization Process, Intervention Deviations, Missing Data, Outcome Measurement, Selection of Results) serve as subsections.

By default, these sections are locked when a new risk of bias rating is started. As you answer the required signaling questions and/or data fields, new sections are unlocked step by step.

At the top, the left column also shows the current version of the Metapsy risk of bias tools on which the assistant is based. Please note that changes or amendments to the tool are possible, meaning that the behavior of the Risk of Bias Assistant may also change slightly over time.

In the top-left area, you can also find the Cite this tool button. When clicked, this button displays the current preferred citation for the Metapsy risk of bias tools.

The center column shows the currently active signaling question and/or data field for which user input is needed. For example, on the starting page, it is possible (but not required) to enter the name of the study and the rated outcome.

The right column shows an Information Box with further links. Typically, these links lead to the relevant sections of the Metapsy Risk of Bias Handbook and the Cochrane Handbook for Systematic Reviews of Interventions.

On most pages, users must answer a signaling question about the present study. To assist with the judgment, an explanation section is provided for each question, which can be expanded by clicking the “Explanation” button.

An answer to the signaling question is provided by selecting one of the three rating options in the bottom-right corner: No/Probably No (PN), No Information (NI), or Yes/Probably Yes (PY).

By default, a signaling question is always coded as “No/PN”. Once the correct answer is selected, you can use the blue “Proceed” button, hit (right), or (Enter) to go to the next section.

Depending on your selections, the assistant may “skip” one or several signaling questions not required for obtaining a risk of bias rating. If you go back to a signaling question (e.g., using the key) and change your selection, new signaling questions may unlock, requiring answers before you can proceed.

Assessment & Download

Once all signaling questions have been answered, the assistant will unlock the “Assessment” page, which provides the rating for each domain as well as the overall risk of bias assessment.

By clicking on one of the domains, you can expand the specific rating for each signaling question. Using the “ Edit” button, you can return to the original page of the question to revise your rating.

At the bottom of the assessment page, you can find three options to download/copy the generated rating:

Download .xlsx (long): Downloads an MS Excel file containing columns for each signaling question, domain rating, and the overall rating.
Download .xlsx (short): Downloads an abridged version of the MS Excel file, which includes only the domain ratings and overall risk of bias rating.
Copy Table: Copies a table with ratings (long format) to your clipboard. This functionality is useful if you plan to rate multiple studies and want to collect all ratings in a single MS Excel file. To collect the data, open an empty MS Excel sheet and paste the copied data into the sheet.

After saving your results, you can click the “ Rate New Study” button (top-right) to start a new rating. Please note that this will delete the current rating.