The Meta-Analysis Unit

Reliability Generalization Meta-analysis: The REGEMA Checklist

Reliability generalization (RG) meta-analyses are a special kind of meta-analyses whose purpose is to investigate how reliability of test scores varies as a function of the context of application of the test and of the composition and variability of the test scores. In an RG meta-analysis, the outcomes from each primary study are reliability coefficients obtained with the data at hand, and substantive, methodological, and contextual characteristics of the studies are the potential moderators that can explain, at least, part of the variability typically exhibited by the reliability estimates.

Several reporting guidelines have been developed in the meta-analytic arena to help researchers report meta-analyses (e.g., PRISMA, AMSTAR, MOOSE, MARS, etc.). However, none of them was developed to be applied on RG meta-analyses, but on meta-analyses on intervention effects or on the relationship between variables. The special characteristics of RG meta-analyses make difficult to efficiently apply the checklists proposed to date in the literature. The REGEMA checklist is a tool elaborated by the research team of the Meta-analysis Unit that intends to help researchers correctly develop and report RG meta-analyses.

The REGEMA checklist is a tool composed by 30 items (1 item for the Title, 1 for the Abstract, 2 for the Introduction, 14 for the Methods, 6 for the Results, 4 for the Discussion, 1 for Funding, and 1 for Protocol) that is very easy to apply. REGEMA can be applied both by researchers that intend to write an RG meta-analysis and to make a critical reading of RG meta-analyses, as well as for editors and reviewers as a tool to assess the adequacy of an RG meta-analysis.

A MS Word version of the REGEMA checklist can be freely downloaded from the following link:

REGEMA checklist

In addition, we have elaborated a flowchart, similar to that proposed by the PRISMA statement, but adapted to the special characteristics of the RG meta-analyses, to help researchers to describe the process for searching and selecting the primary studies that fulfil the selection criteria of an RG meta-analysis.

A MS Word version of the REGEMA flowchart can be freely downloaded from the following link:

REGEMA flowchart

How to cite the REGEMA checklist:

Sánchez-Meca, J., Marín-Martínez, F., López-López, J. A., Núñez-Núñez, R. M., Rubio-Aparicio, M., López-García, J. J., López-Pina, J. A., Blázquez-Rincón, D., López-Ibáñez, C., & López-Nicolás, R. (2021). Improving the reporting quality of reliability generalization meta-analyses: The REGEMA checklist. Research Synthesis Methods, online publication. Doi:10.1002/jrsm.1487.