Research Synthesis Methods

Conventional meta‐analytic techniques rely on the assumption that effect size estimates from different studies are independent and have sampling distributions with known conditional variances. The independence assumption is violated when studies produce several estimates based on the same individuals or there are clusters of studies that are not independent (such as those carried out by the same investigator or laboratory). This paper provides an estimator of the covariance matrix of meta‐regression coefficients that are applicable when there are clusters of internally correlated estimates. It makes no assumptions about the specific form of the sampling distributions of the effect sizes, nor does it require knowledge of the covariance structure of the dependent estimates. Moreover, this paper demonstrates that the meta‐regression coefficients are consistent and asymptotically normally distributed and that the robust variance estimator is valid even when the covariates are random. The theory is asymptotic in the number of studies, but simulations suggest that the theory may yield accurate results with as few as 20–40 studies. Copyright © 2010 John Wiley & Sons, Ltd.

We present a new tool for meta‐analysis, Meta‐Essentials, which is free of charge and easy to use. In this paper, we introduce the tool and compare its features to other tools for meta‐analysis. We also provide detailed information on the validation of the tool. Although free of charge and simple, Meta‐Essentials automatically calculates effect sizes from a wide range of statistics and can be used for a wide range of meta‐analysis applications, including subgroup analysis, moderator analysis, and publication bias analyses. The confidence interval of the overall effect is automatically based on the Knapp‐Hartung adjustment of the DerSimonian‐Laird estimator. However, more advanced meta‐analysis methods such as meta‐analytical structural equation modelling and meta‐regression with multiple covariates are not available. In summary, Meta‐Essentials may prove a valuable resource for meta‐analysts, including researchers, teachers, and students.

There is ongoing interest in including grey literature in systematic reviews. Including grey literature can broaden the scope to more relevant studies, thereby providing a more complete view of available evidence. Searching for grey literature can be challenging despite greater access through the Internet, search engines and online bibliographic databases. There are a number of publications that list sources for finding grey literature in systematic reviews. However, there is scant information about how searches for grey literature are executed and how it is included in the review process. This level of detail is important to ensure that reviews follow explicit methodology to be systematic, transparent and reproducible.

The purpose of this paper is to provide a detailed account of one systematic review team's experience in searching for grey literature and including it throughout the review. We provide a brief overview of grey literature before describing our search and review approach. We also discuss the benefits and challenges of including grey literature in our systematic review, as well as the strengths and limitations to our approach. Detailed information about incorporating grey literature in reviews is important in advancing methodology as review teams adapt and build upon the approaches described. Copyright © 2013 John Wiley & Sons, Ltd.

Meta‐analyses are fundamental tools for collating and synthesizing large amounts of information, and graphical displays have become the principal tool for presenting the results of multiple studies of the same research question. We review standard and proposed graphical displays for presentation of meta‐analytic data, and offer our recommendations on how they might be presented to provide the most useful and user‐friendly illustrations. We concentrate on graphs that specifically aim to present similar sorts of univariate results from multiple studies. We start with forest plots and funnel plots, and proceed to Galbraith (or radial) plots, L'Abbé (and related) plots, further plots useful for investigating heterogeneity, plots useful for model diagnostics and plots for illustrating likelihoods and Bayesian meta‐analyses. Copyright © 2010 John Wiley & Sons, Ltd.

The field of evidence synthesis is growing rapidly, with a corresponding increase in the number of software tools and workflows to support the construction of systematic reviews, systematic maps, and meta‐analyses. Despite much progress, however, a number of problems remain, including slow integration of new statistical or methodological approaches into user‐friendly software, low prevalence of open‐source software, and poor integration among distinct software tools. These issues hinder the utility and transparency of new methods to the research community. Here, I present revtools, an R package to support article screening during evidence synthesis projects. It provides tools for the import and deduplication of bibliographic data, screening of articles by title or abstract, and visualization of article content using topic models. The software is entirely open‐source and combines command‐line scripting for experienced programmers with custom‐built user interfaces for casual users, with further methods to support article screening to be added over time. revtools provides free access to novel methods in an open‐source environment and represents a valuable step in expanding the capacity of R to support evidence synthesis projects.