Methods for estimating the sampling variance of the standardized mean difference

Abstract

One of the most widely used effect size indices for meta-analysis in psychology is the standardized mean difference (SMD). The most common way to synthesize a set of estimates of the SMD is to weight them by the inverse of their variances. For this, it is necessary to estimate the corresponding sampling variances. Meta-analysts have a formula for obtaining unbiased estimates of sampling variances, but they often use a variety of alternative, simpler methods. The bias and efficiency of five different methods that have been proposed and that are implemented in different computerized calculation tools are compared and assessed. The data from a set of published meta-analyses are also reanalyzed, calculating the combined estimates and their confidence intervals, as well as estimates of the specific, between-studies variance, using the five estimation methods. This test of sensitivity shows that the results of a meta-analysis can change noticeably depending on the method used to estimate the sampling variance of SMD values, especially under a random-effects model. Some practical recommendations are made about how to choose and implement the methods in calculation resources