Testing for moderators of the amount of heterogeneity: Location-scale models in meta-analysis

Abstract

Location-scale models are an improvement in the field of meta-analysis since they allow the influence of moderator variables on the mean (location) and variance (scale) of the distribution of true effects to be studied simultaneously. However, the increased complexity of such models can make model fitting challenging. Moreover, the statistical properties of the estimation and inference methods of such models have not been systematically examined in the meta-analytic context. We therefore conducted a Monte Carlo simulation study to compare different estimation methods (maximum or restricted maximum likelihood estimation), significance tests (Wald-type, permutation, and likelihood-ratio tests), and methods for constructing confidence intervals (Wald-type and profile-likelihood intervals) for the scale coefficients of such models. The permutation test yielded Type I error rates closest to the nominal level, whereas the likelihood-ratio test obtained the highest statistical power. In most scenarios, profile-likelihood intervals showed lower coverage probabilities than the Wald-type interval, but nearer to the nominal 95% level. Concerning the estimation method, more desirable rejection rates and narrower intervals were obtained when restricted maximum likelihood estimation was used. Finally, slightly higher rejection rates and coverage probabilities were obtained when a dichotomous moderator was examined rather than a continuous one. Despite the need to use some constraints on the parameter space for the scale coefficients and the possibility of nonconvergence of some procedures that may affect the fitting of the specified models, location-scale models proved to be a valid and useful tool for modelling the heterogeneity parameter in meta-analysis.