U. S. Department of Health and Human Services pdf Empirical Assessment of Within-Arm Correlation Imputation in Trials of Continuous Outcomes

It is common that studies do not report sufficient data to allow meta-analysis of continuous outcomes. The standard error (SE) of the within-group differences is often not reported and cannot be calculated because the within-group correlation is unknown. For meta-analysis of net-changes, one must thus estimate the SE based on an arbitrarily chosen correlation. The objective of this study is to better understand how to impute within-arm correlation for meta-analyses of continuous outcomes when data are missing, this study describes the range of correlation values in a representative set of studies with sufficient data reported, and simulates the effect of using different correlation values on meta-analysis summary estimates when imputing missing data. From articles available to us from previous systematic reviews and from trials registered at ClinicalTrials.gov, we selected those that prospectively compared two or more interventions for continuous outcomes and reported all three of: baseline means and SEs (or equivalent), final means and SEs, and within-group changes and SEs. From these data we back-calculated correlation values for each study group. We described these data and tested for patterns based on study characteristics. We assessed the bias on estimates of within-group change SEs by comparing reported SEs with imputed SEs using arbitrarily chosen correlation values. We simulated meta-analyses, assessing the bias, coverage, and accuracy of the summary estimates derived from studies with missing correlation data. We analyzed 811 within-group correlation values from 123 studies with 281 study groups. The median (interquartile range) within-group correlation values across all studies was 0.59 (0.40, 0.81). Active treatment groups had lower correlation values (median 0.54) than no treatment groups (median 0.73, P<0.001). There was heterogeneity of correlation values across both outcome types and clinical domains. There was no apparent association with followup duration, but correlation values were lower with increasing sample size among no treatment groups. In the empiric dataset, imputing low correlation values (0 or 0.25) yielded an overestimation of the within-group SE in more than 85 percent of cases; imputing a correlation of 0.5 yielded values closer to those actually reported. Imputation had similar effects on the net-change SE. Simulation studies informed by the empirical results, demonstrated that imputation of values does not introduce bias in the meta-analysis estimate. Imputing values higher than the true correlation resulted in coverage probabilities that were lower than those in analyses using the complete data. However, coverage probabilities were generally lower than nominal (<0.95 even with complete data) in the presence of moderate to substantial between study heterogeneity, despite using random effects models (DerSimonian-Laird). Negative within-group correlation values are very uncommon in clinical studies. Imputing values in meta-analyses where some or all within-group correlation estimates are not reported does not introduce bias in the summary estimate of the treatment effect. However, imputation can affect the SE of the summary estimate when the imputed value is different from the “true.” In such cases, sensitivity analyses using alternative imputation values, possibly informed by studies reporting relevant information, are recommended.