Enhancing evidence-based decision making in healthcare by addressing meta-analytic methods

Systematic reviews (SRs) are considered the gold standard of evidence in decision making in healthcare. SRs attempt to provide an objective summary of an evidence base by identifying all available primary studies on a specific topic. This process often includes a meta-analysis, a method used to statistically combine results from two or more primary studies, thereby condensing the evidence into a single effect estimate. SRs and meta-analyses have the potential to greatly influence patient care and outcomes and so the methods used to conduct SRs and meta-analyses need to be carefully considered.

Traditional pairwise meta-analysis models only have the capability of incorporating data for one outcome of interest for a maximum of two comparison groups and are commonly used in practice. However, innovation in meta-analysis methodology means that more complex meta-analysis models are available which are capable of specifying the extent of the missing outcome data (MOD) in each study. Other models are able to incorporate outcome data for multiple outcomes or multiple comparison groups to be meta-analysed simultaneously. In practice, it is rare for no missing data to be present in the primary studies or for only one outcome or treatment of interest to be meta-analysed in a SR, meaning that there is great potential for uptake of these newer models. It has also proposed that using these models may increase the robustness and precision of results from meta-analysis, increasing their ability to accurately reflect the true effect and inform clinical guidelines to promote evidence-based practice.

The central hypothesis of this thesis is that accounting for missing data, analysing multiple outcomes simultaneously or conducting a network meta-analysis will increase the robustness of meta-analyses and change the precision of the effect estimates. This hypothesis was tested by conducting three separate SRs with each containing a complex meta-analysis model to incorporate additional information and a naïve meta-analysis model to serve as a comparison.

Firstly, the impact of accounting for MOD in meta-analysis was explored by conducting an overview of SRs with meta-analysis evaluating the impact of lifestyle interventions during pregnancy to reduce postpartum weight retention (PPWR). Results from these meta-analyses were re-analysed using the same data of the outcome and additional information about MOD. There were three SRs identified through database searching which included a total of 15 randomised controlled trials (RCTs) and 4 cluster RCTs across the SRs. All SRs identified showed statistically significant favourable effects of lifestyle interventions but conducted an available case analysis (ACA) which ignores the extent of MOD during analysis. Inconsistency varied across the SRs (I2 = 0% - 60%). In an updated analysis of all 19 trials included in the previous SRs, PPWR was lower for the intervention group than the control group and was statistically significant (MD: -0.80, 95% CI: -1.30, -0.30). There was substantial inconsistency (I2=51.3%, p=0.234). These results were compared to a meta-analysis which accounted for MOD by assuming that, on average, missing participants from the intervention group retained 0.5kg more than observed participants on average and that missing control participants retained the same amount of weight as observed participants. In this analysis, the effect of lifestyle interventions to prevent PPWR was reduced to -0.63 and remained marginally statistically significant (95% CI: -1.17, -0.08). Inconsistency was also reduced (I² = 17.1%). The differences between results from this analysis and the ACA were analysed using a robustness index (RI) where a RI <0.173 indicates robustness. The updated meta-analysis of all 19 trials had a RI of 0.084, meaning the ACA could be considered robust. When considering whether results from the available case analyses of previous SRs were robust to MOD, one of the three SRs had a RI higher than 0.173 and was not considered to be robust (RI=0.327).

A SR of cohort studies evaluating the impact of preeclampsia (PE) on offspring blood pressure was conducted to allow comparison of results from a pairwise meta-analysis model to a multivariate meta-analysis which was used to analyse both systolic blood pressure (SBP) and diastolic blood pressure (DBP) simultaneously. After database and handsearching and study selection, 42 cohort studies comparing blood pressure of offspring born to preeclamptic and normotensive pregnancies were identified. Of these 42 studies, only seven provided estimates which were adjusted for key confounders. In univariate 2-level random effects meta-analyses of effect sizes adequately adjusted for confounders, offspring of preeclamptic pregnancies had higher SBP and DBP than offspring born to normotensive pregnancies (SBP: 2.07 mmHg, 95% CI: 1.33, 2.82; DBP: 1.26 mmHg, 95% CI: 0.49, 2.04). In multilevel multivariate meta-analysis, there were 21 effect sizes of SBP and 18 effect sizes of DBP extracted from the seven cohort studies. The effect of PE on offspring SBP was stronger (MD: 2.32 mmHg, 95% CI: 1.53, 3.11) but the effect of PE on offspring DBP was smaller and no longer statistically significant (MD: 1.04 mmHg, 95% CI: -0.12, 2.20). The borrowing of strength (BoS) statistic was calculated using the variance of the pooled effects from both pairwise and multivariate meta-analysis. BoS was 51.4% for SBP and 33.4% for DBP, indicating that a sizable proportion of the pooled effect was calculated from correlated evidence.

The final study was a SR of probiotics and maintenance therapies, including polyethylene glycol (PEG), mineral oil, lactulose and other commonly used treatments, to treat functional constipation in children. This enabled comparison of pairwise and network meta-analysis for the effect of probiotics as compared to control for the outcomes of defecation frequency (BMs/wk) and treatment success as defined by the study authors. After database searching, handsearching and study selection there were 49 RCTs selected for inclusion. Ten RCTs compared probiotics and placebo directly and were pooled in a pairwise meta-analysis. The pooled effect for defecation frequency was small and not statistically significant (MD = 0.19, 95% CI: -0.14, 0.51) while the pooled effect for treatment success was marginally statistically significant (RR=1.48, 95% CI: 1.03, 2.13). Heterogeneity in the analysis of both outcomes was high (I²=86.0% for defecation frequency, I2=74.1% for treatment success). When direct and indirect evidence from all 49 RCTs were combined in a network meta-analysis, pooled effects for the comparison between probiotics and placebo were increased for the outcome of defecation frequency (MD = 0.57, -0.13, 1.27) but smaller for treatment success (RR=1.40, 95% CI: 1.06, 1.85). The variance of pooled effects was smaller in network meta-analysis and BoS was 24.5% and 40.6% for defecation frequency and treatment success respectively. This was equivalent to a gain in information of 2.5 and 4.8 studies of a similar size to that included in the analysis.

The findings of this thesis demonstrate that the pooled effects of meta-analyses are influenced by the statistical model used. Accounting for MOD in sensitivity analyses revealed some instances where results from an ACA were not robust to MOD when compared to a sensitivity analysis. Precision around effect estimates was greater in multivariate and network meta-analysis, indicating that these models may be more effective in detecting small effects. These differences in effects could influence conclusions of SRs and may affect patient care when findings from SRs are translated into practice. Given the increase in robustness to MOD in effects from meta-analysis which account for MOD, and in the precision of estimates from multivariate meta-analyses, coordinated efforts from researchers and educators are needed to increase the use of these tools which are not often used in practice. The promotion of these tools could be implemented in meta-analysis guidance documents with a discussion of the potential benefits to encourage their uptake. Promotion of more complex meta-analysis models may also involve increasing the knowledge of SR authors about the limitations of naïve pairwise meta-analysis to incorporate additional information about MOD, multiple outcomes and multiple treatments. This may in turn improve the quality of meta-analyses as a whole by preventing the misuse of naïve models, which may result in improved evidence-based decision making in healthcare.