Meta-analysis and heterogeneity
9 important questions on Meta-analysis and heterogeneity
What are different types of heterogeneity
- Clinical heterogeneity/ diversity:
- Differences in clinical features of a study (patients, treatment, outcome)
- Methodological heterogeneity/diversity:
- Different study methodology (RCTs vs non RCTs)
- Statistical heterogeneity -> heterogeneity:
- Results from clinical/methodological (expected)
- Unknown (unexpected)
How do you address clinical and methodological heterogeneity/ diversity?
- Try to avoid it (expected heterogeneity)
- Adequate inclusion and exclusion criteria
- Define relevant subgroups a-priori
How do you identify statistical heterogeneity: unknown (unexpected)
- Common sense: eye-ball 'test'; overlapping Cls & identify source
- Test for homogeneity (Chi2) but type 2 error is likely
- Quantify heterogeneity (I2)
- Describe the % of the variability in effect estimates due to heterogeneity, not chance
- Value >50% sometimes labeled 'substantial'
- Higher grades + faster learning
- Never study anything twice
- 100% sure, 100% understanding
How to address heterogeneity
- Check if data is entered correctly
- Choose to not do a meta-analysis: because the results are just too different
- Especially if they are inconsistent
- Explore heterogeneity
- Subgroup analyses
- Meta-regression
- Do this on pre-specified characteristics
- Ignore heterogeneity (NOT SMART TO DO)
- Change outcome (note: RD was heavily influenced by back ground risk = control group)
- Exclude studies (unwise, unless obvious reasons) (not because it is just an outlier)
- Perform a random effects meta-analysis (=incorporating heterogeneity)
When do you perform a random affects MA
- Lack of knowledge and then assumption: random
Random effects MA model involves an assumption that the effects of individual studies differ
How do you decide whether you do a random or a fixed effects in MA
- Decide before doing the meta-analyses: Whether the intervention effects are truly identical
- Fixed when this is likely
- Random when this is unlikely (mostly never truly identical)
- Others argue that fixed analysis can be used when there is heterogeneity and that it makes fewer assumptions than a random effects
- Lecturer: decide beforehand
How to interpret subgroups with caution
- Subgroups are observational
- Confounding by other trial-level characteristics
- Defined prior to study or post-hoc
- Multiple post-hoc tests: data dredging
- Is there clinical/ biological support
- Is the difference in effect worthwhile
- Between study comparisons not very reliable
- Because they are observational
How robust are the finings
- In every SR many, many decisions and assumptions need to be made
- What if different choices were made
- For this sensitivity analyses are used -> to make the different choices
Choices re:
- Methodological issues
- Inclusion criteria
- Different outcome measures in different studies
- Reanalyzing the data
- Imputing different (sensible) values for missing data
- Different statistical approaches
- Random instead of fixed effect model, or vv
What to use to report systematic reviews (sr)
The question on the page originate from the summary of the following study material:
- A unique study and practice tool
- Never study anything twice again
- Get the grades you hope for
- 100% sure, 100% understanding
![Trustpilot Logo](/images/design_2020/lp/clp-trustpilot-en.png)