Meta-analysis is a systematic technique for reviewing, analysing, and summarising quantitative research studies on specific topics or questions.
This page provides information and resources about how to conduct a meta-analysis. The target audience includes post-graduate students conducting a meta-analysis or beginning researchers in meta-analysis. These pages could also be used by students involved in research methods coursework.
- A meta-analysis is quantitative technique for conducting a "study of studies".
- Use of meta-analysis has flourished, particularly in the social, health, and medical sciences, since it was developed in the 1970s
- Meta-analysis was initially developed in response to controversy over traditional, subjective literature review methods (specifically, at the time, those used in to review the psychotherapy outcome studies).
- Practical meta-analysis (Lecture slides; Wilson, 1999)
- How to do meta-analysis (Lecture slides; Basu, 2005)
- Meta-analysis: combining information(Lecture Slides; LeBauer, 2010)
How to do a meta-analysis
- Meta-analysis involves analysing the summary data from many studies. It can be performed by hand, using a spreadsheet and formulae, using scripts, syntax or macros with generic statistics software packages, or by using dedicated meta-analysis software packages.
- Before starting, identify a clear question(s), e.g., "What are the outcomes of psychotherapy?"
- Questions can also involve the effect of independent variables, e.g., "Are the outcomes of psychotherapy similar for males and females?"
- Read other related meta-analyses to get a feel for the kinds of questions asked.
- Make sure that any independent variables (IVs) and dependent variables (DVs) are very clearly defined.
- Because of the importance of establishing a well-defined question and variables, developing a peer-reviewed proposal for a meta-analytic study is strongly recommended.
- It can be helpful to identify several similar or related meta-analytic studies as models for your meta-analytic study. Consider the strengths and weaknesses of their methodologies.
- Establish clear criteria for selection of studies, e.g., does it need to be published in a peer-reviewed journal, or will you also accept theses and non-peer reviewed papers (e.g., conference papers)?
- Conduct an exhaustive and systematic literature search, recording your steps along the way (important for the Method - must allow replication)
- Create a "coding sheet" - this is this list of fields (variables) you want to extract from each study, and how each of the variables are to be coded - get this peer-reviewed, otherwise you will limit the potential/quality of your analyses
- Enter the data - one study per row, but note that there may be multiple outcomes and/or groups of interest for each study, in which case each of these will receive their own row in the database, with a column to code which type of outcome was measured.
- Analyse the data using spreadsheet formulae, or by writing syntax commands for a generic statistics package, or by using a dedicated meta-analysis software package (with in-built meta-analysis tools).
- Central to understanding meta-analysis is an understanding of effect sizes.
- The chief value of effect sizes in the context of meta-analysis is that they provide a way to standardise effects across studies using different measures, allowing for common analysis.
- There are many possible effect sizes, but essentially there are two commonly reported types in meta-analysis:
- Correlational: e.g., r (product-moment correlation)
- Mean differences: e.g., Cohen's d, Hedge's g, etc.
- An important limitation of meta-analysis is that its results can only be as good as the original data is valid.
- Meta-analysis can only analyse the role of independent variables in explaining variance in dependent variables if sufficient data is provided in the original studies.
- "Apples and oranges" effect - i.e., there is a risk/tendency in meta-analysis to average/mash together disparate effects.
- Can lack in qualitative insight (e.g., as may be more likely to be contributed by an expert conducting a traditional literature review).
Example meta-analytic studies
- Hattie, J., Biggs, J., Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66, 99-136.
- Hattie, J., Marsh, H. W., Neill, J. T., & Richards, G. E. (1997). Adventure education and Outward Bound: Out-of-class experiences that make a lasting difference. Review of Educational Research, 67, 43-87.
- Purdie, N., Hattie, J., Carroll, A. (2002). A review of the research on interventions for attention deficit hyperactivity disorder: What works best? Review of Educational Research, 77, 61-99.
Some dedicated meta-analysis software includes:
|Name||URL||License||$Cost||Trial or Demo?||Version||Notes|
|MetaXL||http://www.epigear.com||Open||Free||No||2.0||Can run bias adjusted meta-analyses and the IVhet model in addition to conventional models|
|MIX||http://www.meta-analysis-made-easy.com||Proprietary||0-210||Yes||2.0||Student and academic licenses available|
|RevMan||http://www.cc-ims.net/RevMan||?||Free for non-commercial use||Yes||5||For organising reviews; for MA, see |
Non-dedicated, generic statistics software which can be used for conducting meta-analysis include:
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Sage. Thousand Oaks, CA.
Learn more about Meta-analysis