From Wikiversity
Meta-analysis is a systematic technique for reviewing, analysing, and summarising quantitative research studies on specific topics or questions. The purpose of this page is gather information and resources about how to conduct a meta-analysis. Thus, the target audience includes, for example, post-graduate students conducting a meta-analysis or beginning researchers interesting in conducting a meta-analysis. This page could also be useful for students involved in research method coursework which includes a section on understanding the use and application of meta-analysis.
[edit] Introduction
- A meta-analysis can be thought of as a "study of studies".
- Use of the technique has flourished, particularly in the social, health, and medical sciences since it was developed in the 1970's in response to controversy over traditional, subjective literature review methods (specifically, at the time, those used in to review the psychotherapy outcome studies).
[edit] Lecture slides
- Practical meta-analysis (Lecture slides; Wilson, 1999)
- How to do meta-analysis (Lecture slides; Basu, 2005)
[edit] How to do a meta-analysis
- Meta-analysis involves analysing the summary data from many studies. It can be performed by hand, using a spreadsheet and formulae, using scripts, syntax or macros with generic statistics software packages, or by using dedicated meta-analysis software packages.
- Before starting, identify a clear question(s), e.g., "What are the outcomes of psychotherapy?"
- Questions can also involve the effect of independent variables, e.g., "Are the outcomes of psychotherapy similar for males and females?"
- Read other related meta-analyses to get a feel for the kinds of questions asked.
- Make sure that any independent variables (IVs) and dependent variables (DVs) are very clearly defined.
- Because of the importance of establishing a well-defined question and variables, developing a peer-reviewed proposal for a meta-analytic study is strongly recommended.
- Establish clear criteria for selection of studies, e.g., does it need to be published in a peer-reviewed journal, or will you also accept theses and non-peer reviewed papers (e.g., conference papers)?
- Conduct an exhaustive and systematic literature search, recording your steps along the way (important for the Method - must allow replication)
- Create a "coding sheet" - this is this list of fields (variables) you want to extract from each study, and how each of the variables are to be coded - get this peer-reviewed, otherwise you will limit the potential/quality of your analyses
- Enter the data - one study per row, but note that there may be multiple outcomes and/or groups of interest for each study, in which case each of these will receive their own row in the database, with a column to code which type of outcome was measured.
- Analyse the data using spreadsheet formulae, or by writing syntax commands for a generic statistics package, or by using a dedicated meta-analysis software package (with in-built meta-analysis tools).
[edit] Effect sizes
- Central to understanding meta-analysis is an understanding of effect sizes.
- The chief value of effect sizes in the context of meta-analysis is that they provide a way to standardise effects across studies using different measures, allowing for common analysis.
- There many possible effect sizes, but essentially there are two commonly reported types in meta-analysis:
- Correlational: e.g., r (product-moment correlation)
- Mean differences: e.g., Cohen's d, Hedge's g, etc.
[edit] Limitations
- An important limitation of meta-analysis is that its results can only be as good as the original data is valid.
- Meta-analysis can only analyse the role of independent variables in explaining variance in dependent variables if sufficient data is provided in the original studies.
- "Apples and oranges" effect - i.e., there is a risk/tendency in meta-analysis to average/mash together disparate effects.
- Can lack in qualitative insight (e.g., as may be more likely to be contributed by an expert conducting a traditional literature review).
[edit] Example meta-analytic studies
- Hattie, J., Biggs, J., Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66, 99-136.
- Hattie, J., Marsh, H. W., Neill, J. T., & Richards, G. E. (1997). Adventure education and Outward Bound: Out-of-class experiences that make a lasting difference. Review of Educational Research, 67, 43-87.
- Purdie, N., Hattie, J., Carroll, A. (2002). A review of the research on interventions for attention deficit hyperactivity disorder: What works best? Review of Educational Research, 77, 61-99.
[edit] Software
[edit] Comparison table
Some dedicated meta-analysis software includes:
[edit] Alternative (software
Non-dedicated, generic statistics software which can be used for conducting meta-analysis include:
[edit] Other comparisons/lists
- w:Meta-analysis#Software
- http://www.um.es/facpsi/metaanalysis/software.php
- http://www.lehanathabane.com/personal/metalinks.htm
- http://www.med.umich.edu/csp/Course%20materials/Fall%202005/Rogers_Meta%20Analysis%20software%20packages.pdf
[edit] References
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Sage. Thousand Oaks, CA.
[edit] See also
[edit] External links
