User:Jtneill/Presentations/More complex summary statistics

From Wikiversity
Jump to navigation Jump to search
More complex summary statistics: Effect sizes

Dr. James Neill, Assistant Professor, Centre for Applied Psychology, Faculty of Health, University of Canberra

Statistics Networking Day, 6 August, 2015, UC

Statistics in psychology[edit]

Statistics-related Units in Psychology at the University of Canberra
Year Unit Topics Enrolment
1st Introduction to Psychological Research Research design, univariate descriptive statistics, bivariate linear correlations, t-tests ~200-250
2nd Experimental Psychology Experimental design, ANOVAs, non-parametric tests of differences ~150
3rd Survey Research and Design in Psychology Survey design, correlations, exploratory factor analysis, multiple linear regression ~140
4th Research Methods and Professional Ethics Advanced multivariate statistics (ANOVA and MLR) ~25-30

What's your favourite complex summary statistic?[edit]

  1. What would you choose?
  2. For me the choice is easy - although I wasn't taught how to use effect sizes in undergraduate psychology, it is the statistic that I use the most and find most useful.

My favourite: Effect sizes[edit]

  1. Social sciences have tended to over-emphasise null hypothesis significance testing and under-emphasise use of effect sizes:

"I believe that the almost universal reliance on merely refuting the null hypothesis as the standard method for corroborating substantive theories in the soft areas is a terrible mistake, is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology"
(Meehl, 1978, p. 817)

  1. Statistical significance is a function of effect size, sample size, and probability level - but most often, people want to know about the effect size (i.e., "how strong is the relationship or how big is the difference?).
  2. Effect sizes can be re-expressed as other effect sizes or other common language formats such as percentages.

Types of effect sizes[edit]

Types of effect sizes
Type Correlations Means
Bivariate r d
Multivariate R

Graphing effect sizes[edit]

Figure 1 (Stem-and-leaf diagram of all effect sizes for the adventure programs) from Hattie et al. (1997).

Confidence intervals[edit]

  1. Accompanying ESs with CIs offers the best of both worlds - i.e., indicates the size of an observed effect and uncertainty
Figure 7.5 (Error-bar graphs of T1 to T2 LEQ Factor ESns and CIs (N = 3,640)) from Neill (2008).

Take-home message[edit]

  1. In applied social science research, people generally want to know about the strength of relationship or the size of the difference - so, report effect sizes. Don't rely solely on inferential statistical testing.
  2. Graph effect sizes and include confidence intervals.


  1. Hattie, J., Marsh, H. W., Neill, J. T., & Richards, G. E. (1997). Adventure education and Outward Bound: Out-of-class experiences that make a lasting difference. Review of Educational Research, 67, 43-87.
  2. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806-834.
  3. Neill, J. T. (2008). Enhancing life effectiveness: The impacts of outdoor education programs. Unpublished doctoral dissertation, Faculty of Education, University of Western Sydney, NSW, Australia.