Advanced ANOVA/Factorial ANOVA
Factorial ANOVA involves testing of differences between group means based on two or more categorical independent variables (IVs), with a single, continuous dependent variable (DV). In other words, a factorial ANOVA could involve:
- Two or more between-subjects categorical/ordinal IVs
- One interval or ratio DV
The results of interest are:
- Main effect for IV1
- Main effect for IV2
- Interaction between IV1 and IV2
If significant effects are found and more than two levels of an IV are involved, then provide follow-up tests, which could be either:
- A priori (planned) contrasts
- Post-hoc contrasts
(Note: Use appropriate control of the family-wise Type I error rate)
Effect sizes should also be reported. Eta-squared provides an estimate of the percentage of variance in the DV explained by each main effect and interaction effect. Cohen's p provides an estimate of the size of differences between two groups in standard deviation groups.
"What is the effect of Gender (2) and Degree Type (3) on Overall Student Satisfaction?" This could be described as a 2 (Gender) by 3 (Degree Type) factorial ANOVA.
- Establish hypothesis/hypotheses
- Make sure you have separate hypotheses for:
- Main effect for each IV
- Interactions between IVs
- Planned contrasts (if warranted)
- Make sure you have separate hypotheses for:
- Examine assumptions:
- IVs (categorical; between-subjects) and DV (at least interval)
- The data in each cell is normally distributed
- Homogeneity of variance (the variance in each cell is similar)
- Cells are independent
- Examine descriptive statistics, particularly the four moments (M, SD, Skewness, Kurtosis) overall, and also for each cell
- Examine graphs
- Conduct inferential test (ANOVA) and interpret significance of F scores
- Conduct follow-up tests (planned contrasts or post-hoc tests) if F is significant
- Interpret interactions
- Calculate and interpret effect sizes
- Eta-square (omnibus - equivalent to R2)
- Standardised mean effect size (difference b/w two means) e.g., Cohen's d
Example SPSS outputs
- Factorial ANOVA (example) - Are there differences in University Student Satisfaction levels between Gender and Age?
- Factorial ANOVA (example) - Are there differences in University Student Satisfaction levels between Gender and Age? - Are there differences in Locus of Control between Gender and Age?
- A table of descriptive statistics (M, SD, Skewness, and Kurtosis) for each cell and for each marginal total, and grand total should be presented when reporting results.
- For a 2-way ANOVA, the descriptives table it is recommended to provide a breakdown of one IV in the columns and the other IV in the rows, such as illustrated in the following tables.
Note that each of the three columns on the right should be further split into five columns to allow reporting of M, SD, Skewness, and Kurtosis, n. So, the expanded descriptive tables layout could look like this: (note that rows and columns have been swapped around from the table above - this is somewhat arbitrary):
Here's an example of such an APA style table:
- One of the keys to understanding Factorial ANOVA is being able to intepret interactions.
- A recommended experiential exercise for learning about interactions is to fabricate a dataset which can be used to demonstrate factorial ANOVAs in which there are:
- No effects
- Main effect A, no main effect B, no interaction
- Main effect A, no main effect B, interaction
- No main effect A, main effect B, no interaction
- No main effect A, main effect B, interaction
- Main effect A, main effect B, no interaction
- Main effect A, main effect B, interaction
- Interaction, no main effects
A write-up checklist for a factorial ANOVA might include:
- What is the goal/purpose of the analysis?
- What is the design - i.e., what are the IVs and the DV?
- What are the relevant descriptive statistics for the cells and the marginal descriptives? (M, D, skewness, kurtosis, n)
- To what extent does the data meet the assumptions for ANOVA (e.g., independent observations (often assumed), normality, and homogeneity of variance)?
- What are the main and interaction effect statistics, including the direction, statistical significance, and size of effects (consider the merits of reporting eta-squared and/or a standardised mean difference statistic such as Cohen's d; see effect sizes)?
- Provide a figure - often a line graph is used.
- Are follow-up tests necessary and/or warranted? e.g., if there are planned contrasts and/or a significant effect with three or more levels, then appropriate follow-up tests should be conducted using a method for controlling the family-wise Type I error rate as appropriate.
- What are the conclusions?