Advanced ANOVA/ANCOVA

Resource type: this resource contains a tutorial or tutorial notes.

This tutorial teaches use of analysis of covariance (ANCOVA) techniques, with practical exercises based on using SPSS.

Overview

ANCOVA evaluates whether population means on the DV, adjusted for differences on the covariate(s) (or 'nuisance variables'), differ across the levels of the IVs. Thus, the question being tested is whether the adjusted group means vary significantly from each other.
ANCOVA is exactly like ANOVA, except the effects of a third variable are statistically “controlled out” (similar to use of hierarchical multiple linear regression).
Any number of IVs and CVs can be used to create one-way, two-way, and multivariate ANCOVA designs.

Covariates

Typically included to remove extraneous influences from the DV, thus decreasing the within-group variance. Particularly with small sample sizes, well-chosen and reliably measured CVs can markedly improve the power of inferential statistical tests.
Including covariates is appropriate in order to:
1. Eliminate some systematic variance outside the control of the researcher that can bias the results.
2. Account for differences in response due to unique characteristics of the respondents.
This is usually achieved in experimental designs by random assignment to groups, however, in quasi-experimental designs problems related to non-random assignment can be minimised by statistically controlling for the effects of covariates.
Try to minimise the number of CVs; too many covariates will reduce the statistical efficiency of the analysis - rule of thumb is that the number of CVs < (.10 x sample size) - (number of groups - 1).

Example

If you are interested in testing the effect of computer experience on the attitude towards use of internet shopping, and you suspect that those with more positive attitudes toward shopping in general are more likely to have positive attitudes towards internet shopping, you may include attitude toward shopping as a covariate so as to remove its influence from the attitude towards internet shopping measure.

Assumptions

Assumptions to be met are those for ANOVA, plus:

Covariates must be linearly related to the DV. The stronger the correlation, the more useful the CV will be. If there is no correlation, then the inclusion of the CV will slightly weaken the power of the test by needlessly consuming a degree of freedom.
Covariates must have a homogeneous effect on the DV across the IV groups (i.e., homogeneity of slopes or equal effects on the DV across the IV groups). If there is a significant interaction between the covariate and the IV, do not use an ANCOVA.
The covariate should be unrelated to the IV.
Covariates should not be overly correlated with one another.

The first two criteria can be checked via a scatterplot of the DV and the CV, with the IV as a control variable (to check for equal slopes).

The third criteria depends on experimental design (e.g., if the CV is measured prior to the IV, then it cannot be affected by the IV).

The fourth criteria can be checked via correlations and scatterplots between the CVs.

Recall that the main ANOVA assumptions are that:

Each of the observations are independent
The DV (and the CV) must be interval level of measurement
The underlying populations (of adjusted scores) must be normally distributed
Each of the underlying populations (of adjusted scores) must have the same variance

The first two assumptions are a function of experimental design. The third and fourth assumptions are more difficult to test because we do not have the "adjusted scores", so we cannot compare the variances of the adjusted scores across the IV groups, but SPSS provides a Levene's test for the homogeneity of variance for the adjusted scores. Fortunately, provided the samples are sufficiently large, the test is robust to violations of the normality assumption.

Example write-up

A one-way analysis of covariance (ANCOVA) was conducted. The independent variable, vitamin C, involved three levels: placebo, low dose, and high dose. The dependent variable was the number of days with cold symptoms during treatment and the covariate was the number of days with cold symptoms before treatment. The assumptions for ANCOVA were met. In particular, the homogeneity of the regression effect was evident for the covariate, and the covariate was linearly related to the dependent measure.

The ANCOVA was significant, F (2,26) = 6.45, p = .005. The strength of the relationship between vitamin C treatment and the dependent variable was very strong, as assessed by $\eta$ $_{p}^{2}$ , with the vitamin C factor accounting for 33 percent of the variance in dependent measure holding constant the number of days with pretreatment cold symptoms. The mean number of days with cold symptoms adjusted for initial differences were ordered as expected across the three vitamin C groups. The placebo group had the largest adjusted mean (M = 12.01), the low dose vitamin C group had a smaller adjusted mean (M = 7.71) and the high dose vitamin C group had the smallest adjusted mean (M = 6.67). Follow-up tests were conducted to evaluate pairwise differences among the adjusted means. The Holm’s sequential Bonferroni procedure was used to control for Type I error across the three pairwise comparisons. There were significant differences in the adjusted means between both groups that received vitamin C and the placebo, but no significant difference between the two vitamin C groups. [Standardised mean effect sizes to be added] e.g., effect size with covariate calculator

Descriptives

In presenting ANCOVA results, provide a table of means for each of the groups. If the same scale is used to measure the DV and the CV, then the unadjusted group means and SD (from Descriptives) can be presented. If a different scale is used for the DV and CV, then provide both the unadjusted mean (and SD) and the adjusted mean (and SE). The adjusted mean (controlling for the CV) is provided in the estimated marginal means table.

Exercises

Teaching method

In a hypothetical educational psychology experiment, participants were randomly divided into two groups. One group was taught conventionally, and the other were taught using an innovative method. Prior to allocation to the groups, learning motivation was assessed. Improvements in academic achievement were measured as the DV.

Motiv.sav (p. 129; Section 5.2)
- Achievement (achieve) (DV)
- Teaching Method (teach) (IV - 2 levels - "conservative" and "innovative")
- Motivation (motiv) (CV)

Positive effect

Conduct an ANCOVA to test for differences in positive effect between males and females, adjusting for differences in age.

Psychol.sav
- Positive effect (DV)
- Gender (IV)
- Age (CV)

Vitamin C

Do low and high doses of Vitamin C reduce incidences of days suffered with a cold? Conduct an ANOVA using:

ANCOVA.sav
- Group (IV; Vitamin C: placebo, low, high)
- Days (DV; days with a cold)
- Pre-days (CV; previous days with a cold)

Therapy and depression

What is the effect of three different therapy types on depression, taking into account pre-existing depression levels? Use:

Ex ANCOVA.sav
- Group (IV; Counseling and journal therapy; Journal therapy only; Counseling only)
- Depression prior (CV)
- Depression after (DV)