Reliability and validity
Pre-requisites
[edit | edit source]Before reading this section, make sure you understand:
as separate concepts.
Relationship between reliability and validity
[edit | edit source]Reliability and validity are important concepts within psychometrics.
Reliability is generally thought to be necessary for validity, but it does not guarantee validity.
Reliability and validity are, conceptually, quite distinct and there need not be any necessary relationship between the two. Be wary of statements which imply that a valid test or measure has to be reliable. Where the measurement emphasis is on relatively stable and enduring characteristics of people (e.g. their creativity), a measure should be consistent over time (reliable). It also ought to distinguish between inventors and the rest of us if it is a valid measure of creativity. A measure of a characteristic which varies quite rapidly over time will not be reliable over time - if it is then we might doubt its validity. For example, a valid measure of suicide intention may not be particularly stable (reliable) over time though good at identifying those at risk of suicide.
Validity is often expressed as a correlation between the measure and some criterion. This validity coefficient will be limited or attenuated by the reliability of the test or measure. Thus, the maximum correlation of the test of measure with any other variable has an upper limit determined by the internal reliability. [1]
Within classical test theory, predictive or concurrent validity (correlation between the predictor and the predicted) cannot exceed the square root of the correlation between two versions of the same measure — that is, reliability limits validity.
With this in mind, it can be helpful to conceptualize the following four basic scenarios for the relation between reliability and validity:
- Reliable (consistent) and valid (measures what it's meant to measure, i.e., a stable construct)
- Reliable (consistent) and not valid (measures something consistently, but it doesn't measure what its meant to measure)
- Unreliable (not consistent) and not valid (inconsistent measure which doesn't measure what its meant to measure)
- Unreliable (not consistent) and valid (measures what its meant to measure, i.e., an unstable construct)
It is important to distinguish between internal reliability and test-retest reliability. A measure of a fluctuating phenomenon such as suicide intention may be valid but have low test-retest reliability (depending on how much the phenomenon fluctuates and how far apart the test and retest is), but the measure should exhibit good internal consistency on each occasion.
Data analysis exercises
[edit | edit source]See also
[edit | edit source]References
[edit | edit source]Bibliography
[edit | edit source]Howitt, D. & Cramer, D. (2005). Reliability and validity: Evaluating the value of tests and measures (Ch. 13). In Introduction to research methods in psychology (pp. 218-231). Harlow, Essex: Pearson.
External links
[edit | edit source]- Summarizing data: Precision of measurement (A new view of statistics, Will Hopkins)
- Reliability & validity (Lecture slides; slideshare.net)