Sport research/Validity and reliability of data

Your data will only be useful if it is (or you have a concept of how it is) valid and reliable.

Validity

Validity refers to the agreement between the value of a measurement and its true value. You quantify validity by comparing your measurements with values that are as close to the true values as possible, often referred to as a "gold standard".

If you don't measure these things yourself, you should at the very least make a case for how valid the measures that you are taking generally are from evidence presented in the literature.

Reliability

Reliability refers to the reproducibility of a measure if you were to repreat the measure. It's like asking: If I took the measure again, without doing anything that is likely to change the measure (e.g. give them a drink when assessing hydration status, waiting a year to measure body mass), what is the likelihood that I would get the same result? It is obviously important that repeat measures are fairly close. Of course the results aren't exactly the same, there is error associated with the measurement. The error can come from biological error (e.g. the body is constantly changing and providing different results) or technical error (e.g. error associated with the instruments you use to take a measure and the skills of the researcher in taking the measure). It is the technical error in particular that we aim to minimse, but it is important to have an idea of the total error associated with a particular measurement. The amount of error will ultimately influence whether or not we observe differences between groups, or if the differences are too small to distinguish from the typical error (or noise) that we record.

If you don't measure these things yourself, you should at the very least make a case for how reliable the measures that you are taking generally are from evidence presented in the literature.

There are different type of reliability, but the most common is retest reliability - the reproducibility of values of a variable when you measure the same subjects twice or more.

Components of retest reliability

change in the mean
typical error
retest correlation

Other measures of reliability

kappa coefficient - relibility on nominal variables often used in performance analysis of sport
alpha reliability - a reliability variable used for questionnaires often used in sport psychology

There is a lot that can be said about reliability and it's an important concept too. Hopkin's covers the topic extensively, you are best starting your reading off here http://www.sportsci.org/resource/stats/precision.html

Activity

Activities are mini-tasks that will give you some practice with the concepts of each section. Activities should appear here soon, if not, feel free to add some open access ones yourself.

Task

Prepare a 3 slides in 3 minute presentation on "how validity and reliability applies to your research and how it influences your study".

Resources

Will Hopkin's covers the issues of validity and reliability in the precision of measurement section of his website.
Reliability, a Crucial Issue for Clinicians and Researchers (slideshow) was a presentation delivered by Hopkins as part a mini-symposium entitled at the 2001 annual meeting of the American College of Sports Medicine. It covers the essentials of reliability and some of its uses (assessing individuals, estimating sample sizes, estimating individual responses).
The International Journal of Performance Analysis in Sport released a special issue on reliability worth looking at, particularly if this is your field.