Sport research/Research design

Think about and carefully plan your study to get the best results

Your research question will largely determine how you will go about conducting the research. The broad methodology you adopt to answer the question you have is often referred to your research design. Initially you need to determine if you want to answer a question relating to a single case (a case study) or if you want to know what happens in general (i.e. you study a sample which relates to a larger popultion). Most reported research in exercise and sport is in the form of a sample and that's what will be considered here.

The type of research you conduct can be grouped into

• Descriptive research
• Experimental and Quasi experimental
• Meta analyses

Hopkin's refers to this as modes of enquiry. Descriptive and experimental research are the two most common types of research that we deal with.

As part of your research design, it will also be important to consider the validity and reliability of data that you are collecting. This may have implications on parts of your research design. Your research design will also determine what kind of data analysis you can do. You should know what data analysis you are going to do before you collect your data.

Descriptive research

Descriptive research occurs when data is collected in a natural setting (i.e. the researcher has not manipulated the environment in any way). The natural setting doesn't mean a green forest, it just means the researcher does not interfere. So videoing and coding a game of rugby, then going on and describing what happens would be referred to as a descriptive, or observational, study. Although the data description is factual, accurate and systematic, the research cannot describe what caused a situation. Thus, descriptive research cannot be used to create a causal relationship, where one variable affects another. Qualitative research often has the aim of description and researchers may follow-up with examinations of why the observations exist and what the implications of the findings are.

Descriptive, or observational research can be used to describe a single point in time, or changes over time (longitudinal - e.g. changes in strength over weeks, changes in blood glucose during a single exercise bout, the evolving role of a sports manager over the decades). Observational research also often incorporates correlations, where different observations are correlated against each other to see if they are associated.

Hopkin's^[1] describes the types of observational study with a sample, weak to strong:

• Case series, e.g. 20 gold medallists.
• Cross-sectional (correlational), e.g. a sample of 1000 athletes.
• Case-control (retrospective), e.g. 200 Olympians and 800 non-Olympians.
• Cohort (prospective or longitudinal), e.g. measure characteristics of 1000 athletes then determine incidence of Olympic medals after 10 years.

Experimental and Quasi experimental

In an experimental research design, you almost always gather information before and after an intervention (where you, the experimenter, do something to effect the natural world) and then look for changes. The ways in which interventions can be incorporated into a study have been described by Hopkin's^[1], from weak to strong (in terms of study design):

• No control group (time series), e.g. measure performance in 10 athletes before and after a training intervention.
• Crossover, e.g. give 5 athletes a drug and another 5 athletes a placebo, measure performance; wait a while to wash out the treatments, then cross over the treatments and measure again.
• Controlled trial, e.g. measure performance of 20 athletes before and after a drug and another 20 before and after a placebo.

You need up to 4x as many subjects as in a crossover.

In experimental research, it is also important to limit the bias associated with your measures. Bias^[1] is less likely if…

• Subjects are randomly assigned to treatments. (in research where subjects are not randomly assigned, the research is termed quasi-experimental design)
• Assignment is balanced in respect of any characteristics that might affect the outcome.

In other words, you want treatment groups to be similar.

• Subjects and researchers are blind to the identity of the active and control (placebo) treatments.

Single blind = subjects don't know which is which.

Double blind = the researchers administering the treatments and doing the measurements and analysis don't know either.

If you have a controlled trial, then here is a useful decision tree for Controlled Trials

RCTs

If you are doing a randomised controlled trial (an RCT) you should consider the CONSORT statement. The CONSORT statements provide established guidelines on reporting (and by extension designing) a randomised controlled trial - one of the highest levels of designs in terms of evidence that it creates). The statement will ask you to report things like how participants were recruited, how random allocation to groups occurred, how participants and researchers were blinded to interventions... all those kinds of things that need to be considered when designing and reporting on a well-controlled study. Improved reporting of interventions, to allow their replication, is also the aim of the TIDieR checklist and guide. Specific guidelines to determine the quality of evidence (and thus what you should think about in your own research design) of a study include GRADE and the Physiotherapy Evidence Database (PEDro) scale. Student projects often end up with small participants numbers, often underpowered - for these studies it is worth considering the CONSORT pilot and feasibility trials extension. This research is well placed after all to help inform larger subsequent trials, as is often the target of grant applications. You may as well design these experiments well, make sure they really are suitable to inform later trials. So for any pilot or feasibility study (often including student projects) consider designing your study with these guidelines.

Exercise Research

Exercise research also now has a reporting template, in the form of a 16-item internationally endorsed Consensus on Exercise Reporting Template (CERT). Similar to other statements like PRISMA, CONSORT, CERT aims to provide guidance on standardised and complete reporting (and design) of research that incorporates exercise interventions. Although only developed in 2016, CERT will provide a strong guide to authors (and reviewers) in the future. The actual template is here.

Pre-registration

Prompted by findings in social psychology in particular, in which only about half the findings in the field could be reproduced in replication studies (the replication crisis), a growing part of the scientific community has discussed how scientific research can be improved. One belief, is that along with the traditional ideas around what constitutes "gold standard" experimental research - randomized controlled trial, double-blinding type ideas - research should also be preregistered. Preregistration can occur in different ways - registration of clinical trials, publication of study protocols, or even publishers having a process by which study protocols are presubmitted to journals, and results of the study (regardless of what they are) being pre-accepted for publication. This process could potentially reduce publication bias (where significant results appear to get priority), inappropriate manipulation of data or statistical analysis, and improve study design. Some critics of this approach believe this may stifle creativity, but this approach does not have to eliminate extra exploratory analysis, nor valid design. No doubt this area will influence and shape scientific research and publication for a few years to come.

Meta analyses

A meta-analysis combines the results of several studies that address a set of related research hypotheses. In its simplest form, this is normally by identification of a common measure of effect size, for which a weighted average might be the output of a meta-analyses. Here the weighting might be related to sample sizes within the individual studies. More generally there are other differences between the studies that need to be allowed for, but the general aim of a meta-analysis is to more powerfully estimate the true "effect size" as opposed to a smaller "effect size" derived in a single study under a given single set of assumptions and conditions.

Meta-analyses are often, but not always, important components of a systematic review procedure. Here it is convenient to follow the terminology used by the Cochrane Collaboration, and use "meta-analysis" to refer to statistical methods of combining evidence, leaving other aspects of 'research synthesis' or 'evidence synthesis', such as combining information from qualitative studies, for the more general context of systematic reviews. The PRISMA guidelines are an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses also worth consulting during the design and write-up of any meta-analysis.

Hopkin's also has an introduction to meta-analysis worth reading.

Things you (may) need to think about

Power and subject numbers

Bland M (2009) Sample size calculations should be based on the width of a confidence interval, not power (BMJ article)

http://www.sportsci.org/resource/stats/relyappl.html#samplesize

Hopkins WG (2006) Estimating Sample Size for Magnitude-Based Inferences Sportscience 10, 63-70.

Online effect size calculator from University of Colorado academic.

Random allocation of subjects

Hopkins WG (2010) Assigning Subjects to Groups in a Controlled Trial. Sportscience 14, 7-12.

Activity

Activities are mini-tasks that will give you some practice with the concepts of each section. Activities should appear here soon, if not, feel free to add some open access ones yourself.

Task

1. What type of research are you conducting?
2. Explain why you have answered as you have to Q1.
3. Could you do a different type of research on the same thing? What would you need to change?
4. Explain how the type of research you are conducting limits you or allows you to make inferences about causality

Resources

In "research designs" Will Hopkin's provides a powerpoint overview of different research designs often used in sports research. This is an excellent resource.

For quantitive research design, Hopkins has another useful resource that will help you thinak about all apects of your research design.

See Also

Meta analysis on Wikipedia.
Experiment on Wikipedia.
Descriptive research on Wikipedia.

National Statistical Service website offers a series of chapters intending to provide an understanding of the issues involved in survey design. It provides the key issues to be considered when designing surveys and potential survey designs and covers the advantages and disadvantages of the various methods. The site can be accessed from http://www.nss.gov.au/nss/home.nsf/2c4c8bd01df32224ca257134001ea79a/e4e968fc83fb1217ca2571ab00247b3e?OpenDocument