Survey research and design in psychology/Assessment/Lab report/FAQ

Lab report - FAQ

Do I have to analyse all the data?

Q: Do I have to analyse and write about all the data - or only the data I'm analysing? (e.g., Do I have to look at time management if all my hypotheses are based around university student satisfaction?)
A: Only analyse data relevant to the focus you choose for the study. You do not need to analyse data for all the variables. In the report, only discuss data you analyse - there's no need to talk about other parts of the survey except a brief mention in the Method - Materials - that the measures of interest were part of a larger survey.

Is it OK if my lab report is similar to the example write ups?

Q: Is it OK if my lab report is similar to the example write ups?>
A: Marking aims to reflect the level of demonstrated independent understanding. Thus, it is in your interests to provide a marker with as much evidence about your understanding by writing independently as possible. If your report reflects the examples too closely, it won't provide much evidence of independent understanding and writing. Therefore, it is recommended to use the example write-ups as guides, but to avoid simply copying the example write-ups and replacing relatively minor parts with your specific details. Submitted lab reports will be compared with the example write-ups and if the similarities are too close, the report may be considered to have been plagiarised and be dealt with as per the UC Student Academic Integrity Policy.

Can uni student satisfaction (as a whole construct) be used as a variable?

Q: Can uni student satisfaction (or time management) (as a whole construct) be used as a variable (e.g., as an IV or DV in an MLR)?>
A: Short answer: Preferably, no. Longer answer: An exploratory factor analysis should be conducted of all the university student satisfaction items (and/or all the time management items) to determine the most appropriate simplification of the variables (i.e, to explain as much variance as possible using as few factors as possible). If this EFA identified more than one factor, then this suggests that the best model is not unidimensional (one factor), but rather multidimensional (more than one factor), so more than composite score should be created. Unless the factors are closely correlated, it doesn't make sense to combine all university student satisfaction variables or all time management variables into a single factor.

How can general health and well-being or general life satisfaction items be used?

Q: How can general health and well-being or general life satisfaction items be used? (e.g., does a factor analysis need to be conduced or can individual items be used in an MLR)?
A: A factor analysis of these sets of items could be conducted and reported very briefly. Alternatively, the items could be grouped into those about "physical health" and those about "mental health" (or those about "life satisfaction), and internal consistency analyses could be conducted for each of the sets of items to help determine/confirm which items to keep or drop. Then create composite "physical health" and/or "mental health" or "life satisfaction" scores. This approach is probably preferably to using individual items (as the composite measures will be more rounded, with more measurement points). Alternatively, responses to single items could be used.

How can I include the correlation matrix for an EFA in an Appendix?

Q: How can I include the correlation matrix for an EFA in an Appendix?
A: As the correlation matrix for the final EFA is large, present it in an Appendix. The Appendix does not need to be in APA style. Some ways this could be done:

Export or copy the correlation matrix from SPSS output into a word processing document (e.g., r. click on the table in SPSS output, then Copy Special - Image - and Paste)
Retype the correlations into a word processor table (just include correlations above or below the diagonal - there is no need to indicate statistical significance)
Paste a screenshot of the SPSS output for the correlations

What sample size should I report?

Q: What sample size should I report?
A: Here are some suggestions for reporting the sample size:

Abstract and Method (Participants) - Report the final sample size after data screening
Results
1. Data screening - Report the initial sample size, steps/decisions taken to remove any cases, and the final sample size
2. Analyses - where Ns vary between analyses, report N separately for each analysis.

Which items should be used for internal consistency?

Q: Which items should be used for internal consistency?
A: Calculate internal consistency using the variables which make up each factor, then repeat for each factor. Do not put all the variables from a multi-factorial measure (e.g., for time management) into a single reliability analysis.

Does it matter if the results aren't significant?

Q: Does it matter if the results aren't significant
A: Whether or not results are statistically significant is irrelevant in terms of the marking criteria for the lab report. Take care to disconnect your ego and emotions from whether or not results are significant. This is a common problem and challenge for emerging academics. It is just as important to know that a study did not find a likely relationship between variables and is it to know that it did find a likely relationship. It is important to approach the report objectively, as a scientist.

Why are some regression coefficients negative when the correlations are positive?

Q: In MLR, what are some regression coefficients negative when the correlations are positive?
A: Drawing a Venn diagram to help make sense of the set of relationships amongst the variables in this MLR. IV1 and the DV have have a positive correlation - most likely this is small. IV1 must also be correlated with one or more other IVs (i.e., there is some collinearity). And one or more of those other IVs are correlated with the DV. So, the MLR analysis finds that one or more of the other IVs does a better job of predicting the DV. And because the predictors are correlated, IV1 really doesn't explain any unique variance. Its regression coefficients will be vary small and non-significant. In this scenario, it is statistical artifact/anomly that the sign may be negative. Its not the sign that is important here - the size and significance are far more relevant. Basically, IV1 explains no variance in the DV when the variance explained by the other IVs is taken into account.

What if I can't find the DOI for a reference?

Q: Is it necessary to include DOI?
A: Yes, if it is available - check:

What if the URKUND score is high?

Q: What if the URKUND score is high?
A: Some degree of matching between the submitted report and the URKUND database is expected (e.g., cover sheet, item wording from the

Surveys

These surveys were designed for use by an undergraduate psychology class (Survey Research and Design in Psychology, 2005-2018):

Time and Stress Questionnaire for University Students v.1 (TSQFUS1)
The University Student Satisfaction and Time Management Questionnaire v.9 (TUSSTMQ9)
The University Student Motivation & Satisfaction Questionnaire v.2 (TUSMSQ2)

Students used these surveys to collect data, entry data, and conduct analyses for a lab report.

Using these surveys

These instruments and their items are free to use, adapt etcetera under a Creative Commons Attribution International 4.0 license.

However, be aware that the surveys in their current format are intentionally designed to not be "perfect" so that emerging scholars studying subjects such as "Survey research and design in psychology" can collect data and then practice exploratory factor analysis .

There is also intentionally no scoring key . Factor analysis is recommended to help determine the underlying factor structure and to identify which items to use to calculate composite scores. In other words, there is a latent structure, but you'll need to work it out. For example, for university student motivation, see these suggestions. Composite scores representing underlying constructs can then be used for descriptive statistics and hypothesis testing.

Psychometrics

There are no reported psychometrics for newly developed items and scales in these survey instruments. Where intact, previously published measures were included, psychometrics may be available.

Users of these surveys should be prepared to conduct their own psychometric analyses (factor structure, reliability, and validity) based on their own samples.

Can I delete and resubmit?

Q: Can I delete and resubmit?
A: Until you've submitted and confirmed, you can delete your attachment and then upload a new version. However, if you do this after the due date, late penalties will apply.

When will lab report marking be returned?

Q: When will lab reports be marked and returned by?
A: For reports submitted by the original due date, we will endeavour to provide marks and feedback within three weeks of submission. Marking of reports submitted late or with extension may take longer than three weeks from the date of submission.

How will lab report marks and feedback be returned?

Q: How will lab reports marks and feedback be returned?
A: Electronically - check your uni email and/or Moodle for notification