Survey research and design in psychology/Assessment/Project/Lab report

Lab report guidelines

View the accompanying screencast: [1]

Overview

1. Present an independently-developed APA style survey research lab report which uses exploratory factor analysis and multiple linear regression to address research question(s) and hypotheses.
2. Use the provided data file, which contains data aggregated from the data collection and entry assessment exercise, based on the TUSSTMQ6 survey.
3. Consult the detailed marking criteria about what to include for each section.

General marking criteria

Within each of the lab report sections (Introduction, Method, Results, and Discussion), marking will reflect these general criteria:

1. 70%: Quality of execution of the task (e.g., review literature and develop hypotheses, describe method, describe results, discuss findings)
2. 10%: Quality of written expression
3. 10%: Contribution to a cohesive, meaningful report (i.e., a story that makes sense, the sections are not disjointed)
4. 10%: APA style: Some important aspects include:
   1. Times New Roman 12 pt font
   2. Page numbers
   3. Left-justify
   4. Captioning and layout of tables and figures
   5. Citations and referencing (including electronic referencing)
   6. Avoid specific reference to SPSS variable names (these are arbitrary)
   7. Single-spaced (for electronic submission; double-spacing is a relic from paper submission days to allow room for hand-written comments)

A summary of the marking criteria by section follows. The word count ranges per section are suggestions only. There is an upper limit on the overall word count - see word count:

Criteria	Description	%	Suggested word count
Introduction	Establishes the problem, reviews theory and research, develops research question(s) and hypotheses	10%	500-800
Method	Describes method and design, including Participants, Materials, and Procedure	15%	500-700
Results	Screens and analyses data using EFA, internal consistency, composite scores, and MLR	45%	900-1600
Discussion	Summarises and interprets the findings, considers implications and makes recommendations	30%	600-800

For more information, see the detailed marking criteria and sample write-ups.

Detailed marking criteria

Detailed requirements and suggestions are provided for each of the sections of the lab report:

Cover sheet

1. p. 1
2. Download the Coversheet, fill out, and insert at the start of the lab report.
3. Marked as part of the Introduction.

Title page

1. p. 2
2. Title (~10-15 words)
   1. Does the title accurately convey the content of the report?
   2. Is it succinct, specific, and possibly catchy/memorable?
3. Student name, ID
4. Unit number and name
5. Date
6. Marked as part of the Abstract.

Abstract

1. p. 3
2. ~160-200 words.
3. Summarise the purpose of the study and the research questions and/or hypotheses.
4. Summarise the method, avoiding excessive detail.
5. Summarise key results.
6. Summarise theoretical and methodological implications/conclusions.
7. Minus 5% if missing or inadquate (below P) summary of the study.

Introduction

1. pp. 4-
2. Concisely explain the study's purpose.
3. Provide a critical overview of relevant past research (using citations to key background literature) and identify key issues to be addressed in this study.
4. Only review constructs which are analysed in the Results - e.g., see possible topics.
5. eReserve offers some starting references, but additional references are likely to be helpful.
6. Provide logically-derived and clearly-stated research question(s) and/or hypotheses (null and/or alternative). The derivation of the questions and hypotheses should be supported by theoretical argument and citations.
   1. Firstly, ask a research question about the underlying factor structure of either university student satisfaction or time management skills. (→ Exploratory factor analysis (EFA)) e.g.,
      1. How many distinct dimensions (factors) of X are there, what are they, and which items best represent these factors?
   2. Secondly, make hypotheses about whether at least three independent variables (IVs) (you choose) predict a dependent variable (DV) (→ Multiple linear regression (MLR)) e.g.,
      1. It is hypothesised that IV1, IV2, and IV3 will each positively predict DV.
   3. Thirdly (for G students), provide a hypothesis or research question that can be addressed through qualitative analysis of the open-ended questions
7. Sub-headings may be used (optional)

Method

1. Clearly explain how the study was conducted in sufficient detail to allow a replication study, but without extraneous detail.
2. Key marking criteria: Is the study replicable? Is sufficient detail provided for a "naive person" (say, someone in Japan in 20 years time) to be able to fully replicate the study?

Participants (5%)

1. Provide a one to two paragraph description of the participants in the sample.
2. Think about which of the available data are worth summarising in order to provide an accurate and insightful description of the sample.
3. Advanced option: You may wish to compare the sampled data with UC's current enrolment statistics.

Measures (5%)

1. Briefly summarise the development and status of the survey instrumentation.
2. Only provide details for factors/items used in your results.
3. Describe the proposed factors and items (e.g., include definitions and example items). Use table(s) to help present and summarise this information
4. Describe the response format (measurement scale), including the meaning/direction of high/low scores.

Procedure (5%)

1. Sampling:
   1. What was the population and sampling frame?
   2. What sampling technique was used?
2. Administration:
   1. Summarise and reference the Survey administration guidelines
   2. Where and how did you collect data?
   3. How long did participants take?
   4. Refusal rate? (for the surveys you administered)
3. Procedural anomalies? (e.g., were there are unanticipated responses or unplanned occurrences?)

Results

1. The approach to analysis should proceed through three basic steps:
   1. Screen the data - Download and screen the survey data
   2. Psychometric instrument development - At least one EFA (of either university student satisfaction or time management), reliability analysis (internal consistency), composite scores and descriptive statistics, and correlations between factors
   3. Hypothesis testing - At least one MLR with at least three IVs
2. Clearly communicate your depth of your understanding in your own words; avoid writing results in a robotic (mindless) manner (e.g., paraphrasing sample write-ups)
3. Report and interpret the directions of any relationships (e.g., if there is a positive correlation between X and Gender, what does this mean - are higher values of X associated with males or females?)
4. Qualitative analysis (for 6667 (G) students only)
5. Additional analyses may be presented. However, it is quite possible to gain high marks by conducting one of each of the required analyses. If additional analyses are presented, then they must be very clearly related to the research question and hypotheses.
6. In marking, some account will be taken of the scope of the analysis undertaken. Where a more advanced analysis is appropriate (given the research questions(s) and/or hypothesis(es)) and is well conducted, this could represent higher quality work than a simpler analysis. However, there's much also to be said for parsimony (keep it simple, get it right), and focusing on doing a good job of fulfilling the minimum requirements. The best reports are usually not the most complex ones. If in doubt, go with analyses which meet the minimum criteria, which relate to the research question and/or hypotheses, and which you are confident about accurately conducting, interpreting and presenting.

Data screening (5%)

1. Checking and correcting of data file - see data screening

Psychometric instrument development (20%)

1. Report the results of at least one EFA of either University Student Satisfaction (USS) items or Time Management (TM) items
   1. The minimum requirement is to report one EFA. However it may be of interest to obtain composite scores for both USS factors and TM factors. So, you may wish to conduct EFA for both. In this case present one EFA in full detail in the Results and summarise the results of the other EFA perhaps with relevant output in an appendix. Or present both EFAs in full detail in the Results (if word count permits).
2. Indicate the type of EFA used (extraction and rotation)
3. Explain the extent to which EFA assumptions were met (sample size (incl. cases:variables ratio), linearity (e.g., check at least some scatterplots, particularly for bivariate outliers or non-linear relations), factorability of correlation matrix)
4. Provide the inter-item correlations for the final model(s) (in an Appendix) - see FAQ for suggestions about how to do this
5. Focus on the final model but summarise the steps taken to get there (e.g., How many factors were extracted initially? What models/factors structures were examined? Was the expected structure evident? Which factors were retained and dropped and why?)
6.  % of variance explained (for the initial and final model(s)
7. Label and describe/define each factor
8. Summarise which items were retained and dropped and why?
9. Table of factor loadings and communalities (for the final model)
10. Reliability analysis (Internal consistency - (Cronbach's alpha)) for each factor
11. Calculation of composite scores to represent latent factor scores
12. Descriptive statistics for the composite scores
13. Correlations between composite scores

Multiple linear regression (20%)

1. Report the results of at least one MLR with at least three predictors - can use any variables in, or derived from, the supplied data set (if they meet the assumptions for MLR)
2. Explain the purpose (research question and/or hypotheses) of the MLR
3. Explain the type of MLR
4. Describe the IVs and DVs, and any manipulations of the variables (e.g., recoding or creating an interaction term).
5. Explain the extent to which assumptions were met (e.g., multicollinearity, multivariate outliers).
6. Present the correlation between the items (can be part of the MLR coefficients table - see sample write-ups for examples).
7. Report amount of variance explained (R² and Adjusted R² (and the R² change at each step if a hierarchical MLR is being conducted), with inferential tests (F(df), p).
8. Report significance, size, direction and relative contribution of each IV, based on a table showing the MLR coefficients, including B for intercept & IVs and Beta (β), t, p, and possibly the zero-order ('r), partial correlations (r_p), and semi-partial correlations squared (sr²) for each IV.
9. Indicate whether each hypothesis was rejected.

Discussion

1. Provide insightful, balanced interpretation of the results with tangible recommendations for future research and practical implications.
2. Demonstrate a comprehensive understanding of the results and relate this to the research area. (Avoid merely summarising the results without providing additional critical commentary.)
3. Demonstrate breadth and depth of understanding of the results and their implications.
4. Critically review the study's methodology and make practical suggestions for how it could be improved, e.g.,
   1. Statistical power?
   2. Appropriateness of the sampling technique?
   3. Generalisability of the findings?
5. Does the discussion objectively interpret the results - or is it biased towards the researcher's personal preferences?
6. Strengths and the weaknesses of the current study?
7. What are the potential implications and applications of this research?
8. Does the Discussion build on the material reviewed in the Introduction?

References

1. Is the reference list complete (i.e., none missing and all cited)?
2. Does the lab report make effective use of a core set of relevant, high-quality, peer-reviewed, citations? This involves citing and meaningfully discussing appropriate references (as opposed to just dumping citations without explanation), particularly in the Introduction and Discussion.
3. Reference other relevant other resources, such as useful non-peer-reviewed internet references. Make careful selection, however, to ensure you are citing the highest quality and most academically acceptable material.
4. Reference the measures and the survey administration guidelines - do not include them in the Appendices.
5. Use APA style, including for electronic sources.

Appendices

1. Appendices are optional. There are no marks specially allocated to Appendices.
2. Appendices are for additional detail which is relevant to understanding the main body, but which would break the flow of the main report.
3. The Results section should be complete and therefore should not generally require additional documentation.
4. Copies of statistical output can be provided in the appendices, but this is not required. If you do, it may help the marker to diagnose problems with results.
5. Appendix content does not need to follow APA style (e.g., Raw statistical output with out without annotations can be provided to demonstrate how the results were arrived at).
6. Appendices should be well organised, with clear and effective labeling.
7. Do not include a copies of available information in the appendices - use references instead e.g., for the survey and the administration guidelines.
8. All Appendices should be referred to in the main body.

Word count

1. There is no minimum word count
2. Maximum word count
   1. 3000 words + 10% (for undergraduates)
   2. 3500 words + 10% (for graduates)
3. How to calculate:
   1. Count everything using a word processor from the beginning of the Introduction to end of Discussion
   2. Includes all text, headings, footnotes, citations, tables and figures etcetera.
4. Penalty for exceeding maximum word count: Markers will ignore words beyond the maximum (i.e., most likely resulting in a reduced mark for the Discussion).
5. Suggested section word counts provided in the general marking criteria are suggestions only - you should decide for yourself.

Sample write-ups

Explore these sample write-ups for ideas, but you must demonstrate independent thinking and writing in order to satisfy the learning outcomes.

1. Examples of APA style write-ups for specific analyses can be found in the textbooks
2. Lab report from a previous participant: Not everything in this example lab report is correct - in fact, there is a lot of "red ink". This has been done on purpose to illustrate how the report could have been even better.
   1. Sample 2007 HD lab report (.pdf)
   2. Marking and feedback sheet for the sample lab report (.xls)
3. Sample write-ups for specific analyses:
   1. Exploratory factor analysis (.doc)
   2. MLR (.doc)

Graduate participants

Graduate participants in this unit are required to produce a more advanced lab report which incorporates a qualitative analysis within a 3,500 word report. The additional requirements are:

1. 1. Introduction: Develop an extra research question and/or hypothesis which sets up a qualitative analysis.
   2. Method: Mention the open-ended question(s) in the Measures section.
   3. Results:
      1. Present an additional analysis, a qualitative analysis of responses to at least one of the the open-ended questions. The analysis could be approached qualitatively (e.g., using thematic analysis) or quantitatively (e.g., using multiple response frequency analysis).
      2. Explain the data coding and/or interpretative process
      3. Present a thick description of the data (if treating qualitatively) or descriptive results (frequencies/percentages via multiple response analysis, possibly with an accompanying figure, if treating quantitatively), with the themes illustrated by representative quotes.
   4. The weighting for the Results section (45%) will be retained, with the sub-section weighting adjusted to: 0% Data screening; 15% Psychometric instrument development; 15% Multiple linear regression; 15% Qualitative analysis

Submission

1. Submit the lab report electronically via Moodle.
2. Download the cover sheet, fill it out, insert it into your lab report.
3. Acceptable file types: .rtf, .doc, .docx, .odf, and .odt
4. Maximum file size: 20MB
5. Name the file with your student number (e.g., u613374.doc)
6. Submit a single document which includes the coversheet, lab report, and any appendices. Multiple file attachments are not accepted. Use the Upload files/Add button at the bottom - If it doesn't display, make sure to login to Moodle first.
7. Do not submit hard copies. Do not send backup email copies.
8. Follow through to the SUBMIT and CONFIRM buttons, after which no more changes to your file can be made. Then you will get confirmation of receipt of your file.

Late penalty

1. A 2% penalty per day (or part thereof, including weekends) will be applied to late submissions (e.g., 12 hours late = -1% of maximum possible mark).

See also

• Possible topics for investigation
• FAQ
• General feedback about previous lab reports
• Marking dispute process
• Data download
• Data entry
• Survey administration guidelines
• Data screening
• References
• TUSSTMQ6 (includes the downloadable survey, 2011-2012)
• UC grading schema

External links

• eReserve
• Qualitative analysis
• Research questions and hypotheses
• Survey tracks engagement (The Australian, May, 2008)
• Gillard, J. (2009). Universities Australia Conference - March 4, 2009 - speech.
• Lab report writing resources