Survey research and design in psychology/Assessment/Lab report

From Wikiversity
Jump to: navigation, search
Lab report - Guidelines
Gnome-settings-background.svg View the accompanying screencast: [1]

Overview[edit]

  1. Present an independently-developed APA style lab report which examines:
    1. the psychometric properties of a multi-item, multi-factorial measure (either time management or time perspective) using exploratory factor analysis and reliability analysis (guided by a research question)
    2. an explanatory model involving at least three predictors and one dependent variable using multiple linear regression (guided by hypotheses - one per predictor).
  2. Use the aggregated data file from the data collection and entry assessment exercise, based on the TSQFUS1.
  3. For further information, see the general marking criteria and the detailed marking criteria which explains what to include for each section.

Data download, screening, & recoding[edit]

  1. Data download: TSQFUS1_2017_All_Unscreened
  2. Data screening
  3. Data recoding

Marking process[edit]

  1. Lab reports will be evaluated according to the general and detailed marking criteria.
  2. For reports submitted by the original due date, marks and feedback should be available three weeks of submission. Marking of reports submitted late or with extension may take longer than three weeks from the date of submission.
  3. Availability of marks and feedback will be notified via Moodle.

General marking criteria[edit]

Within each of the lab report sections, marking will reflect these general criteria:

  1. 70%: Quality of execution of the task
    (e.g., review literature and develop hypotheses, describe method, summarise results, discuss findings)
  2. 10%: Quality of written expression
    (structure and headings, flow, sentence and paragraph structure, spelling, and grammar)
  3. 10%: Contribution to a cohesive, meaningful report
    (i.e., a story that makes sense - with sections that are not disjointed)
  4. 10%: APA style

Detailed marking criteria[edit]

These detailed guidelines describe the requirements for each section of the lab report:

Overview of sections[edit]

This is an overview of the requirements and weighting for each section:

Criteria Description  % Suggested word count
Title/Abstract Succinct, specific title. Abstract covers purpose, method, findings, and conclusions. 5% 10-15, 160-200
Introduction Establishes the problem, reviews theory and research, develops research question(s) and hypotheses 10% 400-700
Method Describes method and design, including Participants, Materials, and Procedure 15% 500-700
Results Screens (5%) and analyses data using EFA, internal consistency, descriptive statistics and correlations for the composite scores (20%), and MLR (20%) 45% 900-1500
Discussion Summarises and interprets the findings, considers implications, and makes recommendations 25% 500-750

The word count ranges per section are suggestions only. The only restriction is an upper limit on the overall word count.

Cover sheet[edit]

  1. p. 0
  2. Download via Lab report cover sheet - click File - Save As

Title page[edit]

  1. p. 1
  2. APA style example - except use student ID number instead of name (for blind marking)
  3. Running head + Page number - APA style includes a running head on each page, including the title page, along with page numbers
  4. Title
    1. ~10-15 words
    2. Does the title convey the content of the report?
    3. Should be succinct, yet specific to the study (e.g., mentions key constructs)?
    4. Catchy? Memorable?
  5. Institution name
  6. Do not include non-APA style material such as: Word count, Unit name, Date, Tutor name etc.

Abstract[edit]

  1. p. 2
  2. ~160-200 words.
  3. Abstract heading (should not be in bold)
  4. Summarise (avoiding excessive detail)
    1. Purpose of the study
    2. Method, including the sample size and sampling method
    3. Key findings, including the % of variance explained in the EFA, the identified factors, the R2 for the MLR and the relative strength, direction, and significance for each predictor
    4. Conclusions about theory and method, with key recommendation(s)

Introduction[edit]

  1. pp. 3-
  2. Introduce the topic and concisely explain the study's purpose(s).
  3. Provide a critical overview of relevant past research and identify key issues to be addressed in this study.
  4. Only review constructs which are analysed in the Results - e.g., see possible topics.
  5. Use citations to key background literature - Wikiversity (see readings) lists some starting references, but use of additional references is recommended.
  6. Avoid describing methodological details about the current study, such as measurement tools - this belongs in the Method.
  7. Provide logically-derived and clearly-stated research question(s) and/or hypotheses (can be null and/or alternative). The derivation of the questions and hypotheses should be supported by theoretical argument and citations.
    1. Firstly, specify a research question about the underlying factor structure of one of the multidimensional constructs in the TSQFUS1 (i.e., either time perspective or time management) e.g., "How many distinct dimensions (factors) of X are there, what are they, and which items best represent these factors?"
    2. Secondly, make a hypothesis about the extent to which each of the independent variables (IVs) predict a dependent variable (DV) (→ Multiple linear regression (MLR)) (choose any suitable variables, or composite of several variables, from the TSQFUS1) data file e.g., "It is hypothesised that [IV1], [IV2], and [IV3] will each positively predict [DV]".
    3. Thirdly (for G students), provide a hypothesis or research question that can be addressed through qualitative analysis of responses to one or both of the open-ended questions.

Method[edit]

  1. Clearly explain how the study was conducted in sufficient detail to allow a replication study, but without extraneous detail.
  2. Key marking criteria: Is the study replicable? Is sufficient detail provided for a "naive person" (say, someone in Japan in 20 years time) to be able to fully replicate the study?

Participants (5%)

  1. Provide a one to two paragraph descriptive overview of the participants in the sample.
  2. Consider which of the available data can be summarised in order to provide an insightful description.
  3. Advanced option: You may wish to compare the sampled data with population statistics for UC students (e.g., see UC at a glance and Annual Reports).

Measures (5%)

  1. Briefly summarise the development of the survey instrumentation.
  2. For the measures used to collect the data which is analysed in the Results, describe e.g.
    1. type of questions
    2. response format
    3. meaning/direction of high/low scores
    4. There is no point in describing measures which are not used in the study.
    5. Optional: Use a table to help present the proposed factors (e.g., label, definitions and example items)

Procedure (5%)

  1. Sampling:
    1. What was the target population and the sampling frame?
    2. What sampling technique was used?
  2. Administration:
    1. Briefly summarise and provide an APA style reference for the Survey administration guidelines
    2. Where and how did you collect data?
    3. How long did participants typically take to complete the survey?
    4. Refusal rate? (for the surveys you administered)
    5. Procedural anomalies? (e.g., explain any unanticipated responses or unplanned occurrences)

Results[edit]

  1. The approach to analysis should proceed through three basic steps:
    1. Data screening - Summarise in one to two paragraphs how the data was screened and what changes were made. Is enough detail provided for the same steps to be followed by someone else? However, avoid excessive detail (e.g, CaseID number are meaningless to a reader).
    2. Sample size assumptions do not belong in this section - instead, this would belong in the section(s) for the corresponding analyses.
    3. Psychometric instrument development
      1. Conduct EFA of a multidimensional construct (either the time perspective items or the time management items).
      2. For each extracted factor, provide reliability analysis (internal consistency - Cronbach's alpha), composite score descriptive statistics, and correlations between factors.
    4. Multiple linear regression - Conduct at least one MLR with at least three IVs to address one hypothesis per IV
    5. Qualitative analysis (for 6667 (G) students only)
  2. Communicate the depth of your understanding by using your own words; avoid writing results in a robotic (mindless) manner (e.g., avoid overparaphrasing a specific sample write-up)
  3. Most statistics should be rounded to two decimal places unless there is particularly useful information communicated by including a third decimal place (e.g., when reporting exact p values).
  4. Scope and depth of analysis
    1. Additional analyses may be presented. However, it is quite possible to gain maximum marks by conducting one of each of the required analyses. If additional analyses are presented, then they must be clearly related to the research question and hypothesis(es).
    2. In marking, some account will be taken of the scope of the analysis undertaken. Where a more advanced analysis is appropriate (given the research questions(s) and/or hypothesis(es)) and is well conducted, this could represent higher quality work than a simpler analysis. However, there's much also to be said for parsimony (keep it simple and get it right) by focusing on doing a good job of fulfilling the minimum requirements. The best reports are usually not the most complex ones. If in doubt, go with analyses which meet the minimum criteria, which relate to the research question and/or hypotheses, and which you are confident about accurately conducting, interpreting and presenting.

Data screening (5%)

  1. Summarise what was done to check and correct errors in the data - see data screening

Psychometric instrument development (20%)

  1. Report the results of at least one EFA of a multidimensional survey construct (either time perspective or time management)
    1. The minimum requirement is to report one EFA. However, it may be of interest to conduct a factor analysis of another set of items (e.g., stress) in order to develop other composite scores for further analysis. In this case, present one EFA (of other time perspective or time management) in full and briefly summarise the results of the other EFA, perhaps with relevant output in an appendix.
  2. Indicate the type of EFA used (Type of extraction? Type of rotation?)
  3. Explain the extent to which EFA assumptions were met, but not excessively (e.g., one indicator of factorability is quite sufficient; more is redundant)
    1. Sample size (incl. cases:variables ratio)
    2. Linearity (e.g., check at least some scatterplots, particularly for bivariate outliers or non-linear relations)
    3. Factorability of correlation matrix (either examine item correlations, report about anti-correlation matrix diagonals, or use a global diagnostic, either KMO or Bartlett's - but do not report all of these as they are redundant)
  4. Focus on the final model but summarise the steps taken to get there (e.g., How many factors were extracted initially? What models/factors structures were examined? To what extent was the expected structure evident?)
  5.  % of variance explained (for the initial and final model(s))
  6. Label and describe each factor
  7. Which items were retained and/or dropped and the reasons why
  8. Table of factor loadings and communalities (for the final model)
  9. Reliability analysis (Internal consistency - (Cronbach's alpha)) for each factor
  10. Calculation of Composite scores to represent each factor
  11. Table of descriptive statistics for the composite scores
  12. Table of correlations between composite scores

Multiple linear regression (20%)

  1. Report the results of at least one MLR with at least three predictors - can use any variables in, or derived from, the supplied data set (if they meet the assumptions for MLR)
  2. Reiterate the purpose (research question and/or hypotheses) of the MLR
  3. Mention the type of MLR
  4. Describe the IVs and DVs, and any manipulations of the variables (e.g., recoding or creating an interaction term). If its not already clear from the Method, clarify the direction of scoring.
  5. Explain the extent to which assumptions were met (e.g., multicollinearity, multivariate outliers)
  6. Present the correlations between the items (can be part of the MLR coefficients table - see sample write-ups for examples). Demonstrate understanding of the directions of any relationships (e.g., if there is a positive correlation between X and Gender, what does this mean? Are higher values of X associated with males or females?)
  7. Report amount of variance explained (R2 and Adjusted R2 (and the R2 change at each step if a hierarchical MLR is being conducted), along with inferential tests (F(df), p)
  8. Report significance, size, direction and relative contribution of each IV.
  9. Table showing the correlations and MLR coefficients, including B for intercept & IVs and Beta (β), and the statistical significance (e.g., t, p), and semi-partial correlations squared (sr2) for each IV and explain the direction and size of the results.

Discussion[edit]

  1. Build on the introduction to explain the results and what they mean in a balanced manner.
  2. Demonstrate breadth and depth of understanding of the results and their implications. Avoid merely summarising the results without providing additional critical commentary.
  3. Critically review the strengths and weaknesses of the study's methodology and make practical suggestions for how it could be improved, e.g.,
    1. Validity and reliability of the measures?
    2. Statistical power?
    3. Appropriateness of the sampling technique?
    4. Generalisability of the findings?
  4. Provide tangible recommendations for future research and practical implications.

References[edit]

  1. Is the reference list complete (i.e., none missing and all cited)?
  2. Does the lab report make effective use of a core set of relevant, high-quality, peer-reviewed, citations? This involves citing and meaningfully discussing appropriate references (as opposed to just dumping citations without explanation), particularly in the Introduction and Discussion.
  3. Reference the instrumentation and the survey administration guidelines - do not copy the guidelines into the Appendices.
  4. Use APA style, including for electronic sources. Include DOIs where relevant.

Appendices[edit]

  1. Appendices are not needed - they are optional.
  2. Appendices are for additional detail which is relevant to understanding the main body, but which would break the flow of the main report e.g., the correlations between the items used in the factor analysis.
  3. Appendix content does not need to follow APA style but should be well organised, with clear labeling.
  4. Each appendix should be referred to at least once in the main body.

Style[edit]

  1. Follow APA style except:
    1. add a cover sheet
    2. use single-spacing (for electronic submission; double-spacing is a relic from paper submission days - it allowed room for hand-written comments)
    3. do not include author name on the title page (to facilitate blind marking)
    4. include any tables or figures in situ (i.e., embed them without the main body of the report rather than putting them after the references)
    5. formatting of any appendices does not need to follow APA style
  2. Example (owl.english.purdue.edu)
  3. The most important aspects of APA style for this lab report are:
    1. Times New Roman 12 pt font
    2. Running head with page numbers
    3. Page margins
    4. Left-justify body text
    5. Captioning and layout of tables
    6. Right alignment of statistics in tables
    7. Citations and referencing (including for electronic references and use of DOIs)
    8. Symbols, abbreviations, and formatting of statistical symbols (including M, SD, skew, kurt, N, n, F, t, p, r, R, R2, sr2)

Word count[edit]

  1. There is no minimum word count.
  2. Maximum word count:
    1. 2800 words + 10% (for undergraduates)
    2. 3300 words + 10% (for graduates)
  3. Count everything using a word processor from the beginning of the Introduction (i.e., not Title page, Abstract etc.) to end of Discussion, including all text, headings, footnotes, citations, tables and figures etcetera (i.e., but not the sections after the Discussion, such as References and Appendices).
  4. Penalty for exceeding maximum word count: Markers will ignore words beyond the maximum (i.e., most likely resulting in a reduced mark for the Discussion).
  5. Word counts provided in the section overview are suggestions only.

Independent writing and plagiarism[edit]

It is necessary to demonstrate independent thinking and writing in order to satisfy the lab report learning outcomes. In other words, avoid plagiarising from these samples or other guides. Some sample write-ups are provided to give ideas. Also seek out other examples (e.g., see the readings). Then write up your results in your own words.

If you are in some doubt about whether you've demonstrated sufficiently independent writing, then upload the assignment to get a text-matching analysis via Urkund. Text-matching for this assignment between about 10% and 30% is reasonably common and probably indicates no particular cause for concern. However, it is recommended that drafts with text-matching scores above approximately 30% should be reviewed and rewritten in order to more effectively demonstrate independent understanding and writing.

Sample write-ups[edit]

Some previous High Distinction (HD) reports:

  1. Example 2007 HD lab report (.pdf) - Marking and feedback sheet (.xls)
  2. Example 2013 HD lab report (.pdf)
  3. Example 2016 HD lab report (with qualitative analysis for G students and feedback) (.pdf)

Sample write-ups for specific analyses. Note that these samples are longer than is likely to be able to be presented in the lab report within the word count limit.

  1. Exploratory factor analysis (.docx)
  2. MLR (.doc)
  3. Some of the readings also provide examples of APA style write-ups for specific analyses.

Graduate participants[edit]

Graduate participants in this unit are required to produce a more advanced lab report which also incorporates a qualitative analysis within a 3,300 word report. The additional requirements by section are:

    1. Introduction: Develop an extra research question and/or hypothesis which sets up a qualitative analysis (and relevant argument and background literature).
    2. Method: Mention the open-ended question(s) in the Measures section (which are subsequently analysed in the Results).
    3. Results:
      1. Present an additional analysis, a qualitative analysis of responses to at least one of the open-ended questions in the TSQFUS1. The analysis can be approached qualitatively (e.g., using thematic analysis) or quantitatively (e.g., using multiple response analysis).
      2. Explain the data coding and/or interpretative process
      3. Present a thick description of the data (if treating qualitatively) or descriptive results (frequencies/percentages via multiple response analysis, possibly with an accompanying figure, if treating quantitatively), with the key themes illustrated by representative quotes.
    4. The weighting for the Results section (45%) will be retained, with the Results sub-section weighting adjusted to:
      1. 0% Data screening;
      2. 15% Psychometric instrument development;
      3. 15% Multiple linear regression;
      4. 15% Qualitative analysis

Submission[edit]

  1. Submit the lab report electronically via the unit's Moodle site - via the Assessment module and then Lab report submission.
  2. Follow the lab report guidelines.
  3. Acceptable file types: .doc, .docx, .odt, and .rtf
  4. Name the file with your student ID number (use the format u######.*** where ###### is the student number and *** is the file extension (e.g., u613374.doc)).
  5. Submit a single document which includes the cover sheet, lab report, and any appendices. Multiple file attachments are not accepted. Maximum file size: 20 MB.
  6. Use the "Add submission" button or drag and drop the file; if it doesn't display, make sure to login first. Once the correct file has uploaded, click on "Save changes".
  7. The submitted file will then be sent to URKUND to be analysed. A report may take ~24 hours. Availability of the report will be notified via email. Further information on URKUND is available from http://learnonline.canberra.edu.au/course/view.php?id=1344&section=4.
  8. You may wish to modify the report and re-upload by clicking on “Edit submission”.
  9. Click on “Submit assignment” and then "Continue" to submit the assignment. Until this point, the assignment has not been submitted and will not be marked; after this point, the assignment is submitted and can no longer be changed other than by a lecturer manually reverting the submission.
  10. An automatic confirmation that the file has been submitted will be sent to the student email address.
  11. Do not submit hard copies.
  12. Do not send backup email copies.

Late submissions[edit]

  1. Standard UC policy applies:
    1. 5% penalty per day (including weekends)
    2. no submissions accepted or marked after 7 days

See also[edit]

External links[edit]