Survey research and design in psychology/Assessment/Lab reports/3

Lab report 3

Task

Report on the factor structure of one multi-item, multi-dimensional survey instrument using exploratory factor analysis and reliability analysis.

Marking criteria

In addition to the generic guidelines, this report should include:

1. 5%. Title/Abstract: : Provide a succint overview of the study.
   1. Sumarise the study's purpose, method, findings, and implications in less than 150 words.
2. 10%. Introduction: Clearly introduce the topic and the hypotheses:
   1. 5%. Background: Briefly introduce the topic (student satisfaction or time management) and background literature about its dimensionality (factor structure).
   2. 5%. Research questions: Propose logically-derived research question(s) (which are addressed in the Results) - in its simplest form, the research question is likely to be e.g., "What is the underlying factor structure for university students' satisfaction with their university experience?"
3. 10%. Method: Clearly explain how the study was conducted:
   1. Participants: Not application (do not include)
   2. 10%. Instrumentation
      1. Describe and explain the instrumentation - this should include the entire survey, but focus primarily on the instrumentation used in the current study (i.e., student satisfaction or time management).
   3. Procedure: Not applicable (do not include)
4. 50%. Results: Appropriate tests conducted and reported in APA style:
   1. 50%. Describe and present an EFA which identifies the number of factors, total variance explained, factor loadings and communalities, reliabilities for each factor, and correlations between the factors. Describe initial models but only present the results for the final analysis which is likely to drop several items from the initial step. Explain which items were dropped and why. Item-correlations should be included in an Appendix.
5. 25%. Discussion: Explain results and implications:
   1. Explain what the analysis found out about the research question(s) e.g.,
      1. What do the factors measure and represent?
      2. How 'good' is the factor structure? Are there alternative models that could be considered?
      3. How reliable are each of the factors?
      4. How correlated are the factors?
   2. What can be recommended e.g., how could the measure be improved?

FAQ

Which variables should I analyse?

1. Either the Time Management or the University Student Satisfaction variables

How can I decide on a factor structure?

1. Follow the steps in Tutorial 3 - Exploratory factor analysis and the relevant section in the SPSS textbook

What tables are needed?

1. Table 1 - Factor loadings and communalities
2. Table 2 - Factor names and definitions, with number of items, and Cronbach's alpha
3. Table 3 - Factor correlations (depending on number of factors, this could be reported in text instead).

Is Cronbach's alpha needed for each factor?

1. Yes

How do I calculate the internal consistency for each factor?

1. A common confusion is to calculate Cronbach's alpha for all items rather than separately for each factor
2. Follow the steps in Tutorial 3 - Internal consistency and the relevant section in the SPSS textbook
3. Repeat for each set of items which represent a factor

How do I create composite scores for each factor?

1. See Tutorial 3 - Composite scores

How I calculate the correlations between the factors?

1. Create composite scores for each factor
2. Analyze - Bivariate - enter composite scores

Are descriptive statistics and/or graphs needed for each composite factor score?

1. No

How many decimal places should I report to?

• Generally 2, but this may vary depending on the scale of measurement - what is meaningful/useful without being unnecessarily detailed?

How do I add the correlations to an Appendix?

• See item correlations

General feedback

1. Word count
   1. Many assignments went over the maximum word count and thus received penalties
2. APA style
   1. Formatting of citations was often not correct e.g.,
      1. for three or more authors, after initial citation, use First author surname et al., year
      2. User ampersands (&) inside brackets for citations and use the word "and" when outside brackets
   2. Use Australian spelling e.g., many students are using American spelling for words such as organisation (Australian) / organization (American)
   3. Font size and type (should be Times New Roman 12 pt)
   4. Numbers were often not formatted correctly, e.g., for numbers under 10, report in words, e.g., five surveys
   5. Spaces should be used either side of symbols which replace words (e.g., p < .05 instead of p<.05)
3. Abstract
   1. Generally, well done.
   2. In general, do not include references unless they are particularly pertinent to the study
4. Introduction
   1. Quite often there were no literature review of previous research or theory about the factor structure of the target multi-dimensional construct
   2. Research questions were generally clear and appropriate.
5. Method
   1. Generally well done.
   2. An example item could often have been a helpful addition.
   3. Quite often it was not explained how the items were developed.
6. Results
   1. Generally, well done.
   2. Generally, people reported finding between three and five or six satisfaction, with probably around four being most common.
   3. Some models were presented based on all items, rather than eliminating complex items or items with low primary loadings and/or high cross-loadings
   4. A few models were overzealous in removing items, resulting in models with a small number of items.
   5. Correlations between factors were often not presented.
   6. Item numbers generally don't mean much to an external reader - be more descriptive.
   7. Table captions could sometimes have been explicit e.g., "Correlations Between the Factors" -> "Correlations Between the Four University Satisfaction Factors"
   8. Round results for factor loadings and correlations to two decimal places
   9. Rank items by primary factor loading in factor loading table
7. Discussion
   1. Generally, well done.
8. References
   1. It was rare for references to be formatted in complete APA style, although mostly APA style here was quite good.
9. Appendices
   1. Generally, well done.