Motivation and emotion/Book/2017/Delayed reinforcement and motivation
What is the effect of delayed reinforcement on motivation?
- 1 Overview
- 2 Delayed reinforcement in behaviourism and learning
- 3 Terminology
- 4 Early research
- 5 Intrinsic and extrinsic motivation
- 6 Neuropsychological perceptive
- 7 Cognitive psychology theories
- 8 Reducing the effects of delayed reinforcement
- 9 Quiz
- 10 Conclusion
- 11 See also
- 12 References
- 13 External links
When reinforcement for a behaviour is delayed, motivation for that behaviour is reduced (Lattal, 2010; Killeen, 2011). The majority of the scientific investigations into this phenomenon investigate rewards as the reinforcement. In economics this is called delayed discounting or temporal discounting which is heavily linked with the concept of delayed gratification. All these phenomenon describe a tendency for people to be motivated more by immediate rewards then delayed rewards, such that even a substantially better reward that is delayed, is less motivating then a worse but immediate reward (Marzilli, Ericson, White, Laibson, & Cohen, 2015). This seemingly minor preference has major ramifications for how we motivate ourselves and how we achieve our goals. Many tasks we found difficult to get the willpower to do require us to continue a behaviour despite no immediate reinforcement for the behaviour, tasks we may consider important, but do not find enjoyable to perform. A major aspect of self-control is the ability to resist the preference for an imitate reward to achieve a greater but delayed reward (Stolarski, Bitner & Zimbardo, 2011). For example, a student may want to achieve a good grade, but to do so there is a large delay between the work they put in and the reward they get at the end, the good grade. This chapter will explore why people prefer immediate reinforcement, and how counteract this effect to achieve goals and increase motivation.
Behaviourist explication - Interruptions in learning.
Personality theories - time perspective.
Cognitive theories- cognitive models of motivation and goal setting, search cognition's.
Neurological theories- limbic system vs frontal lobe.
Delayed reinforcement in behaviourism and learning
In operant conditioning, a reinforcer is something that increases behaviour. It is used to get animals and people to learn behaviours. It can be pleasant or unpleasant, giving (positive) or taking away (negative). It can be administered during, after or before a behaviour. When the reinforcement is delayed, learning and subsequently motivation to perform the behaviour decreases (Lattal, 2010; Killeen, 2011). This effect can be weaker when the reinforcer is stronger (Doughty, Galuska, Dawson & Brierley, 2012).
Why does it reduce learning?
One of the reasons that learning is not as effective when reinforcement is delayed is because the subject is uncertain what behaviour is being reinforced. If there is a large delay between action and reinforcement, multiple actions may have accorded in the meantime. In support of this classical conditioning. By pairing the desired behaviour with a conditioned stimulus (a cue), and pairing that conditioned stimulus with the reinforcement, the conditioned stimulus can be given to indicate what behaviour is causing the reinforcement. This is hypothesised to be because there is a clear indication of which behaviour is leading to reinforcement (Killeen, 2011). This increases learning, however, is not completely make up for the decrease in motivation. There are effects of delaying reinforcement that go beyond impairing learning.theory, it has been shown that the effects of delayed reinforcement can be reduced if the desired behaviour is clearly shown to be the cause of reinforcement (Lattal, 2010; Killeen, 2011). This can be done by using the principles of
That's quite a bit of theoretical knowledge, letsbreak it up with a quiz to see how it relates to motivation. For the quiz, just click on the box next to the answer you think is most correct and hit submit when your ready.
Investigation into delayed reinforcement effects on motivation has been done in multiple fields of academia with each one having different terminology for the phenomenon to avoid confusion the terms have been simplified for this chapter.
Economic terms simplified.
Delay discounting is linked to delayed reinforcement but with some key differences. Delay discounting refers to the tendency to undervalue delayed rewards. Delay discounting only refers only to rewards, whereas, delayed reinforcement is the delay of anything that reinforces behaviour, whether that reinforcer is pleasant or unpleasant. There is little research outside of behaviourism that focuses on delayed negative reinforcement, so this chapter will focus mainly on positive reinforcement.
In economics delay discounting often refers only to exponential increases in the discounting of rewards, for the purposes of this chapter the term has been used more broadly encompassing hyperbolic discounting, delayed gratification, and the motivational effects of delayed reinforcement.
In operant conditioning positive reinforcement refers to anything that is added to a situation that increases behaviour. In contrast, research on delayed gratification and delay discounting investigate only what individuals would consider rewarding.
The marshmallow experiment
"The Stanford marshmallow experiment" is the name given to three famous experiments regarding delayed disscounting in children. They show empirical evidence of an unwilliness to wait for a reward, even if waiting would result in a greater reward (Mischel et al. 2010). In the seminal study, stanford nursery school children were given the option to either eat the food in front of them (sometimes a marshmallow), or wait 15 minutes without eating it, if they waited they would be given more food (Mischel et al. 1972). They found that when the prospective reward was in-front of children, thinking about the reward increased the chance of choosing the immediate reward, however, if the reward was not in sight (in a container under the table) then thinking about the reward make them more likely wait for the larger reward (Mischel et al. 1972; Lattal, 2010). Follow up research on the children involved in the study found that those who were willing to wait for a greater reward had significantly higher Scholastic Aptitude Test (SAT) scores (Shoda, Mischel, & Peake, 1990; Mischel et al. 2010). A more resent followup found that delayed discounting of preschoolers correlated with their body mass index (BMI) 30 years after the experiment, such that on average each minute waited before eating in the original experiment predicted a .2 reduction in BMI in later life (Schlam, Wilson, Shoda, Mischel & Ayduk, 2013). To investigate what type of cognition's /personalities leads to a tendency to delay gratification several theories have emerged. One of these theories looks at how the perception of time can influcen an individuals priorities.
Zimbardo's time perspective
Zimbardo’s time perspective is a theory that states that perceptions of time differ from person to person, and between situations. The original theory stated that there are 5 perspectives on time: past positive, past negative, present hedonistic, present fatalistic and future oriented. Individuals highly focused on the present and with little focus on the future are likely to choose immediate gratification over delayed gratification. Individuals who are focused more on the future then the present, are more likely to do the opposite (Stolarski, Bitner, & Zimbardo 2011). Although the experimentation on delayed gratification was the inspiration for time perceptive theory, a resent literature review found that despite conceptual similarities, there is only a weak correlation between future time perspective and delay discounting (Teuscher, & Mitchell, 2011). Time perspective and the effects of delayed reinforcement can cause similar impairments on goal completion, but it is increasingly clear that they are distinct mechanisms, and a more comprehensive theory of delayed reinforcement is needed (Stolarski, et al. 2011; Teuscher, & Mitchell, 2011).
The prevalence of poor physical and mental health and socioeconomic related to delay discounting has lead to a view that delayed discounting should be treated as a trans-disease process, meaning it is a shared cause/risk factor in multiple disorders (Bickel et al. 2012).
Intrinsic and extrinsic motivation
A common theoretical perspective in motivational psychology that can be applied to delayed reinforcement is intrinsic and extrinsic motivation. For extrinsically motivated tasks, the time between a behaviour and its reinforcement is variable, as it is separated from the behaviour itself. For example, if the only motivation for a behaviour is that you will be payed for it, then the delay is proportional to when you are payed , which could mean the reinforcement of the behaviour is a week or month after the behaviour is performed. For intrinsically motivated tasks the behaviour is reinforced instantaneously as we find the behaviour itself rewarding. (Dysvik, & Kuvaas 2013). There is little empirical research investigating connections between delayed reinforcement and intrinsic and extrinsic motivation, thus it may be an area of future research.
Studies using fMRI's brain scan techniques have revealed key insights into the differences in reasoning when considering a delayed reinforcement, compared to considering an intimidate reinforcement (Ludwig et al. 2015; Hakimi & Hare, 2015). The limbic system, specifically the paralimbic cortex, activates more thenaverage when considering immediate reinforcement, but not when considering delayed reinforcement. In contrast, the pre-frontal cortex, specifically ventro-medial cortex, activates more then average when thinking about both delayed and immediate reinforcement (McClure, et al., 2004; Ludwig et al. 2015). This suggests that the prospect of an immediate reward activates an emotional response, similar to anticipating a reward. This emotional preparation for a reward does not appear to be present when considering a delayed reward, as indicated by the lack of limbic system activation. This is a potential explanation for why individuals have a bias towards immediate reinforcements. (McClure, et al., 2004; Ludwig et al. 2015).
Imagining the reward is needed to evaluate the value of delayed rewards, whereas imagination plays a less important role with immediate rewards, potentially because anticipatory activation of the limbic system makes imagining its benefits less difficult (Hakimi & Hare, 2015). Heuristics such as the present bias and projection bias show that we are bad at imagining how we will feel in the future, but have a more accurate picture of our feelings in the present (Pychyl, & Flett, 2012). This heuristic explanation of delay discounting, which will be expanded later in the chapter, is supported by the fact that the limbic system activates when contemplating an immediate choice, but not when contemplating a delayed choice. We feel the benefits of the immediate reward but have to imagine the feeling of the future reward (Hakimi & Hare, 2015).
Imagining a delayed reward and imagining a more immediate reward use different neural pathways. With the imagining the value of the immediate reward induces a great amount of motivation, as it also involved in the enjoyment of the reward (it begins the release of dopamine) i.e. it begins the process of reinforcement before the decision is made (McClure, et al., 2004; Ludwig et al. 2015). However, by improving imagination of the future reward, activation of the reward process can begin for the delayed reward as well; i.e. imaging the future reward also releases dopamine (Hakimi & Hare, 2015; McClure, et al., 2004; Ludwig et al. 2015).
Cognitive psychology theories
Cognitive Search model
The cognitive mechinismthat cause delay discounting are still largely unknown, one attempt to create a theoretical explanation for why delay discounting occurs, is from a 2012 study using computer modelling as proof of concept (Kurth‐Nelson, Bickel, & Redish, 2012). The theory rests on three assumptions. First, the value of the rewards are weighed up in a via a cognitive searching processes. Second, the faster the cognitive representation of the reward comes to mind the more valuable the reward will appear to be. Thirdly, the less delayed the reward is the faster it will be found by the cognitive searching process. In essence, the argument made by the theory is that it is easier to think of the value of a reward if that reward is closer to the present. As we think more about the benefits of the closer reward, then the distant reward, it appears to be the better option. If the assumptions are correct it does make logical sense that if we think more about the first reward, we will be more aware of its value. Furthermore, the theory is compatible with the evidence from other studies on delay discounting. There is evidence that we think more about a closer reward then delayed reward, and evidence that the more we think about a reward the more we value it (Hakimi & Hare, 2015; Lattal, 2010). It is still, however, an untested theory, with empirical evidence coming from computer generated neural nets, not humans (Kurth‐Nelson et al. 2012). Empirical research testing if its three assumptions are accurate when applied to humans is needed. Furthermore, this model is thought only to explain delay discounting in tasks requiring cognitive effort, not in habitual tasks (Kurth‐Nelson et al.2012).
This cognitive search theory still has some usefulness despite its limitations. A strength of this theory is its comparability with other theoretical explanations. For example, for individuals who are future oriented in Zimbardo's time perspective value delayed rewards more thenaverage, potentially because they come more readily to mind. Another strength is its easy application to problems, as it demonstrates that merely thinking more about the delayed reward increases motivation to obtain it (Kurth‐Nelson et al. 2012). Increases in working memory have also been shown to improve performance in delay discounting tasks, which is also compatible with the theory, as an improved working memory could improve the cognitive searching process (Lin & Epstein, 2014).
When setting goals setting a far away goal can be counter productive as it makes the goal seem difficult and abstracted. It encourages fantasying about the end state which can reduce motivation (Hoyle, 2013; Dysvik, & Kuvaas 2013). But most relevantly it delays perceived reinforcement for behaviours that lead to goal completion. If completing the goal is a reinforcer, then a delayed goal is a delayed reinforcer. This is potentially one reason it can be more effective to set small goals. Goal setting is not without its limits, achieving a goal is often not very rewarding, unless that achievement comes with other benefits (Hoyle, 2013; Dysvik, & Kuvaas 2013).
Heuristic reasoning theories
Theories about heuristic reasoning have emerged as alternative explanation of delay discounting (Marzilli, Ericson, White, Laibson, & Cohen, 2015). Present bias is the tendency to underestimate how much ones current opinion will change in time. A similar heuristic tendency is projection bias, which describes the use of the present to predict the future, e.g. "I feel good about this situation so I will feel good about it in the future" (Pychyl, & Flett, 2012; Marzilli et al. 2015). One way to reduce heuristic use is more logical, deliberated thinking (Pychyl, & Flett, 2012; Marzilli et al. 2015), this is consistent, this kind of thinking is often associated with a reduction in delay discounting, as seen in the theories above (Lin, & Epstein, 2014; Hakimi & Hare, 2015; Lattal, 2010).
Reducing the effects of delayed reinforcement
The effects of delayed reinforcements can be counteracted, in several ways, perhaps the simplest is to decrease exposure to immediately gratifying options. This has been shown to be an effective method of keeping to goal directed behaviours (Rozental & Carlbring 2014). The fact we choose immediate gratification over delayed gratification is irrelevant if individuals are not presented with the choice. A practical example of this is simply turning off notifications from social media while studying (Rozental & Carlbring, 2014).
The effects of delay discounting can also be counteracted by allocating a place designated for the desired behaviour (e.g., a gym for exercise, or a study for homework). If this area is associated with the task, thoughts will be task focused and less focused on other means of more gratification (Pychyl & Flett, 2012). Associating an area with task focused work, and not with immediate gratification increase cognition'sfocused on the delayed reward not the intimidate reward. A consistent finding between theories is that thinking about the delayed reward, increases resistance to the intimidate reward (Lin, & Epstein, 2014; Hakimi & Hare, 2015; Lattal, 2010)
Here are some slightly harder quiz questions related to multiple sections, If you're up for a challenge:
The reduction in motivation caused by delaying reinforcement, has far reaching consequences, and has been studied in fields a of economics and psychology for years, with a heavy focus on rewards. No theory has conclusively discovered the underliningmechanisms that cause delay discounting, as their are multiple theoretical explanations for each aspect of delay's reduction on motivation. It is possible that there is no signaller underlining mechanism but some combination of the theories mentioned above. Although, more research is needed before the effects of delayed reinforcement are full explored, there are still several practical applications of the theoretical knowledge we do have. One consistency between theories is that deliberation and thinking about the reinforcement increases motivation to obtain delayed reinforcement.
- Operant conditioning (Wikipedia)
- Achievement goal orientation and academic motivation (Book chapter, 2014)
- Self-Regulated Learning(2011) (Wikiversity)
- Emotional self-regulation (Book chapter, 2013)
Doughty, A. H., Galuska, C. M., Dawson, A. E., & Brierley, K. P. (2012). Effects of reinforcer magnitude on response acquisition with unsignaled delayed reinforcement. Behavioural processes, 90(2), 287-290. https://doi.org/10.1016/j.beproc.2012.02.016
Dysvik, A., & Kuvaas, B. (2013). Intrinsic and extrinsic motivation as predictors of work effort: The moderating role of achievement goals. British Journal of Social Psychology, 52(3), 412-430. https://doi.org/10.1111/j.2044-8309.2011.02090.x
Epstein, L. H., Jankowiak, N., Lin, H., Paluch, R., Koffarnus, M. N., & Bickel, W. K. (2014). No food for thought: moderating effects of delay discounting and future time perspective on the relation between income and food insecurity. The American journal of clinical nutrition, 100(3), 884-890. https://doi.org/10.3945/ajcn.113.079772.
Friese, M., Messner, C., & Schaffner, Y. (2012). Mindfulness meditation counteracts self-control depletion. Consciousness and cognition, 21(2), 1016-1022. https://doi.org/10.1016/j.concog.2012.01.008
Hakimi, S., & Hare, T. A. (2015). Enhanced neural responses to imagined primary rewards predict reduced monetary temporal discounting. Journal of Neuroscience, 35(38), 13103-13109. DOI: 10.1037/0278-7322.214.171.1241
Harrison, J. M. D., & McKay, R. T. (2013). Do Religious and Moral Concepts Influence the Ability to Delay Gratification? A Priming Study. Journal of Articles in Support of the Null Hypothesis, 10(1).
Hoyle, R. H. (2013). Handbook of personality and self-regulation. John Wiley & Sons.
Inzlicht, M., Schmeichel, B. J., & Macrae, C. N. (2014). Why self-control seems (but may not be) limited. Trends in cognitive sciences, 18(3), 127-133. https://doi.org/10.1016/j.tics.2013.12.009
Jimura, K., Chushak, M. S., & Braver, T. S. (2013). Impulsivity and self-control during intertemporal decision making linked to the neural dynamics of reward value representation. Journal of Neuroscience, 33(1), 344-3]57. https://doi.org/10.1523/JNEUROSCI.0919-12.2013
Killeen, P. R. (2011). Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid. Behavioural processes, 87(1), 57-63. http://doi.org/10.1901/jeab.2011.96-107
Kishinevsky, F. I., Cox, J. E., Murdaugh, D. L., Stoeckel, L. E., Cook, E. W., & Weller, R. E. (2012). fMRI reactivity on a delay discounting task predicts weight gain in obese women. Appetite, 58(2), 582-592. https://doi.org/10.1016/j.appet.2011.11.029
Kurth‐Nelson, Z., Bickel, W., & Redish, A. D. (2012). A theoretical account of cognitive effects in delay discounting. European Journal of Neuroscience, 35(7), 1052-1064. https://doi.org/10.1111/j.1460-9568.2012.08058.x
Lattal, K. A. (2010). Delayed reinforcement of operant behavior. Journal of the Experimental Analysis of Behavior, 93(1), 129-139. http://doi.org/10.1901/jeab.2010.93-129Lattal, K. A. (2010). Delayed reinforcement of operant behavior. Journal of the Experimental Analysis of Behavior, 93(1), 129-139. http://doi.org/10.1901/jeab.2010.93-129
Lee, N. C., Krabbendam, L., Dekker, S., Boschloo, A., de Groot, R. H., & Jolles, J. (2012). Academic motivation med]iates the influence of temporal discounting on academic achievement during adolescence. Trends in Neuroscience and Education, 1(1), 43-48. https://doi.org/10.1016/j.tine.2012.07.001
Lin, H., & Epstein, L. H. (2014). Living in the moment: effects of time perspective and emotional valence of episodic thinking on delay discounting. Behavioral neuroscience, 128(1), 12. https://doi.org/]10.1037/a0035705
Ludwig, V. U., Nüsser, C., Goschke, T., Wittfoth-Schardt, D., Wiers, C. E., Erk, S., ... & Walter, H. (2015). Delay discounting without decision-making: medial prefrontal cortex and amygdala activations reflect immediacy processing and correlate with impulsivity and anxious-depressive traits. Frontiers in behavioral neuroscience, 9. https://doi.org/10.3389/fnbeh.2015.00280
Marzilli Ericson, K. M., White, J. M., Laibson, D., & Cohen, J. D. (2015). Money earlier or later? Simple heuristics explain intertemporal choices better than delay discounting does. Psychological science, 26(6), 826-833. Retrieved from: https://doi.org/10.1177/0956797615572232
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004). Separate neural systems value immediate and delayed monetary rewards. Science, 306(5695), 503-507. https://doi.org/10.1126/science.1100907
Mischel, Walter; Ebbesen, Ebbe B.; Raskoff Zeiss, Antonette (1972). "Cognitive and attentional mechanisms in delay of gratification.". Journal of Personality and Social Psychology. 21 (2): 204–218. http://doi.org/10.1037/h0032198]
Pychyl, T. A., & Flett, G. L. (2012). Procrastination and Self-Regulatory Failure: An Introduction to the Special Issue. Journal of Rational-Emotive & Cognitive-Behavior Therapy, 30, 203-212. https://doi.org/10.1007/s10942-012-0149-5
Rozental, A., & Carlbring, P. (2014). Understanding and treating procrastination: a review of a common self-regulatory failure. Psychology, 5(13), 1488. http://dx.doi.org/10.4236/psych.2014.513160
Shoda, Y., Mischel, W., & Peake, P. K. (1990). Predicting adolescent cognitive and self-regulatory competencies from preschool delay of gratification: Identifying diagnostic conditions. Developmental psychology, 26(6), 978. http://doi.org/10.1037/0012-16126.96.36.1998
Solomon, B. G., Klein, S. A., & Politylo, B. C. (2012). The effect of performance feedback on teachers' treatment integrity: A meta-analysis of the single-case literature. School Psychology Review, 41(2), 160. Retrieved from: https://search.proquest.com/openview/ef7bee1e9f6ed71824ce0973f1767b25/1?pq-origsite=gscholar&cbl=48217
Stolarski, M., Bitner, J., & Zimbardo, P. G. (2011). Time perspective, emotional intelligence and discounting of delayed awards. Time & Society, 20(3), 346-363. https://doi.org/10.1177/0961463X11414296
Teuscher, U., & Mitchell, S. H. (2011). Relation between time perspective and delay discounting: A literature review. The Psychological Record, 61(4), 613. Retrieved from: https://search.proquest.com/openview/69ab2afb697327296982ec677c1062e1/1?pq-origsite=gscholar&cbl=41800
Schlam, T. R., Wilson, N. L., Shoda, Y., Mischel, W., & Ayduk, O. (2013). Preschoolers' delay of gratification predicts their body mass 30 years later. The Journal of pediatrics, 162(1), 90-93. http://doi.org/10.1016/j.jpeds.2012.06.049