Motivation and emotion/Book/2016/Delay discounting and emotion

Delay discounting and emotion:
What role do emotions play in delay discounting?

Multimedia presentation (3 min)

Overview

Choice is simple when options are only different on a single dimension. If someone was to be offered two monetary rewards that differed only in amount the individual would select the option that had more money. If the options were identical in all but the delay it would take to receive them an individual would select the option they would receive sooner. Multiple studies have shown that these principles apply both to animals and humans alike as both the higher reward option and the sooner reward option make both evolutionary and economic sense (Green & Myerson, 2004). This is easy to understand as from an evolutionary perspective having more or getting something sooner could mean a greater chance of survival, at least in the animal world. What would happen, however, if a reward with the longer delay was larger then the reward with less delay - is a bird in the hand is worth two in the bush?

Studies of delay discounting often focus on when the options or choices are different on two or more dimensions. For example, if an individual could receive either a smaller reward sooner now or a larger reward later. Which should the individual select? Which has more benefit? Which is needed more? According to Keeney & Raiffa (1993) (as cited in Green & Myerson, 2004) the decision becomes more difficult as an individual must weigh the options and make compromises among their preferences. Research into these multi-dimensional choices is important as it has implications for basic decision-making individuals would be undertaking everyday, worldwide. By researching why and how individuals make their decisions, such as in delay discounting, there is potential to understand what motivates choices and how to negate or reduce possible fallout.

Many people would recognise a period in their life where they forewent a larger reward later for a more immediate, smaller reward. This is the most basic premise of delay discounting. In essence, delay discounting represents the ‘decline in the perceived present value of a reward with delay to its receipt’ (Odum, 2011a). This means that outcomes or rewards that are remote in time tend to have less value then those that are more instantaneous. Studies have shown that, across populations, reward types and species value tends to decline hyperbolically with delay (Madden, Francisco, Brewer & Stein, 2011; Odum, 2011a; Weatherly & Feraro, 2011). Odum (2011a; 2011b) explains that the perceived value of a reward tends to decline steeply with shorter delays but more shallowly with much longer delays.

According to Odum (2011b), steep or extreme discounting of delayed outcomes is often associated with a range of social disorders. Furthermore, Odum (2011b) goes on to explain that the ‘degree of sensitivity to delayed outcomes may be a stable and pervasive individual characteristic’. This essentially means that the degree of sensitivity an individual has to the delay of a reward or outcome may very well be intrinsically linked to the individuals personality or even genetics as hypothesised by Mitchell (2011). Although Mitchell (2011) admits that research in the field of genetic heritability and its relationship to the degree to which delay discounting affects an individual, in relation to alcoholism, has not been explored deeply enough to conclusively prove a relationship exits literature and previous studies do suggest so. Mitchell (2011) asserts that a review of literature on both humans and animals suggests a genetic component to delay discounting.

Before Delay Discounting

The study of behaviour and its relation to time has been explored since early psychology. The early studies can be broken up into two distinct ‘traditions’ that had been explored across the twentieth century and can be viewed as the predecessor of delay discounting studies. The traditions can be categorised into operant conditioning and classical conditioning. The first major tradition was in relation to studies conducted by Thorndike (1898) and later Skinner (1936), who experimented with and analysed the control of behaviour by events that occur after it (Commons, et al., 2013). The second tradition “analysed the control and transfer of control by events that proceeded it” (Commons, et al., 2013). This tradition is largely attributed to Pavlov (1897) through to Watson (1913) along with some others. Though classical conditioning is linked to delay discounting and reward operant conditioning has more relevance to the subject. However both are important to the development of delay discounting.

First tradition: Operant Conditioning

Edward Thorndike

Thorndike was the first to be attributed to the experimentation with and analyse the control of behaviour by events that occur after it. The tradition can be traced back to Thorndike’s learning theory (1898) in which he placed cats inside a puzzle box that were then encouraged to escape in order to reach the reward (piece of fish) which had been placed outside. After the cat was placed in the box Thorndike would time how long it took them to escape and reach the reward (Chance, 1999). By 1905 Thorndike put forward the theory of “Law of Effect” stating that “The probability of a specified response occurring is a positive monotonic function of the probability of that response being reinforced” (as cited in Gray & Broembsen, 1976). In more simple terms Thorndike’s Law of Effect proposes that “any behaviour followed by pleasant consequences is likely to be repeated and any behaviour followed by unpleasant consequences is likely to be stopped” (McLeod, 2007). From this it evident that Thorndike’s research along with his Law of Effect focused the control of behaviour through the events that followed it.

Burrhus Frederic Skinner (B.F. Skinner)

Skinner was the second notable figure to explore the control of behaviour by the events that occurred after (Common, et al., 2013). Often regarded as the father of Operant Conditioning, Skinner based much of his model on Thorndike’s Law of Effect along with the work of Pavlov (McLeod, 2015; Weiten, 2013). Skinner believed the best approach to understanding behaviour is to explore at the causes of an action and its consequences. The fundamental principal being that ‘organisms tend to repeat those responses that are followed by favourable consequences’ (Weiten, 2013). This idea forms the foundation of Skinner’s Operant Conditioning.

Skinner proposed that the reinforcement principles of reward, extinction and punishment are responsible for much psychological and behavioural functioning that is developed (Peterson, 2010). According to Weiten (2013) “reinforcement occurs when an event following a response increases an organism’s tendency to make that response”. Skinner explored his model of behaviour through the introduction of his reinforcement principles on small animals, such as rats or pigeons. An animal was placed into a small chamber similar to Thorndike’s puzzle box, later termed ‘Skinner Box’, in which the animal could make a response that was recorded, the consequence of the response was then controlled in the form of reinforcement (McLeod, 2015; Weiten, 2013). He found that in most cases behaviour that is reinforced tends to be repeated more often and conversely behaviour that is not reinforced tends to be repeated less often (Weiten, 2013).

Second tradition: Classical Conditioning

Ivan Pavlov

Pavlov has long been acknowledged as the father of Classical Conditioning (Pavlovian Conditioning) and one of the founding members of modern experimental psychology (Corr & Perkins, 2006). During experiments into dog’s gastric systems in the 1890s, Pavlov unintentionally made the discovery of conditioned response. According to Cambiaghi and Sacchetti (2015), during his chronic experiments on the digestive system of dogs, Pavlov noticed that whenever the experimenter walked into the room the dogs would start salivating, regardless of whether the experimenter held food. Pavlov developed the first experimental model of learning wherein a neutral stimulus preceded the capacity for a specific response.

In 1902 Pavlov coined the terms conditional and unconditional reflexes as a result of his experiments (Cambiaghi & Sacchetti, 2015). In his experiments into the conditioned response, Pavlov used many neutral stimuli (originally produced no response) such as the lab assistant providing food or a bell. The experimenter would ring the bell every time food was presented. This was repeated multiple times and eventually rang the bell without providing food. Pavlov found that just the sound of the bell alone after the pairing was enough to stimulate the conditioned response of salivating (Cambiaghi & Sacchetti, 2015; Corr & Perkins, 2006; Weiten, 2013). The originally neutral stimulus had become a conditioned stimulus. This means that the dogs had created an association between the ringing of the bell and food and so was conditioned to a new behaviour.

In relation to his discovery of conditioning Pavlov formed the Law of Temporal Contiguity. This theory stated that for a strong association to be made the two stimuli must be experienced closely together in time. In his experiments he found that when the bell and the food were presented too far apart association would be less significant. This means that increasing the delay between conditioned stimuli and unconditioned stimuli impedes the learning of conditioned response (Allan, Tangen, Wood & Shah, 2003).

John Watson

In 1913 Watson introduced behaviourism as a movement in psychology expanding on the work of Pavlov into classical conditioning (McLeod, 2014). According to Watson all aspects of human psychology could be explained by the response to stimuli as is inherent to the classical conditioning approach. Watson theorised that everything from emotional response to talking were patterns of stimuli and response denying even the existence of consciousness (Lissek, et al., 2005). This would mean that any differences between individuals would be due to different experiences of learning/conditioning.

Watson expanded on the work of Pavlov, proving that not only animals could be conditioned but humans too. In 1920, Watson along with Reyner, conducted and experiment on child to induce a fear of small furry animals. Dubbed the ‘Little Albert Experiment’, Watson presented small furry animals (the stimuli) to ‘little Albert’ to ensure he was not already afraid. After establishing the control Watson and colleague began the experiment by presenting a white rat to the child and striking a metal bar with a hammer. The loud noise would startle the child into crying. This was repeated 7 times over 7 weeks to establish a conditioned response. The result found that presenting the rat alone was then enough to cause signs of fear in the child. Additionally it was found that ‘little Albert’ had also developed fear for other similar furry animals and objects (Powell, Digdon, Harris & Smithson, 2014).

Models of Delay Discounting

According to Odum (2011a), along with McKerchar and colleagues (2009), the term ‘discounting’ in psychology refers to a decrease in the perceived value of a reward as a result of increasing delay to its recept or decreasing probability. This means that when a individual is given choice between rewards that differ on delay they would be more likely to choose the one that can be delivered sooner, regardless of objective value (McKerchar, et al., 2009). In cases of delayed rewards the perceived value of the more delayed reward often is said to be ‘discounted’. According to McKercher and colleagues (2009), experimental models to determine the qualitative and quantitative nature of delay discounting typically change either the amount of the immediate reward or the time of the delayed reward. Studies have been conducted on both humans and animals and typically follow one of three major models of delay discounting. They are; ‘one-parameter exponential decay; one-parameter hyperbola; and the two-parameter hyperboloid in which the denominator is raised to a power’ (McKercher, et al., 2009). One-parameter exponential decay The model of one-parameter exponential decay, or exponential discounting, is an economic based model which predicts that individuals will have consistent preferences of time that are independent to the amount of reward. Researchers and economists using this method assume that discounting is exponential meaning that the subjective value of a reward decreases by a constant amount over time regardless of its objective value. The equation for one-parameter exponential decay looks like: “V=Ae−bD, V is the subjective value of future reward, A is the amount, D is the delay to its receipt, and b is a parameter that governs the rate of discounting” (Green & Myerson, 2004). This model is useful in explaining outcomes that may be disproportional to ‘normal’ patterns of delay discounting (McKercher, et al., 2009). One example used by Madden, Bickel and Jacobs (1999) is the models use in explaining why alcoholics choose to continue drinking over the reward of sustained sobriety. The economic theory of one-parameter exponential decay suggests that both the rewards of sobriety and the future costs of current drinking are ‘heavily discounted and therefore the personal value of the rewards and costs are minimalised relative to the immediate reward of alcohol which retains its objective value’ (Madden, et.al., 1999). This means that in cases where time is the important factor, whether due to time constraints or outliers such as addiction the subjective value of the more immediate reward is always disproportionately larger than larger future reward. If this model were to describe the choice process in delay discounting it would assume that waiting for a reward that is more delayed involves elements of risk. This would mean that each additional period of time would correlate to an equally additional portion of risk that may prevent receipt of the reward (Green & Myerson, 2004). A criticism of the one-parameter exponential decay model is its inability to account for or predict reversals in preferences, as that would defy the assumption of the model (Green & Myerson, 2004). This makes it more difficult to apply the model to humans then to animals. One-parameter hyperbola An alternative model of measuring delay discounting is by looking at discounting as a hyperbola. Typically more favoured by behaviour psychologists, the one-parameter hyperbola (or Mazur’s hyperbola) ‘predicts time-inconsistent preferences’ (Green & Myerson, 2004). This model has been evidenced in many studies, such as those of Ainslie and Haendel (1983) or Green and colleagues (1994) (McKercher, et al., 2009), to be useful in predicting the ‘time-inconsistant preferences’ in both animals and humans. The equation for one-parameter hyperbola looks like: “V=A/(1+kD), where V is the subjective value of the delayed reward, A is the amount of the delayed reward, D is the delay, and k is a parameter that reflects the discounting rate” (McKercher, et al., 2009) If this model were used to measure and provide a description of the delay discounting the results would indicate “choices between rewards available at different times are really choices between different rates of rewards” (Green & Myerson, 2004). This means that if the assumption is that value of the reward is directly proportional to the delay in time to receiving it then it can be predicted that the individual would be willing to wait. A criticism of Mazur’s one-parameter hyperbola is that it has a tendency to ‘over predict subjective value at shorter delays and over predict subjective values at long delays’ in humans (Green & Myerson, 2004; McKercher, et al., 2009). This causes problems and inconsistencies in results as researchers incorrectly overvalue or undervalue the reward based on the delay. Two-parameter hyperboloid in which the denominator is raised to a power The two-parameter hyperboloid proposed by Green, Fry and Myerson in 1994 (Green & Myerson, 2004; McKercher, et al., 2009) raises the denominator to a power (s). Many psychologists feel that using this model more adequately describes the delay discounting of rewards in humans. The two-parameter hyperboloid, where the denominator is raised to a power, looks like: ‘V=A/(1+kD)s, Where V is the subjective value of the delayed reward, A is the amount of the delayed reward, D is the delay, and k is a parameter that reflects the discounting rate” and “s may represent the nonlinear scaling of amount and/or time and is generally equal to or less than 1.0’ (Green & Myerson, 2004; McKercher, et al., 2009). Due to this model being two-parameter, meaning that it takes two characteristics into account when defining the effect of discounting, it will in most cases provide more validity when describing discounting in humans. According to and colleagues (2009) the equation functions by testing whether the ‘s parameter deviates significantly from 1.0’. Overall results from this model describes the delay discounting of rewards by humans more accurately then the single parameter models (Green & Myerson, 2004; McKercher, et al., 2009). Similar to the one-parameter hyperbola model this model describes the delay discounting as choices between different rates of reward. This means that “choices between rewards available at different times are really choices between different rates of reward” (McKercher, et al., 2009)[[example}}

The Role of Emotions on Delay Discounting

Emotions play a significant role in all decision-making situations. This is true too for situations involving delay discounting were the future reward becomes subjectively less valuable over longer delays. According to Lempert and colleagues (2012), the rate at which an individual may discount delayed rewards can vary as a function of both the given context and individual differences. Further to this, high delay discounting rates have been linked with negative behaviours such as gambling, drug abuse and alcoholism. The tendency to prefer immediate rewards over delayed ones can be maladaptive, as is the case when making decisions to forgo future health for immediate intoxication, or delaying the completion of an assignment to instead play games.

In the study conducted by Lempert and his colleagues (2012) into the effects of anticipatory stress on delay discounting they measured two factors of stress. They were future orientated stress, where the participants gave a speech about their future career, or present orientated stress, where the participants gave a speech about their current appearance. After their speeches, both conditions performed a delay-discounting task where they chose between a smaller, immediate reward and a large, delayed reward. The participants stress scores were also collected. Lampert and colleagues found that “under stressful conditions, delay discounting rate was highest in individuals with low trait perceived stress and lowest for individuals with high trait perceived stress“. This shows a significant correlation between feelings of stress and higher rates of delay discounting.

Another study conducted by Liu and colleagues (2013) focused on emotions and the effect of delayed rewards. By manipulation emotional valence through imaging events they found that positive emotions lead to a tendency of individuals to choose the larger but more delayed rewards. Conversely, induction of negative emotions made individuals choose smaller but more immediate reward. This finding is important as it leads to the exploration of emotion as a possible inhibiter or promoter of delay discounting. Further to this, the degree to which the emotions is altered (towards negative or positive) has shown to create relative change in the amount of discounting (Liu, et al., 2013).

A similar study was repeated by Worthy, Byrne and Fields (2014). They conducted two experiments to examine the role of worry and anxiety on prospection during decision making. They hypothesised that high levels of worry and anxiety would lead to enhanced preference for immediate reward over larger delayed reward. Results of their experiments indicated that those who reported higher levels of worry and anxiety gave greater weight to immediate rewards and that high levels of worry are closely associated with greater rates of delay discounting. These finding have been attributed to higher levels of worry making an individual more uncertain about the future and so more avert to delayed rewards (Worthy, et al., 2014).

Quiz

When are individuals more likely to accept a delayed reward over an immediate one?

	a) When the delay is long and the reward is not significantly higher
	b) When they are feeling happy
	c) When the delay is short and the reward is significant
	d) When they a worried
	e) Both b & c

Conclusion

Emotions play a large role on delay discounting. Negative emotions have the potential to cause higher rates of discounting in decision making and, conversely, positive emotions tend reduce the rates of delay discounting. This means that individuals suffering from negative emotions are more likely to prefer the objectively less valued, immediate reward and that individuals feeling positive emotions are more likely to select the objectively higher valued, delayed reward.

See also

• Temporal discounting (Wikipedia)
• Delayed reinforcement and motivation (Book chapter, 2016)
• Performance enhancing drug usage motivation in elite athletes (Book chapter, 2016)

References

Allan, L., Tangen, J., Wood, R., & Shah, T. (2003). Temporal contiguity and contingency judgments: A Pavlovian analogue. Integrative Physiological & Behavioral Science, 38, 214-229. doi: 10.1007/BF02688855

Cambiaghi, M., & Sacchetti, B. (2015). Ivan Petrovich Pavlov (1849-1936). Journal of Neurology, 262, 1599-1600. doi: 10.1007/s00415-015-7743-2

Chance, P. (1999). Thorndike’s puzzle boxes and the origins of the experimental analysis of behaviour. Journal of the Experimental Analysis of Behaviour, 72, 433-440. doi: 10.1901/jeab.1999.72-433

Commons, M., Mazur, J., Nevin, J. & Rachlin, H. (2013). The Effect of Delay and of Intervening Events on Reinforcement Value: Quantitative Analyses of Behavior, Volume 5. Oxford, England: Routledge

Corr, P., & Perkins, A. (2006). The role of theory in the psychophysiology of personality: From Ivan Pavlov to Jeffrey Gray. International Journal of Psychophysiology, 62, 367-376. doi: 10.1016/j.ijpsycho.2006.01.005

Gray, L., & Broembsen, M. (1976). On the generalizability of the law of effect: Social psychological measurement of group structures and process. Sociometry, 39, 175-183. doi: 10.2307/2786510

Green, L., & Myerson, J. (2004). A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin, 130, 769-792. doi: 10.1037/0033-2909.130.5.769

Lempert, K., Porcelli, A., Delgado, M., & Tricomi, E. (2012). Individual differences in delay discounting under acute stress: The role of trait perceived stress. Frontiers in Psychology, 3, 1-10.

Lissek, S., Powers, A., McClure, E., Phelps, E., Woldehawariat, G., Grillon, C., & Pine, D. (2005). Classical fear conditioning in the anxiety disorders: A meta-analysis. Behaviour Research and Therapy, 43, 1391-1424. doi: 10.1016/j.brat.2004.10.007

Liu, L., Feng, T., Chen, J., & Li, H. (2013). The value of emotion: how does episodic prospection modulate delay discounting?. PLoS One, 8, 1-7. doi: 10.1371/journal.pone.0081717

Madden, G., Bickel, B., & Jacobs, E. (1999). Discounting of delayed rewards in opioid-dependent outpatients: Exponential or hyperbolic discounting functions?. Experimental and Clinical Psychopharmacology, 7, 284-293. doi: 10.1037/1064-1297.7.3.284

Madden, G., Francisco, M., Brewer, A. & Stein, J. (2011). Delay discounting and gambling. Behavioural Processed, 87, 1-9. doi: 10.1016/j.beproc.2011.01.012

McKerchar, T., Green, L., Myerson, J, Pickford, T., Hill, J., & Stout, S. (2009). A comparison of four models of delay discounting in humans. Behavioural Processes, 81, 256-259. doi: 10.1016/j.beproc.2008.12.017

McLeod, S. (2007). Edward Thorndike. Retrieved from Simply Psychology website http://www.simplypsychology.org/edward-thorndike.html

McLeod, S. A. (2014). Classical Conditioning. Retrieved from Simply Psychology website www.simplypsychology.org/classical-conditioning.html

McLeod, S. (2015). Skinner – Operant conditioning. Retrieved from Simply Psychology website http://www.simplypsychology.org/operant-conditioning.html

Mitchell, S. (2011). The genetic basis of delay discounting and its genetic relationship to alcohol dependence. Behavioural Processed, 87, 10-17. doi: 10.1016/j.beproc.2011.02.008

Odum, A. (2011a). Delay discounting: I’m a k, you’re a k. Journal of the Experimental Analysis of Behaviour, 96, 427-439, doi: 10.1901/jeab.2011.96-423

Odum, A. (2011b). Delay discounting: Trait variable. Behavioural Processes, 87, 1-9. doi: 10.1016/j.beproc.2011.02.007

Peterson, C. (2010). Looking forward through the lifespan: Developmental psychology (5th ed.). NSW, Australia: Pearson

Powell, R., Digdon, N., Harris, B., & Smithson, C. (2014). Correcting the record on Watson, Rayner, and Little Albert: Albert Barger as “Psychology’s lost boy”. American Psychologist, 69, 600-611. doi: 10.1037/a0036854

Weatherly, J. & Feraro, F. (2011). Executive functioning and delay discounting of four different outcomes in university students. Personality and Individual Difference, 51, 183-187. doi: 10.1016/j.paid.2011.03.042

Weiten, W. (2013). Psychology: Themes and variations (9th ed.). Belmont, CA: Cengage Learning

Worthy, D., Byrne, K., & Fields, S. (2014). Effects of emotion on prospection during decision-making. Frontiers in Psychology, 5, 1-12. doi: 10.3389/fpsyg.2014.00591