What is the term for the rules that determine whether responses lead to the presentation of a reinforcer?

IV.F. Premack's Principle

Premack's principle (or the differential probability hypothesis) refers to reinforcing a target behavior by awarding some privilege to engage in a more desired behavior afterward. For example, a child may be told that he or she can have dessert after eating his or her vegetables, or be told that he or she can play baseball after completing a certain chore. The more highly desirable behavior is used to entice a person to engage in a usually productive, albeit less enjoyable, activity. Typically, Premacks principle is used by parents in addressing their children, and therapists often build this principle into their parent-training regimens. Premack's principle is used loosely in other clinical settings as well. For example, a person who successfully loses weight may then be allowed to shop for new clothes.

Premack's principle is often extended in parent-training programs to encourage appropriate child behavior and discourage inappropriate behavior. For example, parents may establish a set of house rules that carry rewards if followed and punishments if not. The establishment of this consequence system is often linked to other areas of parent training such as modifying parental commands, setting up daily routines, and increasing supervision of the child.

1.03.3.5 The Response Deprivation Hypothesis

There is evidence that, contrary to the Premack principle, low-probability responses can reinforce higher-probability responses. It is a matter of depriving the animal of the less-probable response. For example, if ingesting X is less probable than ingesting Y, the tendency to ingest Y can be increased above its baseline by depriving the animal of X and making the availability of X contingent on ingesting Y (e.g., Timberlake and Allison, 1974; Allison, 1989). There are many interesting applications of the response deprivation hypothesis. For example, 3-year-old children who loved to run around and scream were induced in a fairly short time to sit still, a not highly probable activity for 3 year olds. Essentially, after the children sat quietly for a specified time period, they were allowed to run and scream for a few minutes (Homme et al., 1963).

The response deprivation hypothesis allows us, like the Premack principle, to determine in advance how reinforcing some activity might be. It also allows us to adjust reinforcers to particular animals or people. For example, a person who enjoys TV will perform other behaviors if those behaviors allow TV viewing, which is otherwise reduced. But if a person enjoys reading more than TV watching, let reading be contingent on TV watching.

A more subtle implication of the response deprivation hypothesis bears mention. Normally we think of instrumental behavior such as bar pressing as different from consummatory behavior such as eating. This distinction is violated by the response deprivation hypothesis. That is, any behavior can serve as a reinforcer if it is made to occur below its normal baseline. Animals deprived of food will run in order to eat, and an animal deprived of activity will eat in order to run.

III.C. Backup Reinforcers

The backup reinforcers are the natural reinforcers that give value to the tokens. The token economy should not deprive any clients of anything to which they are entitled by the Constitution or by statute. The backup reinforcers therefore are additions to these legally mandated minimums. They may involve access to additional amounts or enhancements of entitlements, such as additional exercise time or special foods, as well as anything to which the client is not legally entitled. In general, backup rein-forcers fall in several categories. These consist of edibles or consumables, such as special food or drink; activities, such as extra recreational or computer game time; material objects, such as special athletic shoes or posters; and independence, such as reduced supervision or movement through progressively less restrictive levels in a structured levels program. Each can be considered a to-be-earned privilege.

A number of strategies may be used to identify backup reinforcers. The most straightforward consists of asking the clients what they would like and would work for. Another is based on the Premack principle and involves observing clients during free or unstructured time to identify the things they are most likely to do. These may then be used as backup reinforcers for the typically lower probability target behaviors. Finally, the staff can be asked what they believe will serve as backup reinforcers for their clients. It is important to note, however, that the information gathered through these strategies results only in the identification of potential backup reinforcers. The reinforcing properties of the potential backup reinforcers must be tested to determine whether they are true backup reinforcers. This is done by assessing whether clients will engage in target behavior to earn tokens and then exchange the tokens for the potential backup reinforcers.

What is reinforcing to one client may or may not be available to another client. The token economy must therefore offer a variety of backup reinforcers to ensure that there are sufficient reinforcers for all participants. Although some backup reinforcers may motivate most or all clients, additional backup reinforcers that function as such for only a few or, perhaps, only one client should also be made available to ensure that the unique interests of all clients are addressed. The backup reinforcers should be available to clients on a regular basis. Some should be available several times a day, others several times a week, and perhaps still others several times a month or only on a monthly basis. The schedule of availability of backup reinforcers is dictated, in part, by the characteristics of the clients, and a token economy cannot err in the direction of making backup reinforcers too frequently available to clients. Finally, the backup reinforcers must be available only through the token economy. The availability of “bootleg” backup reinforcers, that is, backup reinforcers that may be obtained through other means, will dilute or negate the reinforcing properties of backup reinforcers and should be prevented.

Application of operant learning

Many interventions are based on operant models of psychopathology. Operant-based interventions are exemplified by identifying reinforcers; teaching new behaviors; increasing existing adaptive behaviors with reinforcement including concurrent schedules of reinforcement; decreasing undesirable behaviors with reinforcement of other behavior, punishment, and extinction; and manipulating motivating operations for reinforcers and punishers.

5.05.5.1.1 Systematic Use of Reinforcement

The most straightforward procedure for strengthening behavior is simply to wait for the behavior to occur, and then to reinforce it. However, there are some important issues related to this seemingly straightforward process. Perhaps the most important of these is to select an appropriate reinforcer. It must be remembered that a reinforcer is defined in terms of its effect on the behavior it is contingent upon. Clinicians can err in what they think will be reinforcing for particular children. The information collected in the assessment phase is critical for the right choice of an effective reinforcer, and this is not always an easy task. Some clinicians develop what are termed reinforcement menus, or lists of things that are potentially reinforcing, and then have the child or caregiver select from the menu. Another very helpful hint is to use what is known as the Premack principle. Premack (1959) noted that the activities children chose to engage in during their free time could be used to reinforce other, less-preferred activities. He technically defined this as using higher-probability behaviors to reinforce lower-probability behaviors. The importance of the principle is that if you are having trouble identifying what is reinforcing for children, observe what they do in their free time.

Reinforcers vary in nature and can be classified in a number of ways. One of the earliest distinctions made was between primary and secondary reinforcers. A primary reinforcer, sometimes referred to as an unconditioned reinforcer, is one that does not depend on learning to have taken place for it to develop its reinforcing properties. Food, water, and warmth are examples of such primary reinforcers. In contrast, learning has been involved in secondary or conditioned reinforcers developing their reinforcing properties. For example, some children find playing games on a computer to be very reinforcing. For this to have happened, some learning about the computer and the particular game must have taken place. It can be argued that there is limited clinical usefulness in the distinction between primary and secondary reinforcers as ethical considerations dictate that children have a fundamental right to unrestricted access to most primary reinforcers, and hence they should not be used in ABA programs. It is the case, however, that edible reinforcers such as candy have been used successfully to motivate children who do not respond to secondary reinforcers.

A second distinction that is made in regard to types of reinforcers is that between naturally occurring and artificial reinforcers. A naturally occurring reinforcer is one that is readily available and typically used in the setting in which the ABA program is being conducted. In most homes, reinforcers such as parent attention, pocket money, and access to television are available to children. Unfortunately, these reinforcers are often given to the children in an unsystematic or unplanned manner. An artificial reinforcer is one that is not usually available in the setting, but is brought in for the purposes of the program. Candy is not usually used as a reinforcer in most classrooms but could be introduced for use in a program for a special purpose. It is generally advisable for clinicians to use artificial reinforcers only when absolutely necessary, as the fading of such reinforcers can be more difficult than is the case for naturally occurring reinforcers.

A third distinction that is made is between extrinsic and intrinsic reinforcers. An extrinsic reinforcer is one that is quite separate from the behavior that it is reinforcing in that the behavior could occur and not be followed by the reinforcer. If a clinician is using praise to reinforce a child's behavior, the behavior can be allowed to occur without the reinforcement being administered. An intrinsic reinforcer can be viewed as one that is actually part of the behavior. The behavior is reinforcing by itself, a process often referred to as automatic reinforcement, and does not need additional reinforcement delivered by another person to maintain it. For most individuals no additional reinforcement is required to keep them watching their favorite show on television. They do it because they like it. The distinction between intrinsic and extrinsic reinforcement can be clinically important. Some behaviors, for example reading, may initially need to be extrinsically reinforced to get them going, but once established, can be maintained by the enjoyment they produce.

Yet another way of distinguishing among reinforcers is to classify them based on the broad type of reinforcer. The types listed can vary among authors in the field, but a common list includes social, food, activity, and token reinforcers. Social reinforcers include praise, hugs, or attention from another person. As the name suggests, food reinforcers are things the child can eat. Rather than being basic food requirements, which, as mentioned earlier, are problematical from an ethical viewpoint, they are special treats that the person does not have regular access to. Activity reinforcers are those things that the person likes to do, for example, playing games and watching television. Finally, token reinforcers do not have value in themselves but derive their value from what they can be exchanged for. Many classroom teachers have children earning points for completion of academic work, and then allow the children to exchange those points for back-up reinforcers such as extra time at recess or reduced homework. Money is certainly the most frequently used token reinforcer.

A final point to be made in regard to the systematic use of reinforcement to strengthen a target behavior is that the fundamental principles derived from learning theory need to be adhered to. Reinforcement usually needs to be immediate, particularly with children at a low developmental level. If another behavior occurs between the target behavior and the delivery of the reinforcement, the more recent behavior might be the one that is strengthened. The second behavior under these circumstances is usually referred to as superstitious behavior. Also, attention needs to be paid to what is known about schedules of reinforcement. A continuous schedule needs to be used to maximize the rate of development of the new behavior, and then there should be a move to an intermittent schedule, referred to as a thinning of the reinforcement, to make the behavior more resistant to extinction.

Theoretical approaches to reward in food acceptance

The idea that contingent rewards increase performance of the rewarded behaviour is the central tenet of instrumental learning (Thorndike, 1911). Incentive effects on performance are equally fundamental to classical economic accounts of human behaviour (Benabou & Tirole, 2003). The Premack principle offers another way to understand improvements in performance, predicting that higher probability behaviours will reinforce lower probability behaviours. Classical learning may also affect performance through conditioned preferences and expectations (Baeyens, Eelen, Crombez, & Van den Bergh, 1992), ultimately resulting from the rewarded food taking on the positive motivational properties of the reward (Bindra, 1978; Toates, 1986). On the face of it, there is no reason to suppose that rewarding children for eating should not have these same positive effects.

However, social psychologists and economists have observed a paradoxical effect of rewards in some situations. Behaviour is believed to be determined by the combination of intrinsic motivation (e.g. enjoyment of the task) and extrinsic motivation (e.g. incentives) (Ryan & Deci, 2000). A meta-analysis of reward studies indicated that under certain conditions, external rewards may actually reduce intrinsic motivation (Deci, Koestner, & Ryan, 1999); indicated by a decrease in performance of or enjoyment of the behaviour when rewards are withdrawn (Cameron & Pierce, 1994). In the classic study, preschoolers who were promised a reward for drawing with magic markers showed less intrinsic motivation for drawing at the end of the reward period than their peers who had received either an identical but unexpected reward or no reward (Lepper, Greene, & Nisbett, 1973). Economists have also noted that offering a reward for pro-social behaviour can reduce compliance to below the spontaneous level; for example, local objection to the construction of a nuclear waste depository was running at 49% before incentives were offered and at 75% after the Swiss Government's offered financial compensation. Increasing the amount of offered compensation had no influence on rates of objection (Frey & Oberholzer-Gee, 1997). This has been termed ‘Motivation Crowding’ (Frey & Jegen, 2001).

Adverse effects of reward have been explained by social psychologists both in terms of ‘over-justification’ theory (Lepper et al., 1973) and self-determination theory (Deci et al., 1999). Both propose that rewarding people for doing something they already enjoy undermines intrinsic motivation and creates conditions for a net decline in motivation when rewards are subsequently withdrawn. Over-justification theory explains it in terms of the individual attributing their behaviour to the external reward and therefore effectively overlooking the intrinsic pleasantness of the task. Self-determination theory focuses on external rewards compromising feelings of competence and autonomy.

Applying over-justification theory in the food context, and construing liking as a form of intrinsic motivation, suggests that while incentives for eating less palatable foods may increase liking, incentives for eating palatable foods should show an adverse effect. Premack's principle makes similar predictions about rates of behaviour. If consumption of the rewarded food is already a highly probable behaviour then it is not likely that it will be reinforced by other behaviours. Conversely, if consumption of a disliked food or ‘lower probability’ behaviour is reinforced with a liked activity, an increase in response can be expected. Classical conditioning offers another possible explanation for drops in preferences for the rewarded food: if being rewarded for a task is experienced as negative (e.g. if the reward is enjoyed less than the rewarded task) then the task itself may take on these hedonic properties and become less liked. It would seem that in both operant and classical conditioning, the initial preference for the rewarded food and the reward itself are important factors in determining a positive outcome.

To some extent that is how the experimental literature plays out. The early studies rewarded consumption of novel but palatable food or drinks (Birch et al., 1982; Mikula, 1989; Newman & Taylor, 1992), and typically found that liking decreased. In contrast, the more recent school studies have rewarded consumption of less palatable foods (often vegetables) and found increased liking (Hendy et al., 2005; Horne et al., 2004; Lowe et al., 2004, 2006).

Simple sensory over-satiation may also explain these effects (Hendy et al., 2005; Rolls, Rolls, Rowe, & Sweeney, 1981). If the offer of rewards pushes consumption of a single food beyond the point of satiation, it can produce negative associations with the target food and hence negative shifts in preference. This idea was tested by Birch et al. (1984) who found no additional negative effects of rewards when children were required to consume their target drink beyond satiation. It is also noteworthy that researchers have avoided over-satiation by typically rewarding consumption of fairly small quantities of foods.