VI. SCHEDULES
(as of 10/5/17)

A. THE ISSUES AND DAILY LIFE ILLUSTRATION: Many times we would like a behavior to occur when we will not be there to reinforce it every time. By systematically introducing intermittent schedules of reinforcement, i.e. not reinforcing every response, a greater stability in responding can be produced.

B. DEFINITION: Continuous reinforcement was the customary approach in early learning studies. Skinner describes the start of his interest in the area of schedules of reinforcement. "One pleasant Saturday afternoon I surveyed my supply of dry pellets, and appealing to certain elementary theorems in arithmetic, deduced that unless I spent the rest of the afternoon and evening at the pill machine (this was the device used at that time to make rat pellets) the supply would be exhausted by 10:30 Monday morning. I decided to reinforce a response only once every minute and to allow all other responses to go unreinforced. There were two results: (1) my supply of pellets lasted almost indefinitely and (2) each rat stabilized at a fairly constant rate of responding." (P. 226). Thus began the formal study of schedules of reinforcement.

Q-1. Operant conditioning is based on having an emitted response followed by a _______.

Answer. (CONSEQUENCE)

Q-2. So far, we have talked about reinforcement for each response, or continuous reinforcement. When each response is reinforced, we call it _______ reinforcement.

Answer. (CONTINUOUS)

Q-3. When we set up a table of the times when we are going to reinforce the subject, we call it a "schedule" of reinforcement. A table telling when reinforcement should be given is called a _______.
(SCHEDULE OF REINFORCEMENT)

Q-4. The fastest way to teach a behavior is to reinforce every correct response. This schedule of reinforcement would be called _______ reinforcement.

Answer. (CONTINUOUS)

In maintaining a behavior, however, it is impractical to use continuous reinforcement. If, for example, we have taught a rat to press a bar by reinforcing it with food each time he/she responds, as we attempt to maintain the behavior in this way, two things will happen — the rat will quickly become satiated so that the food is no longer an effective reinforcer, and we will spend a fortune on rat pellets. For this reason, we use INTERMITTENT scheduling.

Q-5. Reinforcing only some of the responses of a subject is called _______ reinforcement.

Answer. (INTERMITTENT)

One form of intermittent scheduling is reinforcing for the number of responses a subject makes. This creates a numerical relationship of responses to reinforcements, called a RATIO. We therefore call such a schedule of reinforcement a RATIO SCHEDULE.

Q-6. Reinforcing a rat for the number of responses he makes is called _______ schedule of reinforcement.

Answer. (RATIO)

Sometimes we set or "fix" a specific number of responses (say 10) which a subject must make before it is reinforced. An example, Richard X., a factory worker who is paid on a piecework basis, that is, he is paid for each 10 articles he produces. This is called a fixed-ratio schedule of reinforcement.

Q-7. A schedule of one reinforcement for each 20 responses is called a _______-_______ schedule of reinforcement.

Answer. (FIXED-RATIO)

Q-8. When the ratio of responses to reinforcement is the same each time, we call it a _______-_______ schedule of reinforcement.

Answer.(FIXED-RATIO)

Sometimes we decide upon an average number of responses for which the rat should be reinforced and vary the number of responses for which she/he is reinforced. This is called a variable-ratio schedule.

Q-9. Suppose we decide to reinforce, on the average, every 10th response, but reinforce on the basis of every 5, then every 10, then every 15 responses (that is, an average or a mean of every 10 responses). This would be called a _______-ratio schedule of reinforcement.

Answer. (VARIABLE)

Q-10. Reinforcement for a set number of responses is a _______ schedule of reinforcement.

Answer. (RATIO)

Q-11. Reinforcement for the same number of responses each time is called _______-_______ reinforcement.

Answer. (FIXED-RATIO)

Q-12. Reinforcement for varying numbers of responses which average a specific number is called _______-_______ reinforcement.

(VARIABLE-RATIO)

Another basis for reinforcement schedules is the period or interval of time between reinforcement. In interval scheduling, we reinforce the subject for the first response after an interval of time has passed since the last response.

Q-13. Reinforcement on the basis of time is called _______ scheduling.

Answer. (INTERVAL)

There are two types of interval scheduling. The simpler, called fixed-interval schedules, involves reinforcing the subject for the first response after a set or fixed period of time has passed since the last reinforcement occurred. The subject is not reinforced for any responses she/he makes before the interval is over.

Q-14. When a subject is reinforced for the first response after a set interval of time has passed, the required interval being the same after each reinforcement, we say we have a _______-interval schedule.

Answer. (FIXED)

If the interval of time between possible reinforcements varies around some average point, this type of schedule is called a variable-interval schedule.

Q-15. When we reinforce a subject for the first response after 5, 10, 15, minutes respectively, (an average of every 10 minutes), we are using a _______-_______ schedule.

Answer. (VARIABLE-INTERVAL)

Q-16. A table giving the rate of reinforcement is called a _______ of reinforcement. When this schedule is based upon the number of responses between reinforcements it is called a ratio schedule.

Answer. (SCHEDULE)

Q-17. If the number of responses is the same between each reinforcement, we call this a_______-_______ schedule.

Answer. (FIXED-RATIO)

Q-18. If we vary the number of responses between reinforcement around some average we are on a _______-_______ schedule.

Anser. (VARIABLE-RATIO)

Q-19. If reinforcement is dependent upon time between reinforcements, we call it an _______ schedule.

Answer. (INTERVAL)

Q-20. If an interval schedule involves the same period of time between opportunities for reinforcement, we call it a _______-_______ schedule.

Answer. (FIXED-INTERVAL)

Q-21. When, however, the period of time between opportunities for reinforcement is varied around some average, we call this a _______-_______ schedule.

Answer. (VARIABLE-INTERVAL)

22-27, STATE THE TYPE OF SCHEDULE FOR EACH EXAMPLE

Q-22. The frequency of reinforcement is dependent upon time.

Answer. (INTERVAL)

Q-23. Reinforcement depends upon the average number of responses.

Answer. (VARIABLE-RATIO)

Q-24. Reinforcement occurs following each response.

Answer. (CONTINUOUS REINFORCEMENT)

Q-25. A reinforcement opportunity occurs after every twelve seconds.

Answer. (FIXED-INTERVAL)

Q-26. A reinforcement opportunity occurs on the average after every twelve seconds.

Answer. (VARIABLE-INTERVAL)

Q-27. Reinforcement is a function of the number of responses emitted.

Answer. (RATIO)

Through the study of reinforcement schedules, it was noticed that each schedule produces a certain characteristic response pattern. Further, these patterns were found to apply to different species and to varying behavior situations. The dependability of these relationships has made possible some generalization of the findings, and thus the control of schedules has been recognized to be of considerable importance.

Let us first consider some fundamental aspects of schedules. When an organism is being reinforced for every response, and then the reinforcement is stopped, the next response is not followed by reinforcement. If this continues, the organism will soon stop producing that behavior. This process is referred to as extinction of the response. On the other hand, if an organism is on a fixed ratio of ten, reinforcement occurs only each tenth time the behavior is performed. For extinction to begin, there will be at least ten responses before a change in the usual reinforcement schedule is apparent. Considering this, you might not be surprised to read that experimental evidence has shown that a behavior that is reinforced intermittently does not extinguish as fast as behavior that is reinforced continuously. Consequently, to condition an organism to respond for a longer period of time without reinforcement, and also without extinction, some form of intermittent reinforcement is more effective than is continuous reinforcement.

Q-28. What type of reinforcement schedule is best if you do not want a certain behavior to extinguish?
a. reinforcing every response
b. no reinforcement
c. intermittent reinforcement

Answer. (C)

A second point involves the fact that in ratio schedules, the subject (for example, Bob) can adjust when he gets reinforced. If Bob completes his ratio quickly he gets reinforced sooner.

Q-29. The faster the subject responds, the _______.

Answer. (SOONER THE REINFORCER)

Q-30. Ratio schedules are characterized by a _______ rate of response.
a. high
b. low

Answer. (A)

In interval schedules it makes no difference how fast the organism works. They only get reinforced when the period is up and a response follows.

A third useful aid in studying the characteristics of schedules of reinforcements is that the organism has no way of learning when a reinforcer will be delivered in variable-ratio or variable-interval schedules of reinforcement. The result of this is that steady rates of response are produced on both variable-interval and variable-ratio schedules.

Q-31. Considering what you learned earlier in this unit, every thing else being equal, which of these two schedules – variable-interval or variable-ratio – would produce a higher rate of responding?
a. variable-ratio (VR)
b. variable-interval (VI)

Answer. (A)

Let us take each schedule and discuss it separately. The organism learns a time discrimination in Fixed-Interval (e.g., a second reinforcement does not immediately follow the first reinforcer–in VI it does sometimes). This discrimination, without a pronounced external cue, usually is not perfect. As a result, as the time for reinforcement comes closer, the rate of behavior generally increases until it is very high at the time of reinforcement. (Graphically this looks like a scallop.)

Q-32. Which of the following would represent a fixed interval schedule?
A)                                                             B)


C)                                                             D)

Answer. (A)

Reinforcement which is presented following a response after a set time has elapsed since the last reinforcement is a definition of the schedule of reinforcement called fixed-interval.

Pay by the hour is not an exact equivalent to fixed-interval reinforcement. Why? In FI, the reinforcer follows the first response given after the time has elapsed. Pay by the hour requires only the passage of time, without reference to time between responses.

Q-33. Where the recorded behavior pattern is in the form of a fairly smooth accelerating scallop (like an ocean wave), we most likely are observing a schedule of
a. fixed-ratio
b. fixed-interval
c. variable-ratio
d. mixed-interval

Answer. (B)

Q-34. "Scallops" are characteristic of the schedule termed:_______-_______

Answer. (FIXED-INTERVAL)

Reinforcement is presented following a response after a varying time has elapsed since the last reinforcement in variable-interval.

Q-35. The kind of reinforcement that best applies to an Indian Rain Dance is _______.
a. continuous reward
b. variable-interval
c. variable-ratio
d. fixed-interval

Answer. (B)

Q-36. The legendary perseverance of salesmen may illustrate the resistance to extinction created by _______ schedules (assuming, that, on the average, sales will occur every so many calls).
a. fixed-ratio
b. variable-ratio
c. fixed-interval
d. variable-interval

Answer. (B)

Q-37. Patriots may have undergone long periods of non-reinforcement but still persist in their behavior. The performance of this response might suggest that the behavior was built on _______ schedules of reinforcement.
(INTERMITTENT)

Q-38. Which of the prior diagrams, from Q-32, represents a variable-ratio schedule?

(D)

Q-39. variable-interval schedule?
(C)

Q-40. The schedule which produces a pattern showing a slow rate of responding after each reinforcement with the rate of responding increasing until the next reinforcement, is the _______-_______ .

(FIXED-INTERVAL)

Q-41. The fixed-interval schedule characteristically produces a slower rate after each reinforcement, and the rate of responding _______ to a maximum rate until the next reinforcement.

(INCREASING)

Q-42. Because there is no cue to indicate the length of the interval preceding the next reinforcement, a _______ schedule produces a steadier rate of responding than does a fixed-interval.

(VARIABLE-INTERVAL)

Q-43. A pigeon learning to peck on a variable-ratio receives access to a small quantity of grain as a reinforcement. Once this conditioned operant is established, the pigeon will _______.
a. peck intermittently
b. peck even under punishment
c. peck at a high, steady rate

Answer. (C)

Q-44. A high rate of responding is generated by a _______ schedule.

Answer. (RATIO)

Q-45. The persistence of some gambling behavior is most likely to be related to _______.
a. fixed-interval reinforcement
b. fixed-ratio reinforcement
c. variable-ratio reinforcement
d. variable-interval reinforcement

Answer. (C)

Q-46. A variable-ratio schedule produces a high rate of responding with no _______ after reinforcement.

Answer. (DECREASE OR PAUSE)

Q-47. Availability of reinforcement depends on the passage of time in _______ schedules, and on the number of responses in _______ schedules.

Answer. (INTERVAL, RATIO)

Q-48. Ratio schedules may be either fixed or _______.

Answer. (VARIABLE)

Q-49. Since the subject has no cue when reinforcement will occur, a _______-ratio schedule produces a high, steady rate of responding without a pause or decrease.

Answer. (VARIABLE)

In a fixed-ratio schedule the organism is reinforced only after a fixed number of responses, for example five, have been performed. Once the organism begins to respond he works at the highest rate until reinforcement occurs (following reinforcement a pause, or decrease in rate of performance), occurs.

Q-50. A schedule of reinforcement which produces a fast, steady rate of responding and a pause after reinforcement is the
a. fixed-ratio
b. fixed-interval
c. variable-interval

Answer. (A)

Q-51. The schedule which is characterized by a high rate of response, with pause following reinforcement, is called _______.

Answer. (FIXED-RATIO)

Abrupt increases in the size of a fixed-ratio can lead to ratio strain, i.e. extended pausing after reinforcement, and often a breakdown in performance.

C. Applications:

1. Gambling: A study of schedules of reinforcement can help us to explain not only the causes of many sought after behaviors, but also the causes of many undesirable behaviors. Consider for example the pathological gambler. He/she appears to be acting against his/her own best interests. It is sometimes said that he/she is masochistic, i.e. he/she has a need for self-punishment. Given a particular behavioral history, stimuli which are usually punishing may act as positive reinforcers. As we shall see later in this set of materials, experiments have been done which demonstrate such processes without reference to inner states. A different explanation can more plausibly account for the behavior of the pathological gambler.

It seems likely that a gambler is a victim of a high variable-ratio schedule. Perhaps when he/she began to gamble he/she won several large sums in a row. Then, as he/she continued to gamble as a result of these reinforcements, winnings gradually became smaller and less frequent until now his/her gambling behavior is being maintained at a high rate by infrequent and small reinforcements. A thorough knowledge of schedule effects should help us to successfully treat this and many other behaviors.

2. Hospitalized Patients: Narrol's program applied schedules of reinforcement with alcoholics. They first worked on the ward at a hundred points per hour. One point equaled one penny. This did not maximize the principles of reinforcement. Rather, a "piecework" system was introduced; thus, if the alcoholic was satisfactorily shaven, helped in the care of individual B, etc., the person received points for each. The amount of time taken by the individual to perform the particular task is up to him/her, but the pay-off comes only when the task is performed. The quicker he does it, the quicker he/she might get to another task which would be equally lucrative.

D. Implication:

1. Adolescence and Schedules of Reinforcement: Changes in the kind and amount of behavior required for reinforcement occur in adolescence. A 14-year-old may have to do more than exist for his allowance. A 17-year-old, named Pat, may have to earn his money on quite an elaborate schedule of reinforcement. At school, where assignments used to be frequent and short with a lot of feedback, the older adolescent may have to work for a long time before he finds out how he is doing. The adolescent must perform a number of responses before his/her behavior is reinforced.

2. Scientists and Schedules of Reinforcement: The dedicated scientist is on a high variable-ratio schedule. He/she works for many months, sometimes years, on each experiment. And many of his/her experiments are not successful. When we consider that most scientific behavior has been adjusted to such a schedule more or less by accident, it is no wonder that there are so few dedicated scientists. When we know more about schedules of reinforcement, we will be able to develop as many dedicated scientists, artists, etc., as our society needs, without having to wait for accidental environmental events to develop them.

DEFINITIONS FOR SCHEDULES OF REINFORCEMENT:

Continuous reinforcement (CRF)

– A reinforcer follows each response.

Schedule of reinforcement:
– The way reinforcement occurs
– Because of the number of responses,
– Time between responses,
– And stimulus conditions.

Fixed-ratio (FR) schedule of reinforcement:

– A reinforcer follows
– a fixed number of responses.

Fixed-ratio responding:
– After a response is reinforced,
– no responding occurs for a period of time,
– then responding occurs at a high, steady rate
– until the next reinforcer is delivered.

Variable-ratio (VR) schedule of reinforcement:
– A reinforcer follows
– after a variable number of responses.

Variable-rate responding:
– Variable-ratio schedules produce
– a high rate of responding,
– with almost no post-reinforcement pausing.

Fixed-interval (FI) schedule of reinforcement:
-A reinforcer is contingent on
-the first response
-after a fixed interval of time
-since the last opportunity for reinforcement.

Fixed-interval scallop:
-A fixed-interval schedule often produces a scallop-
-a gradual increase in the rate of responding,
-with responding occurring at a high rate
-just before reinforcement is available.
-No responding occurs for some time after reinforcement.

Fixed-interval schedule of reinforcer delivery:
-A reinforcer is delivered
-after the passage of a fixed period of time,
-independently of the response.

Variable-interval (VI) schedule of reinforcement:
-A reinforcer is contingent on
-the first response
-after a variable interval of time
-since the last opportunity for reinforcement.

Variable-interval responding:
-Variable-interval schedules produce
-a moderate rate of responding,
-with almost no post-reinforcement pausing.

Resistance to extinction:
-The number of responses or
-the amount of time
-before a response extinguishes.

Resistance to extinction and intermittent reinforcement:
-Intermittent reinforcement
-makes the response
-more resistant
-than does continuous reinforcement.