Marker: A marker is a stimulus which has become a secondary reinforcer by the process of both classical and operant conditioning.
What is the function of MARKERS?
First of all, their function is communication with the dog. If used correctly, they constitute a clear way of conveying information to our pet.
- They reassure the learner that what they are doing is right. In other words, they take the randomness and ambiguity out of the task the animal performs, thereby reinforcing the target behavior.
- They separate the behavior from the reinforcer.
Before the concept of a marker (first of all a clicker) appeared, the movement of a hand holding a treat, and the delivery of that treat marked the desired behavior, confirming for the animal that this is what the trainer expects of them. This often led to the situation where both a given position (e.g. “down”) and where the handler was, became an integral part of the behavior. Talking of “down” – apart from “sit” this is one of the most popular behaviors taught to dogs (by the way, it beats me why). Usually the way it is taught is by luring with a treat held in one hand. The handler guides the dog down to the ground and waits until the dog lies down, and then gives them the treat. In 99% of cases, the dog is opposite the handler. And this relative position of the dog and handler becomes a prerequisite for the behavior to be performed again. When we add a cue, dogs will first position themselves opposite the handler, before they perform the “down” behavior. What is more, during the whole process of teaching what was reinforced was not the process of getting into the down position but the final effect – a lying dog. What I often want to reinforce when training dogs is the process, i.e. a specific way of performing a behavior, the activity, not the final effect. Using markers lets me do that. My marker will reinforce a specific fragment of behavior and will allow me to deliver a reinforcer also when the dog is not in the position.
- Markers force us, the handlers, to put more thought into what we are doing. When we use a marker in a conscious way, we are immediately forced to analyses our training. What exactly have we marked? How will this influence the process of learning, the performance of the behavior, its dynamics, expression, precision? It opens us up too many ideas and ways of analyzing where we can go with our training.1(see below)
Should we deliver a treat after every click?[1]
Or rather, should we use a primary reinforcer after each secondary reinforcer in order to maintain the effectiveness of the latter?
Arguments for:
- Otherwise the effectiveness of the clicker can be weakened
- Bob Bailey in his book clearly states that the sound of a marker should never be used on its own, nor as a form of teasing the animal, to raise expectations. The marker should be clear and unambiguous.
- This view is justified by research on secondary reinforcement which clearly indicates that secondary reinforcement should predict primary reinforcement.
Arguments against:
I will only mention one, as this is the main sensible one:
- the extinction process and variable schedule of reinforcement, which consists in reinforcing some, but not all responses of a behavior. It is very resistant to extinction. But we rarely use variable schedule of reinforcement in our training. I could say it is rather a laboratory setup so we should not worry about it.
A much better solution is teaching chains of behavior in an appropriate way, where a preceding behavior becomes a stimulus and a reinforcement for the next behavior. In my opinion a click should always be followed by a primary reinforcer. The marker should inform the dog that the process of reinforcement has begun and therefore to maintain the meaning and effectiveness of the marker, it must be followed with food, a toy, etc. This does not mean that we cannot use a variable schedule of reinforcement in our training
(but we rarely do). However, in this case we do not use a clicker to reinforce behaviors which will not be reinforced with a primary reinforcer. In this way, we will maintain the clarity and transparency of training, thus avoiding frustrating the animal and unpredictability of our behavior.
What does a clicker mark?
- the target behavior, as we would want it to look like. But frequently, the behavior is just one of the elements which we anchor with a clicker.
What else do we get in the package?
- speed
- the environment and stimuli present in the environment
- emotions accompanying the behavior (yes, we don’t work with emotions, but they still occur, so we’d better conduct our training in such a way as to foster the best possible emotions, and this can be assessed on the basis of behavior, or it would be more correct to say respondent behaviors associated)
- other behaviors such as a slight turn of the head, movement of the paw, holding breath.
- MO (motivating operations) connected with the behavior
- context
Timing:
This is something I have learned from, one of the trainers I have been working with. It is the learner that chooses what is relevant for them. What isn’t, will not be followed with a response. It is our dog that evaluates our timing, not we ourselves. Only by observing his behavior can we assess whether our timing was good or not. It is not the handler that decides what the behavior of the dog will be like. We only provide feedback by using a marker.
When does a marker not work?
- Too general to evoke a response.
It is like getting hold of a tennis racket for the first time and managing to hit a difficult ball on the first attempt. Then somebody would click and shout that it was brilliant! Nice, but would it really teach me to hit the ball correctly? Do I now have the skills that will let me repeat the behavior?
- Too much at once.
When we focus on several criteria in one session, this can be counterproductive, and the marker will not cause the right response in the dog.
- Focusing on what not to do.
When we focus on what the dog should not do during a session, the effect of the marker can be much weakened, due to the frustration of our learner. We need reinforcement to learn new behavior. Not punishment.
- Too late to bring the desired effect.
When we click even a few seconds after the desired behavior, we can expect that the dog may find a completely different behavior more relevant and not the one we mean.
- If it imposes additional conditions.
The marker starts the process of reinforcement; it doesn’t signal an extra task. When after a marker the reinforcement is withheld, because we expect another behavior, then it does not function as a marker, but as a cue for the next behavior. However, this often happens with markers which indicate a specific pattern of delivering reinforcers. The handler says YES, which means he will bring a treat to the dog (home delivery) and the dog will get it without moving out of his current position. If the dog moves towards the handler after hearing “yes”, the handler suspends delivering the reinforcement and waits until the dog returns to position. In this case YES is not a maker. Therefore, if you use different markers, they can of course indicate different ways of delivering reinforcement, BUT you still have to deliver it! It should not be connected with ANY other requirement. The most striking example I observed was a situation during a competition when the dog heard a “marker” indicating a toy and started moving towards it, but it turned out that this particular “marker” indicated “a toy on the left” and not the one on the right, which the dog approached. So, the handler recalled the dog, said “oops” and repeated the marker for the left-hand toy.
WAYS OF DELIVERING REINFORCEMENT
For a long time, I supported having many cues/markers indicating different ways of delivering reinforcement. Now my list is much shorter:
- get it
- clicker
- catch
To tell you the truth, each of them is often preceded by a clicker. The verbal cue just indicates to the dog where he will find the food.
Why have I introduced this change? Because I did some experiments in which I simultaneously taught two behaviors (a static one and a dynamic one) both with a clicker and some verbal markers.
Firstly, the clicker is much more salient. Why? Because it is neutral stimulus, much more successful during pairing procedure. Secondly, the dog quickly associates a given behavior with a given type of delivery. When I used clicker in teaching a behavior, I never had a problem with the dog abandoning position in a static behavior to come to me; in a dynamic behavior the dog would not wait for me to bring him the treat. Simply put, the way of reinforcing delivery becomes embedded in the behavior.
Then why do I still use verbal markers? Because there are some complex behaviors, like heeling, where we are working on different aspects and that need to differentiation. It is helpful to distinguish clicker – the calm delivery of a treat and GET IT, i.e. a dynamic chase after a treat.
How do we teach the meaning of a marker?
- when teaching them to stay on a mat, give the treats in such a way that the dog does not have to move away to eat them. When this works well, use the new cue/old cue protocol to introduce a marker indicating delivering the treat in the position. New cue – marker, old cue – movement of the hand with the treat
- You can also start using a clicker (if it has already been conditioned) and then the new cue – marker, old cue – clicker
How do we choose a way of delivering reinforcement?
- appropriate for the type of behavior. If you are working on staying in a position, calm behavior, that needs to have a certain duration, then choose a way that will additionally reinforce it. That is delivering the food in this position. If you are working on a dynamic exercise, choose a different way. For example, I am working on a collected trot in heel position. One of the ways of reinforcing the movement upward rather than forward is giving the food from your hand, under the arm, so that the dog has to jump up to get it. This reinforces the movement upward, and because the dog predicts this way of delivering reinforcement, they will start moving in a collected trot sooner.
- the least demanding way in the given situation. If getting the reinforcement requires performing several complex behaviors, its value decreases. In the same way, if getting the reinforcement is physically or mentally demanding, its value is weakened. This can lead to a situation when the reinforcement loses its function and stops being reinforcement altogether.
- shortening time between behavior and reinforcement (B and C) in the learning cycle. If you delay delivering the reinforcement, firstly you interfere with the contingency between the marker and the primary reinforcement, thus weakening the effectiveness of the marker and, secondly, it can turn out that you will reinforce something different than you had intended.
- shortening time between reinforcement and antecedent (C and A) in the learning cycle. Choose a way of delivering reinforcement so as to prime the dog for the next repetition, will mean you avoid many undesirable behaviors between loops, or the dog hesitating as to what to do. A good choice can immediately set the dog in a position enabling them to start the next cycle
How to enter non-clicker markers and is it worth it?
As I mentioned above, I have limited my repertoire of markers. However, I still kept a few. The most common I use is clicker, even if I use a verbal one afterwards. My verbal markers have become embedded into certain behaviors, which is why I resigned from many.
I have left just a few, the rest was changed into a clicker with a salient, predictable pattern of deliver (what we did for example with home deliver – wrist flick)
Is clicker not enough?
It is! I know a lot of handler at an unbelievably high level who only use the clicker. However, if we introduce the markers well, we will use them correctly, predictably, a few verbal ones can help us. It is important that the marker cue is just a marker! We cannot expect subsequent behaviors AFTER the marker, and in the case of the lack of their performance, suspend the release of the reinforcement. Then we deal not with the marker but the NEGATIVE punishment. The most important is the realization that the markers WE TEACH not only like any other cues, we must first of all remember the classical conditioning (and many markers do not meet this point) part.
Agnieszka Janarek Dog Trainer | TROMPLO
Above points are based on Kay’s Laurence book „Clicker Revolution”