Tag: learning theory

Associative Fear Learning in the Brain

Skynetlikesponies

An interesting article came out in Nature (abstract and link below) providing more evidence that neural links between the hippocampus and amygdala are necessary for associative fear learning. This evidence reinforces the idea that to reduce / undo fear association learning, we need to focus on amygdala activity reducing training techniques and environmental constructs, such as:

  • Training using positive reinforcement and pleasant stimuli, rather than aversives that may further activate the amygdala-hippocampus network.
  • Deconditioning fear through methods less likely to trigger fear/panic: such as systematic desensitisation, cognitive activation and counter conditioning.
  • Watching the animals body language to make sure you are not near the fear/panic threshold for the fear triggering stimuli while training so that the amgdala -hippocampal pathways are not activated and the associative memory strengthened.
  • Increase oxytocin through stable, social interactions allowing supportive attachments to form
  • Reduce overall environmental stress

“Abstract

In contextual fear conditioning, experimental subjects learn to associate a neutral context with an aversive stimulus and display fear responses to a context that predicts danger. Although the hippocampal–amygdala pathway has been implicated in the retrieval of contextual fear memory, the mechanism by which fear memory is encoded in this circuit has not been investigated. Here, we show that activity in the ventral CA1 (vCA1) hippocampal projections to the basal amygdala (BA), paired with aversive stimuli, contributes to encoding conditioned fear memory. Contextual fear conditioning induced selective strengthening of a subset of vCA1–BA synapses, which was prevented under anisomycin-induced retrograde amnesia. Moreover, a subpopulation of BA neurons receives stronger monosynaptic inputs from context-responding vCA1 neurons, whose activity was required for contextual fear learning and synaptic potentiation in the vCA1–BA pathway. Our study suggests that synaptic strengthening of vCA1 inputs conveying contextual information to a subset of BA neurons contributes to encoding adaptive fear memory for the threat-predictive context.”

 

https://www.nature.com/articles/s41467-020-15121-2

Training Multiple Cues – How Jack got his Voice

Skynetlikesponies

Often it is useful, potentially even necessary, to have more than one cue for a behaviour. For example, most horses need to know a voice and physical cue for the paces and to move back from a light touch to the chest and a voice command. Dogs will clue into the point and “off” signals that come soon after that pleasant, muddy romp on the two-legs furniture. Rats learn the screeching and frantic hand waving that prevents them chewing, or at least tries to. And cats train their two-leg servants to clean up after them through a combination of auditory retching cues and olfactory ones which trigger retching. Below my dog Jack performs the speak command to a voice and hand signals to demonstrate a behaviour which has been trained to multiple cues.

 

 

If you are training an animal, or are a particularly well-read cat training a human, it is useful to be able to teach more than one cue for a behaviour without confusing the trainee. There are several ways duel cuing or multi cuing can be achieved. You can:

  • Teach the cues separately
  • Duel cue
  • Add cues to an established behaviour

 

Teaching Two Cues Separately

Often people teach two or more cues for a behaviour separately. For example,when teaching a horse to trot whilst free schooling a visual cue or a voice signal is often used. However, during ridden work the horse will learn trot from a physical cue – a light touch with the rider’s legs. The desired end behaviour is always the same despite more than one type of cue. Separate, multiple cues can be taught to animals in this manner, the same way the horse is taught to trot from a voice command from the ground and a physical cue on board. Although, occasionally you may find that the cue-response establishment becomes dependent on the situation and that the cue may not transfer to other situations (a process know as generalisation). Lots of horse that free school very well will do the ear flick of confusion if verbally asked to trot whilst ridden. This can be overcome by introducing the cues in a range of environments so that they don’t become place or time specific.

 

Duel Cues

                Most duel cues are taught almost accidentally by the cues appearing together during training. Animals, like humans, have implicit cognitive processes specifically designed to quickly identify patterns of events within environments, whether said patterns are circumstantial, correlatory or causal. The implicit mind is responsible for or has a hand in several human behaviours, including what we call ‘gut feeling’, stereotyping, superstitions and learning.  For an animal, picking up on the pattern of the cues associated with the behaviour and reward will result in the animal registering both cues as being associated with reward when a certain behaviour is performed. Humans learn associations in much the same way, if you want someone to wash up more often, make sure something pleasant happens soon after and repeat until the washing up behaviour is established. The pleasant occurrence may include a beer or chocolate appearing in the hand of the trainee after the desired behaviour, or maybe just some affection if you are of a mind to be subtle in your human training, or of course, if you are a cat and limited in your reward options.

Back to non-human animal training. A common duel cue is the use of the word “off” and a physical pointing gesture to train an animal to move from an area. both cues are associated with the “off” behaviour being rewarded  (assuming the correct training protocol is being used, even if the cream sofa now has muddy paw prints on it).

Duel cuing, thanks to the incredible quickness of animals ability to pick up patterns, is often a successful way to teach two cues for a behaviour. However, occasionally the animal can become stuck needing both cues to perform a behaviour or only learn one of the cues. To solve the animal needing both cues try fading one cue and making one more obvious whilst rewarding the response behaviour until the obvious cue alone will trigger the behaviour. Then reverse the process until both cues trigger the wanted behaviour. If your pet/trainee has only learnt one cue and not the other, the second cue can be taught separately or added as described below.

 

Adding Cues 

As not all training can be planned in advance, sometimes extra cues need to be taught to the animal. It is possible to add additional cues to already established cue-response behaviour patterns. This is how we taught Jack to speak on command to both a voice signal and a hand one. When we first brought him home from the rescue all those years ago, I did a lot of training with him to help establish a relationship, to help him settle in and just for fun because he is a very smart and happy dog to work with.  As part of our training Jack was taught to speak through association training, if he barked I said “speak” and rewarded verbally until me saying “speak” caused Jack to bark. There are other ways to teaching the “speak” command but this one is good fun and without pressure. If your dog is liable to be over enthusiastic with the barking you can also teach a silence command or train specifically for one bark.

Once the “speak” command was taught, I thought it would be fun to teach a hand movement to also cue a bark. I paired this cue with the voice command several times, rewarding the response, and then faded the voice command very gradually and emphasised the hand movement until both the hand movement alone and voice command alone would result in a bark from Jack. The same procedure can be employed to teach two cues for other behaviours.

*Note* Some people worry that teaching a bark command encourages off command barking, however, I have never found this to be the case unless the dog is confused about the cue. Certainly Jack is never a problem barker, although he will gently, but persistently, moan at you if you are writing a blog post at walk time.

(https://www.facebook.com/skynetlikesponies)

A Thank You to Book Reviewers

Skynetlikesponies
bookI just found two really great reviews of my book on the American Amazon. They are quite old but I had somehow never seen them. A belated thank you to both reviewers, your comments are appreciated.
“5.0 out of 5 stars A Must Read For Every Horseman, Great Value, December 28, 2011
By
Bill Baehr “whipperin1” (Partsunknown, Wyoming, USA)
 Knowing Your Horse: A Guide to Equine Learning, Training and Behaviour (Paperback)
My second favorite horsemanship book of all time. I’ve read many horsemanship books and this one beats all the others except for “Equitation Science” which I consider to be the very best. This book contains more important training knowledge than reading all of the books, watching all the dvds and attending all the clinics by Parelli, Clinton Anderson, Buck Brannaman, Gordon Wright, George Morris, Denny Emerson, John Lyons, Stacy Westfall, Cherry Hill and all the rest of that ilk. No ego tripping, bragging or marketing of “horse whispering” products either, just practical horsemanship solidly based upon the science of learning. Do yourself and especially your horses a big favor and study this book.”
“5.0 out of 5 stars Better than Equitation Science, August 7, 2012

By
This review is from: Knowing Your Horse: A Guide to Equine Learning, Training and Behaviour (Paperback)
Knowing your horse outlines the fundamentals of learning theory as applied to horse training. This book provides correct definitions of learning theory and examples to help the reader grasp the concepts. There are training plans included to help people apply learning theory, as well as case studies and research reviews. Whilst I am also a huge fan of Equitation Science, Knowing Your Horse is much clearer and easier to read.”
http://www.amazon.com/Knowing-Your-Horse-Learning-Behaviour/dp/1405191643

Questions and Answers with the Equitation Science Group

Skynetlikesponies

I was recently asked to do a question and answer session for the Facebook group Equitation Science (http://www.facebook.com/groups/equitationscience/). The questions asked were very interesting so I thought I would do an article including some of the Q and A session. I would like to note that there were many fantastic comments made by the other members of the group leading to some great discussion. These comments haven’t been included here for reasons of anonymity and credit, should you wish to read these discussions simply request membership to the group.

QUESTION 1 – Negative reinforcement and avoidance learning.

In horse training, negative reinforcement involves moving away from pressure or in essence avoidance learning. When a horse has a strong disposition towards a flight response or is inclined to quickly move away from threatening stimuli, what training methods are most effective and what research is there to support their efficacy?

Answer – Firstly in this situation I would ask – why is the horse exhibit such a large stress response to the presence of such stimuli? Is the disposition really a personality trait innate to the horse or is the sensitised stress response indicative of the horse manifesting a higher base level of stress or is the response learnt? If the stress level of the horse is higher than ideal even at rest (this could be tested by heart rate or salivary cortisol) the the horses environment needs to be adapted to lower the horse’s base stress level. If the horse’s stress level is higher than it should be this will likely present itself in greater stress reactions to stimuli; this is because the threshold for such a reaction is closer to baseline level of stress in the horse. Isolation of the environmental stress will require some work but, again, analysing whether the horse has access to forage, friends and freedom is a good place to start.

Secondly, if the response is learnt training the horse using positive reinforcement methods will help reduce the stress response. Targeting could be used to train the desired behaviour and put it on a cue, subsequently a secondary cue of a very gentle pressure cue, such the horse would not try to escape it, could then be added if required. Such a training strategy would eliminate the need for stressful aversive stimuli through the use negative reinforcement training but would allow a gentle pressure cue if needed. If the horse has become more generally fearful of an environment/object/situation, rather than just the stimuli used to implement negative reinforcement, counter conditioning stimuli associated with fear will be helpful. Desensitisation could also be used to reduce the stress experienced by the horse through not over facing the horse with them the stimuli they are fearful of.

Evidence for positive reinforcement methods:

http://www.ingentaconnect.com/content/ufaw/aw/2007/00000016/00000004/art00007

http://www.sciencedirect.com/science/article/pii/S0168159107002869

http://www.springerlink.com/content/4122111x7620v040/

http://www.sciencedirect.com/science/article/pii/S0003347209006034

One for targeting: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1284337/

http://onlinelibrary.wiley.com/doi/10.2746/042516406778400574/abstract

Additional comment – In this case we were examining a horse with a large stress response to negative reinforcement stimuli and thus would require training to eliminate this response to pressure (or the stimuli used for negative reinforcement). Although I believe we can use very gentle negative reinforcement without too much stress to the horse, you raise an interesting point because unfortunately positive punishment (the addition of an unpleasant stimulus to lower the likelihood of a behaviour reoccurring) has to of occurred in order that the stimulus can be removed for negative reinforcement. If the stimulus did not start it could not be removed. The two concepts, although distinct, are not mutually exclusive, they work in tandem (see http://www.theequineindependent.com/home/?p=103).

QUESTION 2 – Equine Learned Helplessness

The American psychologist Martin Seligman published most of the early work on learned helplessness. This is the technical term used to describe a condition in which a human/animal has learned to behave helplessly, failing to respond even when there is an opportunity for it to help itself by avoiding unpleasant circumstances or gain a positive reward. In people, learned helplessness is associated with depression and other mental health problems. I am just wondering what the possible epidemiology of equine learned helplessness might be, the “symptomatology” and possible health ramifications.

Answer – Learned Helplessness is a psychological phenomenon which occurs when an animal, be it horse or human, no longer tries to escape an aversive stimulus (or in some cases multiple aversive stimuli). Such behaviour usually manifests because the horse has repeatedly been exposed to an aversive stimulus, tried to escape it, and failed. Eventually the animal stops trying to escape and thus behaves in a helpless manner. Often the horse may only exhibit this behaviour to one or two stimuli, however, sometimes you can see this helplessness response generalise in the same manner as other behaviours may generalise. Therefore, the helplessness may not be stimulus or situation specific. In the horse world sometimes such horses are considered ‘shut down’.

Specifically in horses restraint, pressure and punishments have been considered a potential source of learned helplessness if incorrectly utilised. Examples of potential sources of learned helplessness include the incorrect use of riding gadgets such as draw reins, strong bits (even kinder bits in the wrong hands), spurs, whips … I am sure we can all think of more. Some specific training techniques e.g. leg tying and dare I say Rolkur, rely on learned helplessness, however, any technique that uses aversive stimuli can be at risk of inducing such a response if wrongly applied.

Symptomology:

*The most obvious symptom is a lack of escape behaviour in response to an unpleasant stimulus. The stimulus may be pressure, fear or pain based.
Other symptoms that have not been examined closely in horses but are documented in humans include:
*Sensitised and adapted stress response. If a prolonged period of exposure to an inescapable unpleasant stimulus it experienced, the results can present in the form of both the psychological and physiological symptoms of stress. These may continue if the horse if exposed to stimuli associated with the inescapable stressor, even if the stressor itself is no longer present.

*Psychologically the horse may experience anhedonia, lack of motivation, disrupted emotional processing, unusual stress responses (fight and flight) and inhibited learning/cognitive ability.

*Physiologically the horse may experience increase stress, a reduced immune response and an increased risk of the disorders associated with a high stress environment and life experience (e.g. stomach ulcers). It is possible that these symptoms could all occur in the horse although I stress little specific research has been done in this area, and given that most learned helplessness studies on animals were not entirely ethical this may not be a terrible thing.

There are theories of depression which concentrate on the role of learned helplessness, however these are widely debated, certainly there is a cross over in both symptomology and neurological activation if you are interested in reading about any of the above a quick google search will find you a lot of information.

Specifically with regards to horses I can recommend the paper – “Is There Evidence of Learned Helplessness in Horses?” Hall et al, 2008.

Neurology :

I don’t have time to write out all the neurological information so you will have to forgive me quoting.

“Evidence suggests an important role for 5-HT neurons in the dorsal raphe nucleus (DRN) in mediating learned helplessness (see Maier and Watkins 2005, for reviews). The DRN is a midline brainstem structure that contains a high concentration of 5-HT neurons that provide 5-HT to higher brain centers via multiple fiber tracts. …5-HT neurons in the DRN have long been associated with depression … anxiety …and behavioral responses to stress… The DRN projects to structures involved in fear, anxiety, and depression, such as the cortex, amygdala, periaqueductal grey (PAG), and locus coeruleus (LC)” Greenwood and Fleshner (2008). You can see that stress can affect the functioning of these pathway.

Rehabilitating the learned helplessness horse:

Here are a few idea for undoing the learned response, remember the brain is plastic even when the horse is old and thus often the horse can relearn/unlearn their response to stimuli.

*It’s cliché but time is a great healer, especially time in a stress free environment where they no longer experience the stressor which induces the learned helplessness response. Ideally the horse will be out as much as possible, be eating for 16hrs-ish a day and have a stable peer group to socialise with. The old adage of forage, friends and freedom can go a long way towards the rehab of any horse. The brains stress response will often (but not always) ‘reset’, if you like, in such an environment making further training much easier. Removing the stressor(s) is the first step!

*If the stressor is something which the horse has to come into contact with in their environment, a training strategy including counter conditioning and desensitisation combined will help the horse to relearn to be relaxed and even enjoy the presence of the previously stressful object/environment. Obviously you would only do this for objects and situations associated with the aversive events/helplessness and not the events themselves! For example, if the horse had become helpless when ridden you could work on encouraging the horse to enjoy being ridden by training without the use of large aversive stimuli but instead with positive reinforcement. I have found that reward inhibits stress in the horse. Indeed research shows that activation of the reward pathways of the brain actively dampens stress responses and therefore will help the horse to be without a heightened stress response and the psychological and physiological manifestations of increased stress.

Additional comment – Grass is included in the forage part of the phrase. The phrase is applicable to the horse as a management system as it describes the most prominent innate needs of the horse in order that they can be without stress. Therefore, as you say, it is necessary at all stages of the horse’s life. I was describing it as part of the rehabilitation for learned helplessness because I suspect that none here would drive a horse into learned helplessness but they may acquire such a horse or be called out to one. A slightly more complex version of the same paradigm might be an adapted version of Maslow’s heirarchy of needs. Regarding the relationship between submission and learned helplessness it would certainly be valid to suggest a behavioural parallel between the two psychological states (unfortunately, I don’t own the Equid Ethogram). Possibly it would be accurate to say that all learned helplessness could be described as submission but not all submission is learned helplessness, of course this depends on your definition of submission. The relationship between these two psychological concepts seems to be complex and their isn’t a huge amount of research available, however, this paper is worth a read (again I don’t agree with the methods used) http://www.ncbi.nlm.nih.gov/pubmed/17708544. Hope this answers your question.

QUESTION 3 – There seems to be a lot of confusion out there with regards to definitions of negative punishment (response cost, time out). Do you think horses actually understand negative punishment ? What research has been done in this area?

Answer – Negative punishment is possibly the trickiest of the four primary learning theory concepts to apply well to horse training. The removal of a desirable stimulus in consequence to an unwanted behaviour in order to punish said behaviour requires both timing and an understanding of frustration/defensive behaviours. Obviously, removing a highly desirable stimuli from the horse could trigger unwanted behaviours over and above the original unwanted behaviour, so care is needed. For example, removing food from a horse with food related issues may trigger defensive aggression, but this technique may not unduly stress another horse, therefore each horse and behaviour needs to be considered with regards to their individual personality. So yes horses can be trained with negative punishment, it’s the human understanding of punishment and the side effects which can occur when such methods are used which is key. Rewarding a incompatible behaviour in place of the unwanted behaviour may be an effective alternative technique, circumventing the need for punishment.

Research in this area is thin on the ground, probably because the ethics of such research would be hard to navigate, similarly to work on positive punishment.

Additional related question – So when I am clicker training my horse and withhold food whilst I am waiting for the correct response, is this negative punishment? As I have not actually taken anything away, rather I am withholding a positive reinforcer?

Answer – This is a tricky question, if positive reinforcement is being used the reward should never be given to the horse and then removed creating negative punishment. However, sometimes this is case when the trainer is not sure whether to reward or not, so the trainer needs to be definite with the timing. The trainer also needs to be aware that if the horse is too hungry or gets anxious regarding food negative reinforcement could also come into play as the food removes briefly the aversive stimuli of hunger(interestingly there are theories of drug addiction which focus on the role of negative reinforcement). These effects can be reasonably simply averted by observing the horse for signs of stress or learning disruption which might suggest their role and changing strategy to ensure the positive reinforcement acts exclusively.

Additional comments –

The training strategy should be defined before it occurs however, within the training observation and evaluation should be regularly considered to ensure that the trainer is training in the manner they intend and that the horse is happy and progressing in said training.

To clarify the negative punishment with food stimuli does not occur simply by the presence of food because you have not removed anything from the horse, the horse never had the food. It would only occur, as I said before, if the trainer was ambiguous in timing and gave the horse the food and then removed it due to a change of mind.

If you have a question about any of the answers or the terminology used, please feel free to leave a comment or email me and I will happily answer your questions.

Is Pressure-Release Without Punishment?

Skynetlikesponies

For a while a conundrum has been playing with my thoughts. It all started one chilly but sunny Saturday, during an ordinary training session, whilst training a horse that for the purposes of this article we will call ‘The Pony’. It was nearing the end of the session when The Pony tripped over a protruding root an came very close to stand on my foot. In an effort to preserve my foot I quickly asked the horse to move back wit

h a little pressure on the chest. A fairly benign act in response to which The Pony quickly ceased stumbling toward my vulnerable metatarsals and politely backed away. The training session continued without incident but a question remained in my mind.

Pressure and release is a commonly used method of horse training, within equestrian literature it is often cited as a simple but effective use of negative reinforcement. Whether it is a pressure on the chest of a horse to ask for a reverse, the use of the head collar to lead the horse, a gentle pressure on the reins to ask for a halt or the use of advance and retreat to approach a nervous horse, it is almost impossible to avoid the use of negative reinforcement in training. However until the incident with The Pony it had never occurred to me that positive punishment maybe an inextricable part of negative reinforcement. Can negative-reinforcement occur with out positive punishment? The Pony had be negatively reinforced for reversing away from me by the release of pressure, but it had been the addition of the pressure which had prevented further advancement.

Negative reinforcement is defined within all behaviour literature as the removal of an unpleasant or aversive stimulus in response to a wanted behaviour to reinforce the behaviour, and thus encourage the behaviour to reoccur in the future. This is a definition that most people are familiar with. However, the training incident with The Pony made me think – in order for something to be removed it must have been applied at some point in the past. For the aversive stimuli to be removed to reinforce a behaviour it must at some point have been applied. The addition of an aversive stimulus is the definition of positive punishment, the effect of positive punishment is that it reduces the likelihood of the behaviour it is a consequence of occurring in the future. Pressure on the chest of The Pony prevented further forward advancement towards my foot, so is it that I positively punished the forward movement? And as an extension of this thought, is it that every use of negative reinforcement begins with the use of positive punishment?

This is may seem like a conundrum based in the semantics of academic definitions but the practical consequences of positive punishment being inextricable from negative reinforcement are not dismissible. The most important practical consequence of positive punishment is that it discourages the behaviour it is associated with from occurring again. When applying the negative reinforcer, be it pressure or the advance of advance-retreat training, we must be careful that the behaviour it is being applied to is unwanted or the positive punishment would diminish a desired behaviour. The training should thus ensure that the negative reinforcer is attended to with regards to not only the timing of its release but also it’s application, this will ensure that wanted behaviours are not punished.

The problem of positive punishment being inextricable from negative reinforcement and the two training methods not being mutually exclusive is one that has impacts on training which could effect the psychology of the horse, the effectiveness of the training and the welfare of the horse. Punishment has been correlated with side effects which are important to our training of horses and must be understood to preserve the horse’s well being within training. Although this article is not the place to detail the problems and side effects of punishment I will briefly outline the most important ones below.

  • Punishment teaches only what not to do and does not suggest a more appropriate behavioural replacement for the one being punished.
  • Punishment can invoke emotional reactions in horses, such as fear or aggression. These reactions are more likely with physical punishment. In order to avoid these reactions, any punishment applied should be sympathetic to the horse’s personality i.e. how reactive they are and also to the situation. Positive punishment and negative reinforcement are both based in use of stimuli which are to a greater or lesser extent unpleasant for the horse, such as pressure, and as such it must be ensured that the horse is not stressed by the punisher in order to ensure emotional reactions are not experienced.
  • Pain-elicited aggression can be induced if painful physical punishers are used. Pain can heighten a flight/fight response and cause aggressive reactions in the horse as they try and escape the threat of pain, therefore positive punishers which cause pain should never be used in training.
  • Anxiety caused by punishment can actually impair the horse’s ability to concentrate and learn effectively. Extremes of emotion inhibit the brains cognitive abilities and thus impair attention.
  • Learned helplessness is a condition induced through the incorrect use of punishment. Learned helplessness occurs when the horse feels they cannot avoid punishment over a sustained period of time. The horse learns that any attempts to escape are futile and thus the horse will not attempt to escape or avoid the punishment, even once an escape or avoidance method is offered.
  • Avoidance behaviours – if the horse learns to associate a person or situation with punishment, the horse may logically try to avoid that situation or handler.
  • It is also possible for horses to selectively suppress the punished behaviour until punishment is less likely, either when the punishing handler is no longer present, or when the horse believes that it is less likely to be punished for the behaviour.
  • Punishment can reduce the horse’s interest in their work, if a horse is punished the horse’s motivation will be diminished and thus the horse is less likely to participate willingly in training.

If negative reinforcement by its definition begins with a positive punishment these problems that are associated with punishment are consequently also a problems intrinsic to the use of negative reinforcement. It is therefore essential that they are considered carefully if negative reinforcement is to be used in training. The application of the negative reinforcer must follow the rules of applying positive punishment if side effects are to be avoided in the horse.

The rule of applying punishment to avoid side effects are as follows, the punishment must be –

  • Immediate
  • Consistent
  • Never painful
  • Never dealt in anger
  • Specific to targeted unwanted behaviours and not delivered randomly or accidentally.
  • Never used during confusion

Obviously each horse has their own tolerance levels for different stimuli. An aversive stimulus for one horse may not be unpleasant for another. However given that negative reinforcement is based upon the release of an aversive stimulus, it is highly likely that the stimulus use as a negative reinforcer could also be a positive punisher for the horse. I would be interested to hear if anyone could think of a training scenario in which the negative reinforcer when applied could not be considered a positive punisher because I must admit I could not think of one.

To finish this article I would like to say that I don’t believe that we can avoid the use of negative reinforcement in training but any part of training that uses aversive stimuli, i.e. negative reinforcement or positive punishment should be carefully considered with regards to the strength of the stimulus and its application.

The Different Techniques Known as Clicker Training

Skynetlikesponies

Saffy-trot-300x225When considering a way to train their horse using positive reinforcement, most horse owners find themselves investigating clicker training. However, once the horse owner starts to read into clicker training, or visits a few clinics, it soon becomes apparent that different trainers use clicker training in different ways. Clicker training is not one singular technique, but a tool, applied in different ways by different trainers. The benefits and potential difficulties associated with each of these approaches to clicker training will be discussed in this article, with the aim that this will hopefully this will abate some of the confusion that can be experienced by owners new to clicker training.

Before we begin, I will quickly review the basics of clicker training theory as applied to practical horse training. Very simply clicker training is a form of positive reinforcement training. Positive reinforcement being the addition of something pleasurable to the horses environment in consequence to the horse performing a desirable behaviour. Positive reinforcement encourages the desired behaviour to reoccur in the future. Anything that the horse finds pleasurable, for example food rewards or stroking, can be used for the purposes of positive reinforcement training, although food rewards are most commonly used. During positive reinforcement the reward must be delivered immediately as the desired behaviour is performed by the horse, so that only the desired behaviour is reinforced.

The definition of positive reinforcement – An increase in the future frequency of a behaviour due to the addition of a pleasurable stimulus immediately following said behaviour.

Positive reinforcement alone is a very effective training method, however, it relies on the immediate delivery of the reward as the horse performs the desired behaviour. Clicker training makes reinforcement of behaviour at the correct moment easier, because, rather than having to deliver the reward to the horse’s mouth at the moment they perform the desire behaviour, the click noise can mark the desire behaviour and the reward can be delivered as soon as possible. The association of the click noise with food reward, transforms the click noise into a secondary reinforcer, which simply means that the click has taken on reinforcing properties and thus become rewarding. Once an association between the click and food reward has been establish, and the click has become a secondary reinforcer, the click can then be used to communicate to the horse when they have performed a desired behaviour. Marking the behaviour using the audible ‘click’ of the clicker is beneficial to any training where the trainer can’t deliver reward immediately following a correct behavioural response, e.g. when the horse is at distance or being ridden. The click of the clicker is a good sound for marking correct behavioural responses because it is short and crisp. Some trainers prefer to use a ‘cluck’ sound made by the tongue for the same purpose. Currently, there is no scientific evidence to suggest that the use of a tongue ‘cluck’ is less or more effective than the use of a clicker.

The definition of a secondary reinforcer – A secondary reinforcer, also known as a conditioned reinforcer, is a stimulus (such as a click) that when consistently paired with a pleasurable stimulus (such as food) functions as a reinforcer.

The use of the click sound within clicker training has been applied in different ways by different horse trainers. The key factor, which will be discussed in this article, is how different trainers apply the clicker practically during training. To address this topic, we will consider the use of the click as a terminal bridge and as an intermediate bridge. Now the key to understanding the use of clicker in training is to understand, but not get bogged down in, the terminology. I will explain the theory, but also how the theory is practically applied in everyday horse training. The first thing that needs to be explained is that the click of the clicker is know as a bridging stimulus, this is because it bridges the gap between the desired behaviour and the arrival of the food reward. The click says to the horse ‘yes that’s the behaviour I want and your reward is coming’. However, the click can be one of two types of bridge. It can be a terminal bridge that says ‘yes, well done, finished’, or an intermediate bridge which says to the horse ‘yes, keep going your on the right track’. In practise this mean that the click sound either signals to the horse that they were performing the desired behaviour and they can stop for reward (a terminal bridge), or in the case of the intermediate bridge, the click signals to the horse that they are doing the correct behaviour and to continue until the terminal bridge, which will be a different signal.

It is most common in training to use the click sound of the clicker as a terminal bridge. In practical terms this means that the click is used to signal to the horse to stop and receive their reward. For example, if you were teaching a horse to touch a target with there muzzle, you would click the horse once they touch the target and then reinforce the behaviour with the food reward. If you wanted the targeting behaviour to last longer you would shape the behaviour by gradually leaving longer periods of time between the start of the targeting behaviour and the click. This method of clicker training is used by Alexander Kurland (2001) and Becky Holden, amongst others. There are both pros and cons to this method.

The pros of the terminal bridge clicker training method –

◦This method can be used to teach everything, from basic ground work to advanced riding exercises.

◦The horse can be easily rewarded for desired behaviour, even at a distance or whilst ridden.

◦Owners can usually pick up this method easily under instruction.

The cons of the terminal bridge clicker training method –

◦The method doesn’t include a intermediate bridge stimulus so the horse can be told to stop to be rewarded but not to keep performing the same behaviour, instead the behaviour is modified using shaping or chaining.

Now to discuss the use of the clicker as an intermediate bridge stimulus. When the click sound is used as an intermediate bridge the click says to the horse – ‘Yes, keep going you’re on the right track’. Using the targeting example given earlier, to teach a horse to touch a target using the click as an intermediate bridge, the trainer would click the horse for touching the target to encourage the horse to continue touching the target, until the terminal stimulus was given. The click, which can occur a variable amount of times before the terminal stimulus is given, encourages the horse to continue the behaviour they are currently performing. Ben Hart (2008) is the most famous trainer that uses the clicker as an intermediate bridge stimulus. Ben trains using the hand going to the reward holder as the terminal stimulus. There are also pros and cons to the intermediate bridge method of clicker training.

The pros of the intermediate bridge clicker training method –

◦This method can be used to teach all ground work activities.

◦The horse can be easily rewarded for desired behaviour, even at a distance.

◦The horse can be given guidance as to whether or not the behaviour they are performing is desirable, and be given confidence to continue the behaviour, without stopping for reward.

The cons of the intermediate bridge clicker training method –

◦Some owners find applying the clicker as an intermediate bridge stimulus more difficult, although I suspect this is because most of the literature available describes the terminal bridge method.

◦The terminal bridge stimulus of this method of clicker training often isn’t audible, and thus this method is a little more difficult to apply if the horse can’t directly see the hander, e.g. during ridden work.

Both these methods of clicker training are effective modes of communication with the horse, as such both methods have been applied with great success to training horses for many jobs. Interestingly, neither method has been scientifically shown to be more effective than the other, therefore the deciding factor when choosing how to apply clicker training with your own horses must be which method best suits your horse, your ability and your training. I highly recommend reading literature from many different clicker trainers, and ideally, also seeing the methods demonstrated, before you decide which method will be best for you and your horse.

References

Alexandra Kurland (2001). Clicker Training For Your Horse. Ring Books.

Ben Hart (2008). The Art and Science of Clicker Training for Horses: A Positive Approach to Training Equines and Understanding Them. Souvenir Press Ltd.

Positively Encouraging Behaviour

Skynetlikesponies

For those embarking on training their horses and wishing to use mostly or completely training which is based in positive reinforcement, the problem of how to encourage the behaviours they want to train to occur, so that they may be rewarded and propagated, is often encoutered. In conventional training desired behaviours are often encouraged through the use of pressure and there is the misconception that only free-shaping is available to those who practise positive reinforcement training. In free-shaping the trainer waits for the horse to perform the desired behaviour and then rewards its presentation. However, there are methods which can be used to encourage behaviour without the use of pressure or, indeed, waiting for the behaviour to occur of its own volition.

Targeting is the most popular positive method of encouraging wanted behaviour in the horse. For the purpose of targeting the horse is taught, using clicker training or another positive reinforcement method, to go to or follow a target object on command. This can be a static marker or a movable object. Once trained, the horse can easily learn to perform new and/or wanted behaviours by following the target. Full guides on how to teach targeting can be found in most clicker training books and my own book (http://www.amazon.co.uk/Knowing-Your-Horse-Learning-Behaviour/dp/1405191643/ref=sr_1_1?ie=UTF8&qid=1339968870&sr=8-1).

Teaching your horse to target can be invaluable for training both basic and complex behaviours; really the only limitation to training is the imagination of the trainer. Once the wanted behaviour is reliably occurring in response to the target, it can be put on an appropriate cue and the target is gradually removed over a short period of time. A common misconception in clicker training is that the target remains as part of the trained behaviour forever; however, this does not represent the goal of target training.

Some common applications of targeting training in horse training include:

Leading, head-lowering, staying in a desired location, basic safety behaviours (e.g. backing and coming on cue), head collar/bridling routines, mounting/dismounting, spook busting, teaching lungeing, and loading into trailers or horse boxes.

Target training has also been studied scientifically, and been observed to be an effective method of horse training. The links below describe research which investigated training horses to load using targeting.

Click to access Hendriksen%202011%20postive%20negative%20reinforcement.pdf

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1284337/

http://www.ncbi.nlm.nih.gov/pubmed/21310219

The second most popular method of positively encouraging the horse to present a wanted behaviour is ‘lure and reward’. The term ‘lure’ can often put people off due to negative associations with the word; a better name for this technique is possibly ‘guide and reward’. At its core the method is very similar to the previously describe target training, the horse follows a food guide, thereby performing a desired behaviour and receiving reward. Most

trainers reward from a treat held in the other hand. Obviously doing this training with horses who have not yet learnt not to mug is unwise. However, other than this caveat, the training can be very effective and enjoyable for both horse and trainer. Again, once the horse is reliably performing the behaviour with the target a cue is given and the guide gradually removed. The guide should not be the cue. This process should not take longer than a few sessions, especially for a basic behaviour. Interestingly, this is one of the most commonly used training methods employed by respected dog trainers. Again comprehensive instructions on how to successful use this method with your horse can be found on the internet or in appropriate books.

 

Finally, using a cue to mean ‘well done, keep going’ as well as a separate ‘good, finish’ cue, can be useful for encouraging the expression of new desired behaviours. How you apply this in training will depend on the individual preferences of the trainer and the previously employed method of positive training. Personally, I like to use to different sounding clicks, which I make with my mouth rather than a clicker, but this is not the only possible method of application. One click sound means ‘continue as you are’, the other communicates ‘finish’. This allows me a more elegant flow of communication to the horse, as well as an active means of encouraging the horse to perform a wanted behaviour in a positive manner. Once the horse has performed the desired behaviour, they may be given the finish signal to indicate they did well and to rest and wait for reward (particular useful if the horse is at a distance from the handler).

If you would like more information on these training techniques briefly discussed here, please feel free to comment or message me at my email address