I was recently asked to do a question and answer session for the Facebook group Equitation Science (http://www.facebook.com/groups/equitationscience/). The questions asked were very interesting so I thought I would do an article including some of the Q and A session. I would like to note that there were many fantastic comments made by the other members of the group leading to some great discussion. These comments haven’t been included here for reasons of anonymity and credit, should you wish to read these discussions simply request membership to the group.
QUESTION 1 – Negative reinforcement and avoidance learning.
In horse training, negative reinforcement involves moving away from pressure or in essence avoidance learning. When a horse has a strong disposition towards a flight response or is inclined to quickly move away from threatening stimuli, what training methods are most effective and what research is there to support their efficacy?
Answer – Firstly in this situation I would ask – why is the horse exhibit such a large stress response to the presence of such stimuli? Is the disposition really a personality trait innate to the horse or is the sensitised stress response indicative of the horse manifesting a higher base level of stress or is the response learnt? If the stress level of the horse is higher than ideal even at rest (this could be tested by heart rate or salivary cortisol) the the horses environment needs to be adapted to lower the horse’s base stress level. If the horse’s stress level is higher than it should be this will likely present itself in greater stress reactions to stimuli; this is because the threshold for such a reaction is closer to baseline level of stress in the horse. Isolation of the environmental stress will require some work but, again, analysing whether the horse has access to forage, friends and freedom is a good place to start.
Secondly, if the response is learnt training the horse using positive
reinforcement methods will help reduce the stress response. Targeting could be used to train the desired behaviour and put it on a cue, subsequently a secondary cue of a very gentle pressure cue, such the horse would not try to escape it, could then be added if required. Such a training strategy would eliminate the need for stressful aversive stimuli through the use negative reinforcement training but would allow a gentle pressure cue if needed. If the horse has become more generally fearful of an environment/object/situation, rather than just the stimuli used to implement negative reinforcement, counter conditioning stimuli associated with fear will be helpful. Desensitisation could also be used to reduce the stress experienced by the horse through not over facing the horse with them the stimuli they are fearful of.
Evidence for positive reinforcement methods:
http://www.ingentaconnect.com/content/ufaw/aw/2007/00000016/00000004/art00007
http://www.sciencedirect.com/science/article/pii/S0168159107002869
http://www.springerlink.com/content/4122111x7620v040/
http://www.sciencedirect.com/science/article/pii/S0003347209006034
One for targeting: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1284337/
http://onlinelibrary.wiley.com/doi/10.2746/042516406778400574/abstract
Additional comment – In this case we were examining a horse with a large stress response to negative reinforcement stimuli and thus would require training to eliminate this response to pressure (or the stimuli used for negative reinforcement). Although I believe we can use very gentle negative reinforcement without too much stress to the horse, you raise an interesting point because unfortunately positive punishment (the addition of an unpleasant stimulus to lower the likelihood of a behaviour reoccurring) has to of occurred in order that the stimulus can be removed for negative reinforcement. If the stimulus did not start it could not be removed. The two concepts, although distinct, are not mutually exclusive, they work in tandem (see http://www.theequineindependent.com/home/?p=103).
QUESTION 2 – Equine Learned Helplessness
The American psychologist Martin Seligman published most of the early 
work on learned helplessness. This is the technical term used to describe a condition in which a human/animal has learned to behave helplessly, failing to respond even when there is an opportunity for it to help itself by avoiding unpleasant circumstances or gain a positive reward. In people, learned helplessness is associated with depression and other mental health problems. I am just wondering what the possible epidemiology of equine learned helplessness might be, the “symptomatology” and possible health ramifications.
Answer – Learned Helplessness is a psychological phenomenon which occurs when an animal, be it horse or human, no longer tries to escape an aversive stimulus (or in some cases multiple aversive stimuli). Such behaviour usually manifests because the horse has repeatedly been exposed to an aversive stimulus, tried to escape it, and failed. Eventually the animal stops trying to escape and thus behaves in a helpless manner. Often the horse may only exhibit this behaviour to one or two stimuli, however, sometimes you can see this helplessness response generalise in the same manner as other behaviours may generalise. Therefore, the helplessness may not be stimulus or situation specific. In the horse world sometimes such horses are considered ‘shut down’.
Specifically in horses restraint, pressure and punishments have been considered a potential source of learned helplessness if incorrectly utilised. Examples of potential sources of learned helplessness include the incorrect use of riding gadgets such as draw reins, strong bits (even kinder bits in the wrong hands), spurs, whips … I am sure we can all think of more. Some specific training techniques e.g. leg tying and dare I say Rolkur, rely on learned helplessness, however, any technique that uses aversive stimuli can be at risk of inducing such a response if wrongly applied.
Symptomology:
*The most obvious symptom is a lack of escape behaviour in response to an unpleasant stimulus. The stimulus may be pressure, fear or pain based.
Other symptoms that have not been examined closely in horses but are documented in humans include:
*Sensitised and adapted stress response. If a prolonged period of exposure to an inescapable unpleasant stimulus it experienced, the results can present in the form of both the psychological and physiological symptoms of stress. These may continue if the horse if exposed to stimuli associated with the inescapable stressor, even if the stressor itself is no longer present.
*Psychologically the horse may experience anhedonia, lack of motivation, disrupted emotional processing, unusual stress responses (fight and flight) and inhibited learning/cognitive ability.
*Physiologically the horse may experience increase stress, a reduced immune response and an increased risk of the disorders associated with a high stress environment and life experience (e.g. stomach ulcers). It is possible that these symptoms could all occur in the horse although I stress little specific research has been done in this area, and given that most learned helplessness studies on animals were not entirely ethical this may not be a terrible thing.
There are theories of depression which concentrate on the role of learned helplessness, however these are widely debated, certainly there is a cross over in both symptomology and neurological activation if you are interested in reading about any of the above a quick google search will find you a lot of information.
Specifically with regards to horses I can recommend the paper – “Is There Evidence of Learned Helplessness in Horses?” Hall et al, 2008.
Neurology :
I don’t have time to write out all the neurological information so you will have to forgive me quoting.
“Evidence suggests an important role for 5-HT neurons in the dorsal raphe nucleus (DRN) in mediating learned helplessness (see Maier and
Watkins 2005, for reviews). The DRN is a midline brainstem structure that contains a high concentration of 5-HT neurons that provide 5-HT to higher brain centers via multiple fiber tracts. …5-HT neurons in the DRN have long been associated with depression … anxiety …and behavioral responses to stress… The DRN projects to structures involved in fear, anxiety, and depression, such as the cortex, amygdala, periaqueductal grey (PAG), and locus coeruleus (LC)” Greenwood and Fleshner (2008). You can see that stress can affect the functioning of these pathway.
Rehabilitating the learned helplessness horse:
Here are a few idea for undoing the learned response, remember the brain is plastic even when the horse is old and thus often the horse can relearn/unlearn their response to stimuli.
*It’s cliché but time is a great healer, especially time in a stress free environment where they no longer experience the stressor which induces the learned helplessness response. Ideally the horse will be out as much as possible, be eating for 16hrs-ish a day and have a stable peer group to socialise with. The old adage of forage, friends and freedom can go a long way towards the rehab of any horse. The brains stress response will often (but not always) ‘reset’, if you like, in such an environment making further training much easier. Removing the stressor(s) is the first step!
*If the stressor is something which the horse has to come into contact with in their environment, a training strategy including counter conditioning and desensitisation combined will help the horse to relearn to be relaxed and even enjoy the presence of the previously stressful object/environment. Obviously you would only do this for objects and situations associated with the aversive events/helplessness and not the events themselves! For example, if the horse had become helpless when ridden you could work on encouraging the horse to enjoy being ridden by training without the use of large aversive stimuli but instead with positive reinforcement. I have found that reward inhibits stress in the horse. Indeed research shows that activation of the reward pathways of the brain actively dampens stress responses and therefore will help the horse to be without a heightened stress response and the psychological and physiological manifestations of increased stress.
Additional comment – Grass is included in the forage part of the phrase. The phrase is applicable to the horse as a management system as it describes the most prominent innate needs of the horse in order that they can be without stress. Therefore, as you say, it is necessary at all stages of the horse’s life. I was describing it as part of the rehabilitation for learned helplessness because I suspect that none here would drive a horse into learned helplessness but they may acquire such a horse or be called out to one. A slightly more complex version of the same paradigm might be an adapted version of Maslow’s heirarchy of needs. Regarding the relationship between submission and learned helplessness it would certainly be valid to suggest a behavioural parallel between the two psychological states (unfortunately, I don’t own the Equid Ethogram). Possibly it would be accurate to say that all learned helplessness could be described as submission but not all submission is learned helplessness, of course this depends on your definition of submission. The relationship between these two psychological concepts seems to be complex and their isn’t a huge amount of research available, however, this paper is worth a read (again I don’t agree with the methods used) http://www.ncbi.nlm.nih.gov/pubmed/17708544. Hope this answers your question.
QUESTION 3 – There seems to be a lot of confusion out there with regards to definitions of negative punishment (response cost, time out). Do you think horses actually understand negative punishment ? What research has been done in this area?
Answer – Negative punishment is possibly the trickiest of the four primary learning theory concepts to apply well to horse training. The removal of a desirable stimulus in consequence to an unwanted behaviour in order to punish said behaviour requires both timing and an understanding of frustration/defensive behaviours. Obviously, removing a highly desirable stimuli from the horse could trigger unwanted behaviours over and above the original unwanted behaviour, so care is needed. For example, removing food from a horse with food related issues may trigger defensive aggression, but this technique may not unduly stress another horse, therefore each horse and behaviour needs to be considered with regards to their individual personality. So yes horses can be trained with negative punishment, it’s the human understanding of punishment and the side effects which can occur when such methods are used which is key. Rewarding a incompatible behaviour in place of the unwanted behaviour may be an effective alternative technique, circumventing the need for punishment.
Research in this area is thin on the ground, probably because the ethics of such research would be hard to navigate, similarly to work on positive punishment.
Additional related question – So when I am clicker training my horse and withhold food whilst I am waiting for the correct response, is this negative punishment? As I have not actually taken anything away, rather I am withholding a positive reinforcer?
Answer – This is a tricky question, if positive reinforcement is being used the reward should never be given to the horse and then removed creating negative punishment. However, sometimes this is case
when the trainer is not sure whether to reward or not, so the trainer needs to be definite with the timing. The trainer also needs to be aware that if the horse is too hungry or gets anxious regarding food negative reinforcement could also come into play as the food removes briefly the aversive stimuli of hunger(interestingly there are theories of drug addiction which focus on the role of negative reinforcement). These effects can be reasonably simply averted by observing the horse for signs of stress or learning disruption which might suggest their role and changing strategy to ensure the positive reinforcement acts exclusively.
Additional comments –
The training strategy should be defined before it occurs however, within the training observation and evaluation should be regularly considered to ensure that the trainer is training in the manner they intend and that the horse is happy and progressing in said training.
To clarify the negative punishment with food stimuli does not occur simply by the presence of food because you have not removed anything from the horse, the horse never had the food. It would only occur, as I said before, if the trainer was ambiguous in timing and gave the horse the food and then removed it due to a change of mind.
If you have a question about any of the answers or the terminology used, please feel free to leave a comment or email me and I will happily answer your questions.
I just found two really great reviews of my book on the American Amazon. They are quite old but I had somehow never seen them. A belated thank you to both reviewers, your comments are appreciated.
When considering a way to train their horse using positive reinforcement, most horse owners find themselves investigating clicker training. However, once the horse owner starts to read into clicker training, or visits a few clinics, it soon becomes apparent that different trainers use clicker training in different ways. Clicker training is not one singular technique, but a tool, applied in different ways by different trainers. The benefits and potential difficulties associated with each of these approaches to clicker training will be discussed in this article, with the aim that this will hopefully this will abate some of the confusion that can be experienced by owners new to clicker training.
m.nih.gov/pubmed/21310219
trainers reward from a treat held in the other hand. Obviously doing this training with horses who have not yet learnt not to mug is unwise. However, other than this caveat, the training can be very effective and enjoyable for both horse and trainer. Again, once the horse is reliably performing the behaviour with the target a cue is given and the guide gradually removed. The guide should not be the cue. This process should not take longer than a few sessions, especially for a basic behaviour. Interestingly, this is one of the most commonly used training methods employed by respected dog trainers. Again comprehensive instructions on how to successful use this method with your horse can be found on the internet or in appropriate books.
Emma Lethbridge Horsemanship 