Classification of complex movement patterns using computer vision and machine learning

Last changed: 20 February 2019

With the help of international, interdisciplinary research collaborations, we are developing smart technology that can be used to facilitate early and correct diagnosis of diseases in large animals.Such an automated recognition of pain can subsequently open the road for automated recognition of pain in a number of other relevant species, including cattle, swine and rodents.

Orthopedic injuries cause great suffering in horses and cows. Effective care is based on early detection of symptoms and subsequent correct diagnosis. Objective motion analysis has therefore been developed to increase the reliability in this assessment by using sensors or 3-D camera technology with markers attached to the animal's body. The technology is used both in research and in clinics. The systems are sensor- or optical based and measure asymmetries in the vertical movement of central body segments such as the pelvis and head.

But does today's objective motion measurement give us the whole truth? The assessment of movement patterns and behavioral interpretation of the horse is a complex task for veterinarians investigating orthopedic injuries. Most probably, the human brain uses pattern recognition to solve the task. Some experienced veterinarians criticize the objective movement systems and claim that they provide an incomplete picture of the movement pattern and miss relevant information. Perhaps there is a partial explanation for this in that we should look at the movement pattern using more advanced analysis techniques? Such technology is now available in the form of computer vision and machine learning.

With the help of funding from Formas and our international network, we are research how to use machine learning to utilize unexplored dimensions of the movement data that the objective systems collect. Through computer learning, a computer can be trained to classify very complex data (eg a movement pattern such as content or content) via training examples. By introducing this technology, we hope to create a smarter way of assessing the complex movement from the lame animal and thus get a more specific and sensitive health diagnosis.

In the two main research lines we operate in this field, painful conditions are detected in horses via analysis of facial expressions and body positioning and the analysis of movement patterns (also including cows). In collaboration with "computer vision" researchers from the Machine Learning Lab at the University of California (UC) Davis as well as the UC Davis Center for Equine Health we are applying the same machine learning technology that has been tested to automate the recognition of pain expression in humans today, we want to use horses with and without pain to "train" the machine learning system to discriminate between horses with mild to moderate pain and horses without pain. Our other project with body positioning is financed by the Swedish Research Council and is run together with our partners at KTH.

We have also started a pilot project that is financed by SLU Future Animals, Nature and Health, where an artificial 3D model of the horse's movement is built, which will give movement interpretation from ordinary video films. It is a collaboration with the Division of Robotics, Perception, and Learning (RPL) at KTH and Max Planck Institute for Intelligent Systems, Tübingen Germany.

The long-term goal is the development of a computer vision system that can help the veterinarian with a proper assessment of the lameness of horses and cows and intelligent monitoring systems in animal facilities that can detect lameness early. To that end, huge amounts of data are needed, a task we are currently intensively to collect.

We believe that smart technology will give us fundamentally new conditions for diagnostics in complex diseases. It can be an important building block for us to be able to keep animals in a responsible way in the future, where the animals receive the right care and treatment on time.



Elin Hernlund
Clinical Veterinarian, University Animal Hospital; Large Animal Clinic
Postdoctor at the Department of Anatomy, Physiology and Biochemistry (AFB); Division of Anatomy and Physiology

Telephone: 018-672142