Robot vision helps detect diseases and keep fish healthy
Robot vision has given us self-steering drones, and may also help keep an eye on salmon in fish pens and make sure that our children are healthy.
ROBOT VISION: Robots have been able to take over a number of tasks that are too dangerous, time consuming or simply not practical for humans to do. Annette Stahl knows all about this—she’s actually a mathematician, but her job is to give us robots that can see.
It takes advanced algorithms to allow robots to monitor fish. Photo: SINTEF IKT
“Robot vision is all about getting robots to understand and interact with their environment using optical sensors,” Stahl explains. She is an associate professor at the Department of Engineering Cybernetics.
Mechanical robots need to be integrated with computers that use different methods of collecting, processing, analysing and understanding images.
Robot vision is a useful and important tool in multiple fields that at first glance may seem quite different: It is important in everything from space flight and self-steering drones to salmon farming and keeping children healthy.
Healthier salmon
Stahl has spent the past few years focusing on monitoring salmon pens. There are a number of ways in which robots can help with this task, with the ultimate goal to have almost entirely automated surveillance to allow fish farmers to take the best possible care of the fish.
Maritime Robotics’ OceanEye Aerostat system. This is a balloon filled with helium with a camera that is capable of filming in daylight and with an infrared light at night. The data collected by the system is transferred wirelessly to the ground. Photo: SENSODRONE project SINTEF FH; Partner: Maritime Robotics
“Automation saves work and makes systems much more practical to use,” Stahl says.
The battle against salmon lice is high on the priority list. The system needs to be able to detect changes in the way that a fish acts—the way it moves or breathes may be able to show if it is stressed. Stressed fish are much more likely to die when they are treated for salmon lice, which is an important factor for the farmers.
“The system is useful for fish welfare, but it also makes it easier for us to see holes in pens and prevent fish from escaping. We can help optimize feeding and measure the number of fish in each pen, which allows farmers to be able to include their numbers when looking at market prices,” Stahl explains.
In short, her system helps yield healthier fish, but it is also directly profitable for farmers.
Robot vision is essential to reading salmon behaviour, but underwater conditions make this challenging. Water in different areas has differing clarity and particle content, and is always changing. The light conditions are difficult, and reflections from shiny salmon scales doesn’t make matters easier. The cameras and corresponding software need to be able to handle fish that move and change direction quickly.
A lot of work has been done, but plenty remains. This will still be part of Stahl’s work in the future.
Healthier children
Robot vision can also be useful to completely different fields of study– monitoring infant movements for signs of cerebral palsy (CP), for example.
Robot vision can be used to read an infant’s movements to determine if it has cerebral palsy. Photo: NTNU, St. Olavs Hospital
Early diagnosis is important when it comes to CP, so that children can get the treatment and care that they need. CP can be diagnosed in infants by analysing movements, but this usually takes a lot of time. Usually, diagnosis in infants requires health care personnel to conduct extended monitoring and comb through hours of video. Automating this process would mean a robot with robot vision could monitor the infant instead, freeing up health care workers limited time. Researchers at NTNU and St. Olavs hospital have developed a method that can do just this.
“We want this to be a tool. It won’t completely take over diagnosis,” Stahl says.
The diagnosis tool is currently 93.7 per cent correct. But the work is far from over, and there is always room to improve.
Much more
The possibilities are endless, the areas of use are varied within this field of study, and this robot vision expert is unlikely to be without a job any time soon.
Tesla is working on self-driving cars. Humanoid robot need vision to be able to work with humans, in areas like service jobs or health care. Robot vision is also completely essential to unmanned spacecraft of different types.
Annette Stahl. Photo: Thor Nielsen, NTNU
Underwater robots need to be able to navigate in varying water conditions. Floating, self-steering drones make it possible to monitor fish behavior in pens. Robots are need to inspect oil and natural gas facilities as wells as offshore wind turbines, this both in the water and air.
The mathematics behind it all is comprehensive and complex. This is where algorithms come in. Algorithms are essentially a precisely described, finite set of mathematical operations that are repeated to solve a problem. Stahl’s job is to make these algorithms.
Dream job
Stahl was recently hired at NTNU as part of the Onsager Fellowship Programme. Her position in robot vision is part of the university’s commitment to ocean research, but also falls within cybernetics, artificial intelligence and Big Data processing.
The Onsager Fellowship Programme
The NTNU Onsager Fellowship Programme was established to recruit young, internationally renowned researchers to strengthen NTNU's academic environment.
Researchers accepted to the programme have to have at least 3-5 years as a postdoc, expertise in their field, and be able to work independently. The positions are temporary, but with the possibility of a permanent position after the programme is done.
NTNU announced 12 positions in 2015. “We had 15-20 qualified applicants for each position. It was a tough contest, but we are looking forward to our researchers flourishing at NTNU,” says NTNU Rector Gunnar Bovim.
Source: NTNU Onsager Fellowship Programme
Her background is varied. Stahl did her doctorate in applied mathematics and computer vision, and has worked in computer vision, machine learning, mathematics, visualization, cybernetics and computer engineering at NTNU for a number of years, while also working in data processing and robot vision for SINTEF Fisheries and Aquaculture. She also has ties to NTNU AMOS, a centre of excellence at NTNU that works with autonomous marine operations and systems.
This moving around has made her especially well equipped for a position where she works with people in a number of different areas of study.
“I’ve jumped around a lot, but now everything is coming together. Computer vision, robotics, mathematics and cybernetics. This is my dream job,” she says.
Sources:
An Optical Flow-Based Method to Predict Infantile Cerebral Palsy. Annette Stahl, Christian Schellewald, Øyvind Stavdahl, Member, IEEE, Ole Morten Aamo, Lars Adde, and Harald Kirkerød.
Maskinsyn – økt presisjon og nye innovasjoner for bedre forståelse av laks? (Machine vision – increased precision and new innovations for better understanding of salmon?) Presented at TEKMAR 2.12.2015 – Trondheim for SINTEF Fisheries and Aquaculture AS.
