AI systems for the airborne detection of plastic waste in the oceans

The drone image shows the test setup from a bird's eye view on the beach of Spiekeroog.

Robert Krell, everwave

Aircraft that routinely fly over bodies of water around the world to monitor pollution could in future not only detect oil and chemical spills on the high seas, in coastal waters and on beaches, but also plastic waste floating on the surface of the water. In the PlasticObs+ project, a consortium led by the German Research Center for Artificial Intelligence (DFKI) is working to develop the first airborne monitoring of larger, contiguous marine areas that continuously detects plastic in waters and not selectively, as has been the case so far. The first results are now available.

Plastic waste in waters continues to be a global, pressing environmental problem, as it endangers the marine ecosystem and thus a vital resource for humans and animals. Around ten million tonnes of plastic waste end up in the world's oceans every year. This corresponds to about one truck load per minute. Leftover bags, disposable packaging or drink bottles can be found all over the world, from the Arctic to the deep sea to the North and Baltic Seas.

Litter floating on the surface of the water has already been recorded by air in the past, but previous findings were mainly based on measurements limited in time and space. This is where the PlasticObs+ joint project comes in. The long-term goal is to equip surveillance aircraft, which are already routinely used around the world, with AI-supported sensor technology and thus develop a measuring system that can record the load of plastic waste in the environment from the air. In this way, a continuous and comprehensive inventory would be made possible for the first time, providing information on the type, quantity and size of the waste as well as possible sources of pollution. This would provide a scientific basis to initiate measures, laws and investments for the collection, recycling and ultimately the prevention of plastic waste.

The tasks of the DFKI, represented by the research department Marine Perception in Oldenburg, include the development of a total of four AI systems. The first two are to recognise plastic waste during the overflight and take a closer look at hotspots. A third system, which classifies the waste according to type, size and quantity, will be used later on the ground. Finally, a feedback system that includes human expertise in viewing the images will help to continuously improve the first systems and optimise their predictions.

The DFKI researchers obtain the data for their AI systems from test flights in northern Germany, which the Jade University of Applied Sciences Wilhelmshaven/Oldenburg/Elsfleth carries out with its research aircraft. A sensor under the nose of the aircraft takes overview images of the region. On these, the AI must recognise waste hotspots within seconds, so that a second sensor, located further back under the fuselage, takes detailed pictures of them. The challenge, say DFKI researchers Mattis Wolf and Dr Christoph Tholen, is that "on the one hand we are flying over a large area and the overview sensor is taking a low-resolution image of the scene, but on the other hand the evaluation has to be carried out within seconds but with a high degree of accuracy".

A first test took place last year on the island of Spiekeroog. The project consortium laid out a plastic test field on the beach and in the salt marshes. First, a drone flew over the field at heights between 15 and 100 metres, and then the research aircraft flew over it at heights from 150 to 1200 metres. The test field consisted of a precise arrangement of different types of plastic such as black PP coffee lids, PS white and cream lunch boxes as well as LDPE blue and transparent waste bags. The team fixed the containers in different sized clusters under nets so that no drifting could take place. The main question that the researchers wanted to answer with the campaign was: From what heights can the sensors on the drone or the aircraft reliably detect plastic waste? Project leader Wolf and his colleague Tholen consider the results so far positive, because "they showed that plastic can be detected with satisfactory accuracy at the heights we were aiming for".

It turned out that the colour and size of the objects as well as the background played an important role. On grass, for example, all types of plastic could be detected with high accuracy, with the exception of black PP coffee lids from more than 700 metres. On sand, the accuracy of all types of plastic decreases at 750 metres, with LDPE-transparent and again PP-black being particularly affected.

The researchers have recorded these and other results in a paper that they present these days at the OCEANS Conference 2023 in Limerick, Ireland. Carolin Leluschko, who also contributed to the scientific publication, says: "To find out how the AI performed, the images from the plane were each independently examined by five people and labelled whether they contained plastic or not." The accuracy of the AI was 93.3%, while the accuracy of the images labelled by humans was 92.6%.

Although the consortium still has a lot of work to do before the project ends in spring 2025, the figures prove that airborne remote sensing combined with AI methods works and can be an important tool in tackling the global plastic waste problem. PlasticObs+ is in line with policy initiatives such as the EU Marine Strategy Framework Directive and the UN Sustainable Development Goals, which aim to reduce marine pollution and protect the oceans. It is funded by the Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection (BMUV) with 1.9 million euros over three years and is part of the BMUV funding initiative AI Lighthouses for Environment, Climate, Nature and Resources.

In addition to the DFKI, three other partners are involved in PlastiObs+. The Jade University of Applied Sciences Wilhelmshaven/Oldenburg/Elsfleth is using its specially designed aircraft for research purposes, which serves as a measuring platform for taking aerial images. Optimare Systems GmbH from Bremerhaven, whose core business is the development and production of sensor systems and mission equipment for airborne marine surveillance, is contributing a method that generates high-resolution detailed images based on multi-sensor data. Optimare also takes care of the technology and its installation on an aircraft. everwave GmbH, which collects, sorts and recycles plastic waste in waters and on coasts worldwide using innovative waste collection boats and stationary river platforms, informs the public and raises awareness for a more sustainable use of plastic to protect the oceans.

One of the next steps in the PlasticObs+ project is to collect more data. To this end, the Jade University of Applied Sciences research aircraft recently took off again from Wilhelmshaven and flew over an area in northern Germany where festivals had previously taken place. Further field tests took place near Friedeburg in the Wittmund district of Lower Saxony, where the researchers had laid out an artificial litter carpet on two lakes, similar to the one on Spiekeroog before. "We use this data to train our AI models, which we will then test on the German coast with the research aircraft," explains Wolf, who is responsible in the project for, among other things, making computationally intensive and slow deep neural networks more efficient, so that the AI systems reliably find and record litter hotspots in seconds during the overflight. In a final step, the system will eventually be installed and tested in oil spill surveillance aircraft, probably in Brazil.

Machine-learning based discrimination of multiple signals can detect dissolved polymers