Pest: How can eating expose my ID number? !

Spring has come, and farmers have started their spring ploughing. In the fields, a group of scientists are taking pictures of insects with poles. It turns out that "insect face recognition" has entered the "battlefield" again. Two articles will take you into the "black technology" of the Institute of Intelligent Machinery, Hefei Institutes of Physical Science, Chinese Academy of Sciences - "insect face recognition" technology.

Scientists' new job: Taking photos of bugs

"Insect face recognition" is a technology that is mainly based on artificial intelligence image recognition and detection technology, which enables the machine to automatically identify the type and number of pests in the current photo. It can assist field plant protection monitoring and reporting personnel and large farmers to determine the current extent of pests and diseases in the field, and provide decisions and suggestions for subsequent precise prevention and control, precise application of pesticides and rapid reporting.

Let's take a closer look at what this tool looks like.

The "insect face recognition" device includes a front-end camera (CCD camera), a mobile smart terminal (Mobile Terminal Client) and an algorithm server. (Image source: author)

The "bug face recognition" tool consists of three parts.

The first part is the front-end shooting device, which was independently developed by scientists from the Institute of Intelligent Machinery of the Hefei Institutes of Physical Science, Chinese Academy of Sciences. It is similar to a selfie stick. It has a high-definition camera on the front end, which can reach the roots of crops, the tops of fruit trees and other places that are difficult for investigators to enter and observe, making image collection more convenient.

The second part is the mobile smart terminal, which is equipped with a special survey APP. After the front-end is collected, this APP can quickly upload the selected survey images to the algorithm server in the background.

The algorithm server is the third part. After receiving these pictures, the algorithm server will analyze and comprehensively judge the information contained in these images based on artificial intelligence technology. The server will then return the recognition result data to the mobile terminal. The whole process only takes about 1 second.

When operating the "insect face recognition" device, extend the front-end camera into the site where the pest occurs to take pictures and upload them through the APP. You can immediately see on the terminal which pests are included in the current image and the number of pests. The machine will comprehensively evaluate the possible level of pest occurrence in the current field based on the recognition results of multiple sampling points, assist agricultural plant protection experts to complete rapid field surveys, and provide appropriate prevention and control suggestions.

These data not only provide real-time feedback on mobile devices, but are also stored in a cloud database. Staff can review them more carefully through a computer client, and can edit, annotate and download the results to complete the entire monitoring and reporting work.

Pest identification results (Image source: author)

The first step of "bug face recognition": replacing human face with "bug face"

Facial recognition technology has been applied to all aspects of our daily lives. Both "insect face recognition" and human face recognition are based on machine vision's recognition of objects contained in images, but changing the recognition object from a human face to a "bug face" is extremely difficult.

There are dozens of key points on a human face, and the machine can basically know a person's appearance by looking for eyes, nose, mouth, etc. However, "bug face" is different, and the machine cannot accurately identify it through simple key points set manually.

In the fields, there may be hundreds of pest species on my country's major cash crops; each pest may be in different ages and development stages, such as larvae and adults, resulting in very different appearances of the same pest. This means that "insect face recognition" in the field requires the identification of pests in multiple postures, types, and forms, which is much more difficult than human face recognition.

In order to identify insects of all shapes and sizes, the most important thing is to establish a database of "insect faces". At the beginning of the development of "insect face recognition" technology, data accumulation was the top priority for scientists. From 2016 to 2018, scientists from the Institute of Intelligent Machinery of the Hefei Institutes of Physical Science, Chinese Academy of Sciences, led students to live in almost every county and city in Anhui Province, collecting data on pests in the field and completing rapid data accumulation.

Scientists complete data collection in the field (Image source: author)

After the database is established, agricultural plant protection experts first analyze and organize the database based on their judgment of the pests, and then use artificial intelligence deep learning algorithms to allow the computer to automatically summarize and conclude the common characteristics of a certain type of pests - mouthparts, wing textures, patterns and spots on the back... These will become the pest characteristics automatically summarized by the computer, which is the "insect face."

After several years, we finally achieved intelligent recognition of "insect faces" just like recognizing human faces.

But the work of establishing the database did not stop. After the first version of the recognition system was built, scientists continued to collect images from local plant protection stations and other relevant channels. At the same time, they would make dozens of business trips abroad every year to collect relevant image data on site, so that the image database would continue to grow and improve, and the recognition rate of "insect face recognition" would increase year by year.

The “bug face” is so small, how can we recognize who it is?

As long as you have the three major supports, you can quickly report the "ID card" of the pest.

The first is the gradual development of camera technology. The resolution and image quality of the images we take now, whether it is a mobile phone, camera or fixed monitoring equipment, are gradually increasing, which is the basis for providing better and clearer pest image data.

The fall armyworm in the picture is very clear (Image source: author)

The second is a high-quality team of plant protection experts. They are experts from the Chinese Academy of Agricultural Sciences and the Provincial Plant Protection Station. They have a clear understanding of pests and are able to accurately identify and finely label relatively small pests, providing a high-quality data foundation for subsequent machine learning.

The third is the rapidly developing artificial intelligence technology. With the support of artificial intelligence technology, this equipment can identify very small pests. The current more mature technology is a way to simulate the visual perception of the human eye - "coarse to fine".

For example, when we look into the distance, we first see a vast panorama. When an object of interest appears in a certain area of ​​the panorama, our visual perception mechanism will mobilize the brain's attention, allowing us to observe more attentively and see the details of the object in this area.

The computer algorithm actually simulates the human visual perception process. It first roughly identifies the current image to determine where pests may appear in the image, then zooms in on the pest area, and finally further identifies the type and number of pests in the zoomed-in area. This method greatly improves the speed of artificial intelligence in recognizing images, saves computing resources, and also ensures the accuracy of identifying small pests.

In the field, the factors that most affect recognition accuracy are usually sunlight and shadows, which have a great impact on the target features photographed by the staff. Therefore, in order to collect high-quality images, scientists have also cooperated with relevant institutions to publish intelligent image acquisition standards for rice and wheat. The standards cover the specifications for correctly collecting various field pests and diseases summarized through experiments, thereby improving the standardization of data and improving data quality.

Wheat aphids under strong light, the color features are almost indistinguishable (Image source: author)

Pests of different ages also interfere greatly with image recognition, especially in the larval (caterpillar) stage. Even plant protection experts find it difficult to tell the difference between different types of larvae from just one picture. At this point, more information is needed, such as crop type, geographic location, collection time, temperature, humidity, etc., and then the type of pest can be determined based on experience.

When photographing pests, scientists will of course also photograph beneficial insects, but generally they will not include beneficial insects in the database. On the one hand, this is because beneficial insects account for a relatively small proportion of the total, and on the other hand, it is also to minimize the complexity of the identification database and improve the accuracy of identifying pests.

The "insect face recognition" technology has been relatively successful in identifying some major migratory pests and key explosive pests of wheat and rice. The recognition accuracy in the field is around 70% to 80%.

Not only can it identify insects, it can also "diagnose diseases"

Diseases, insect pests, and weeds are all extremely harmful to crops. For this technology, identifying insect pests is relatively simple, because individual pests are obvious enough and their numbers can be counted. However, in terms of disease identification, its algorithm is more complicated. Diseases may appear in the form of black spots, white spots, or curled leaves... The characteristics of each disease are completely different, which means that disease identification must take a different path from insect pest identification, and corresponding algorithm modeling work needs to be carried out according to the characteristics of each disease.

Rice grain smut (Image source: author)

Scientists have now basically completed the research on common wheat and rice diseases, and have basically reached the application level. Research on diseases of other crops still needs to be awaited.

Conclusion

Insect face recognition is similar to human face recognition in principle, but it is many orders of magnitude more difficult to recognize an insect face than a human face. So how useful is insect face recognition? What are the bitter experiences of researchers in the process of research? What is the future development direction of smart agriculture? Let's wait for the next time to find out.

Author: Du Jianming

This article is from the "Science Academy" public account. Please indicate the source of the public account when reprinting.