When autumn harvest becomes intelligent: Can you diagnose crops by just taking a photo with your mobile phone?

Produced by: Science Popularization China

Author: Du Jianming (Hefei Institutes of Physical Science, Chinese Academy of Sciences)

Producer: China Science Expo

Smart Autumn Harvest Assistant Business Card

Assistant: Automatic identification and remote diagnosis system for crop diseases, insect pests and weeds

Two magic weapons: automatic recognition of pests, diseases and weeds; multi-terminal mode

Three key technologies: automatic image segmentation technology based on depth information; image feature extraction technology based on multi-feature fusion; image classification and recognition technology based on machine learning

Energy value: 5 stars

ID photo:

Including front-end camera equipment (CCD camera), mobile intelligent terminal (Mobile Terminal Client) and algorithm server (Image source: author)

Smart Autumn Harvest Assistant Self-Introduction

Nice to meet you. I am the Automatic Identification and Remote Diagnosis System for Crop Diseases, Insects and Weeds , and a member of the big family of Hefei Institutes of Physical Science, Chinese Academy of Sciences.

We all know that crop diseases, insect pests and weeds have always been one of the most troublesome problems for the planting industry. However, with my help, crop diseases, insect pests and weeds can be easily solved.

The eaten corn leaves (Photo source: veer photo gallery)

Two core functions

As an intelligent system for preventing and controlling crop diseases, insect pests and weeds, I have to introduce to you my "two magic weapons" - automatic recognition of disease, insect pest and weed images and multi-terminal mode.

The realization of automatic recognition of pests, diseases and weeds images and multi-terminal adaptation mode is inseparable from the high-performance image recognition algorithm for pests, diseases and weeds developed by the scientific research team and the data server built within the Chinese Academy of Sciences.

First of all, the high-performance server of pests and diseases on my body can not only realize high-precision and accurate recognition of images collected by various channels (such as mobile phones, cameras and even surveillance cameras), but also lightweight and high-speed processing of the recognition model, so that the model can ensure data processing speed while ensuring high recognition rate when processing a large number of pictures at the same time.

In addition, the research team packaged the recognition model into the form of interface calls, which can achieve data transmission, access, analysis and result acquisition through standardized transmission protocols and fields.

As a result, I can have the ability to use the same recognition model on multiple terminals at the same time. It is also necessary to emphasize that the data processing server is placed in the data center of the Chinese Academy of Sciences, which provides a nationally leading secure and stable environment for the transmission and calculation of various types of data, and is a solid guarantee for the normal operation of the system in my "brain".

My other magic weapon: in multi-terminal mode, my mobile phone mode is the pioneer of field survey . It can not only be used with the field survey probe developed by the scientific research team, but also directly collect data through the mobile phone camera.

Automatic identification of pests, diseases and weeds on mobile phone (Image source: author)

Mobile phones have the ability to collect images and geographic environment information, and can upload images and information through the Internet. In just a few seconds, information such as the types of pests and weeds in the image, occurrence assessments, and commonly used prevention and control methods can be obtained.

My Computer Mode is designed for users to organize and report data . Here, users can not only view all images, recognition results and environmental geographic information uploaded by their mobile phones, but also see professional prevention and control drug use guidelines given by agricultural experts.

In addition, this mode also supports the result export function, allowing users to obtain complete field survey data tables with simple operations, making it easier for them to report.

Three key technologies

If I only have these two magic weapons, I will not be fully qualified for the work of crop pest control. I also have three key technologies , which are automatic image segmentation technology based on depth information, image feature extraction technology based on multi-feature fusion, and image classification and recognition technology based on machine learning.

My research and development team has been at the forefront of the field of intelligent identification of pests and diseases at home and abroad since its establishment. Every year, it produces a large number of excellent research results combining artificial intelligence and agriculture, which are published in the world's top journals.

Among them, automatic image segmentation, multi-feature fusion extraction, and image classification and recognition technology are only a general direction. In fact, each direction can be subdivided into several small directions to solve the prominent and difficult problems encountered in the current identification of agricultural pests and diseases.

Pest identification results (Image source: author)

For example, in automatic segmentation technology, the research team has developed a surface pest detection and segmentation algorithm specifically for stacked pests, and also developed a multi-layer refinement segmentation algorithm for densely clustered extremely small pests; in multi-feature fusion extraction, the research team not only studied the relationship between crops and pest species based on environmental context, but also studied the potential correlation between pests and their posture and morphological characteristics; in classification and recognition technology, the research team not only proposed multiple high-grained recognition algorithms, but also proposed special classification technologies for a variety of extremely similar moths.

With two core functions and three key technologies, I can replace plant protection experts to effectively and timely complete automated crop diagnosis, thereby reducing pesticide use and improving the quality of agricultural products.

Database and plant protection experts

In order to ensure that I can always maintain a high level of recognition accuracy and precision, the database in my system has been constantly iterating.

At present, in addition to the scientific research team organizing field data collection in the outbreak areas of pests and diseases in Anhui Province every year, it also communicates with the Anhui Provincial Plant Protection Station and many county plant protection stations in Anhui Province, and invites experts from the Anhui Academy of Agricultural Sciences for guidance.

The research team collects data in the field (Image source: author)

My database has been growing at a rate of more than 100,000 images per year. Don’t you think that both I and the scientists who develop me are amazing?

Currently, whether it is the mobile APP or WEB terminal, the latest database and algorithm library will be displayed, and it will be updated and pushed from time to time, striving to allow users to use my latest technology.

In addition, my current function is not only to identify pests and diseases, but also to provide many new functions for everyone, including directly calling the built-in geographic meteorological function of the mobile phone to realize multi-information recording, or to view historical records at any time on the WEB side and export them with one click, which is convenient for field investigators to output survey results.

In addition, the most important point is that the scientific research team has cooperated with the Anhui Provincial Plant Protection Station since 2019 and carried out a three-year benchmarking experiment between artificial intelligence and manual field surveys, conducting statistical intelligent identification of eight common field pests and diseases and comparative analysis of the evaluation results between manual surveys.

This enables me to directly provide users with field survey pest and disease occurrence level assessments based on artificial intelligence identification results. Users no longer need to record and calculate by themselves, which greatly improves the user's survey speed and intelligence level.

In order to better carry out pest and disease control, I also have many partners - plant protection experts .

The research team has maintained a close cooperative relationship with experts from the Plant Protection Center and Anhui Academy of Agricultural Sciences. Although I can accurately identify pests and diseases and provide prevention and control suggestions, in fact, all the knowledge bases are carefully compiled and verified by them one by one, and have good scientificity and applicability. At the same time, the expert knowledge base in my system will be updated regularly.

At present, the scientific research team will also organize users to communicate with experts, and is planning to go further on the basis of the current identification system, establish an expert communication and question-and-answer channel, and organize experts in the agricultural field to answer questions raised by users.

Message from the Assistant

Of course, my system is still being iterated and gradually adding functions to provide everyone with a better user experience.

As one of the intelligent agricultural machines serving my country's agricultural production, my development also urgently needs the efforts and wisdom of various scientific researchers with ideas and strength. The research team of the Hefei Institute of Physical Science of the Chinese Academy of Sciences also hopes that when it matures, it can open the development qualifications of me to the majority of developers in the form of software interface API, so that more teams with ideas and ideas can apply this technology to a wider agricultural field and provide more complete, professional and intelligent services for the majority of agricultural workers.