Smart Autumn Harvest | "New Employees" in the Tomato Planting Greenhouse Report

Produced by: Science Popularization China

Author: Kong Deyi, Wu Yinting (Institute of Intelligent Machinery, Hefei Institutes of Physical Science, Chinese Academy of Sciences)

Producer: China Science Expo

Smart Autumn Harvest Assistant Business Card

Little Assistant: Tomato Picking Robot

Brain: ROS system

Four core technologies: Deep learning algorithm improves fruit recognition rate; micro-camera determines fruit maturity; robotic arm realizes picking; trackless mobile chassis ensures flexible driving

Energy value: 5 stars

ID photo:

Tomato picking robot

(Image source: author)

Smart Autumn Harvest Assistant Self-Introduction

Everyone has eaten tomatoes, but who picked them? Me!

I am a tomato picking robot , and my main job is to pick tomatoes. I was born in 2021 and was created by researchers from the Hefei Institutes of Physical Science, Chinese Academy of Sciences. Although I am only 2 years old, I can already skillfully complete the automated picking of cherry tomatoes in indoor environments such as plant factories.

The reason why I can complete the picking work flexibly and efficiently is that the ROS system is implanted in my brain. ROS is the abbreviation of "Robot Operating System". It is a highly flexible open source software architecture that can utilize a large number of shared resources and greatly improve R&D efficiency. Therefore, it is being adopted by more and more robot developers.

Unlike human arms, I mainly use my robotic arms and hands to pick ripe tomatoes one by one.

I have a tiny camera installed on my arm, which is like my "eyes". It can help me judge the ripeness of the fruit by its color . For the ripe fruit, I can quickly calculate its location. Then the ROS system (my brain) will control the robot arm to move near the fruit, and then use the robot hand installed at the end of the robot arm to complete the picking of the fruit.

Tomatoes of different ripeness

(Photo source: veer photo gallery)

Is it ripe? How to pick it? I know all of this!

Using deep learning algorithms , I will conduct recognition training on a large number of cherry tomatoes in advance, which will greatly improve my recognition accuracy of the target; through the careful design of the picking robot and precise control of the robot arm's movement process, I can also improve the success rate of picking cherry tomatoes.

Generally speaking, the size of small tomatoes grown in a plant factory environment is not very different, so my robotic arm can accurately pick each small tomato without worrying about it falling. If there are large differences in the size of tomatoes between different varieties, the research team will also design a special picking robot for me.

Tomatoes are soft and have thin skins. To ensure that the fruits are not damaged during picking, my mechanical fingers are made of a special soft rubber material , so they will not be damaged during the process of picking. In addition, the surface of the soft rubber material is rough and has a large friction force, which can prevent the fruits from sliding and falling off in the "hand".

My picking targets are dwarf tomatoes. The fruit positions of small tomatoes are relatively scattered, so my picking method is to pick only one small tomato fruit at a time. It takes about 10 seconds to pick a single fruit.

For tall tomato plants with concentrated fruits, I can pick a bunch of small tomatoes at a time by replacing the robot arm.

Bunch of cherry tomatoes

(Photo source: veer photo gallery)

Some harvesting robots use a track-type mobile chassis, and the robot can only move along a fixed guide rail. This requires that the position of the tomato plant and the position of the robot guide rail must remain fixed and cannot be changed at will. It is not adaptable to the work site environment and is not suitable for semi-structured environments such as plant factories.

But I am different. I use a trackless mobile chassis that can move autonomously in all directions and over a large range. I can turn 360 degrees on the spot. It is flexible and maneuverable , especially suitable for operating in the narrow space between the cultivation racks in the plant factory. There is no need to lay rails or guide landmarks on the ground, which makes it more "friendly" to the plant factory.

Precision feeding: placement after picking

I have introduced to you many functions of my picking process, but are you curious about how I put the picked tomato fruits accurately into designated locations such as fruit trays and baskets after picking?

It’s actually very easy! I just need to detect the fruit plate target signal through the camera installed on my arm in advance, then locate the center of the fruit plate, and then send the position coordinates to my brain (ROS system). Then the ROS system will issue a command to control the robot arm and manipulator to move to the target position, release the fingers, and put the fruit accurately into the fruit plate.

Between my palm and the robot arm is my "forearm", which, on the one hand, extends the robot's operating space , allowing the robot to pick tomatoes farther away. On the other hand, a motor is installed in the "forearm" to control the opening and closing of the robot . When the motor rotates forward, the robot tightens and clamps the fruit and picks it off, then moves to the designated position, the motor rotates in the reverse direction, and the robot releases and puts the fruit down.

Message from the Assistant

Of course, my function is not just to assist in picking tomatoes. After researchers develop my technology, I can also pick other fruit and vegetable varieties, or perform pollination and other operations.

Although I am a robot and cannot taste the tomato like everyone else, it is a happy thing for me to let everyone taste the delicious tomato with my help. I will also have more new features in the future, so please look forward to my future performance!