Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN

To solve the problem of high labour costs in the strawberry picking process, the approach of a strawberry picking robot to identify and find strawberries is suggested in this study. First, 1000 images including mature, immature, single, multiple, and occluded strawberries were collected, and a two-stage detection Mask R-CNN instance segmentation network and a one-stage detection YOLOv3 target detection network were used to train a strawberry identification model which classified strawberries into two categories: mature and immature. The accuracy ratings for YOLOv3 and Mask R-CNN were 93.4% and 94.5%, respectively. Second, the ZED stereo camera, triangulation, and a neural network were used to locate the strawberry in three dimensions. YOLOv3 identification accuracy was 3.1 mm, compared to Mask R-CNN of 3.9 mm. The strawberry detection and positioning method proposed in this study may effectively be used to supply the picking robot with a precise location of the ripe strawberry.