Automatic detection of pecan fruits based on Faster RCNN with FPN in orchard

Although the development of the robot picking vision system is widely applied, it is very challenging for fruit detection in orchards with complex light and environment, especially for fruit colors similar to the background. In recent, there are few studies on pecan fruit detection and location based on machine vision. In this study, an accurate and efficient pecan fruit detection method was proposed based on machine vision under natural pecan orchards. In order to solve the illumination problem, a light compensation algorithm was first utilized to process the collected samples, and then an improved Faster Region Convolutional Neural Network (Faster RCNN) with the Feature Pyramid Networks (FPN) was established to train the samples. Finally, the pecan number counting method was introduced to count the number of pecan. A total of 241 pecan images were tested, and comparison experiments were carried out. The mean average precision (mAP) of the proposed detection method was 95.932%, compared with the result without uneven illumination correction (UIC), which was increased by 0.849%, while the mAP of the Single Shot Detector (SSD)+FPN was 92.991%. In addition, the number of clusters was counted using the proposed method with an accuracy rate of 93.539% compared with the actual clusters. The results demonstrate that the proposed network has good robustness for pecan fruit detection in different illumination and various unstructured environments, and the experimental achievement has great potential for robot-picking visual systems.