https://doi.org/10.4081/jae.2024.1549 Apple recognition and picking sequence planning for harvesting robot in a complex environment
Published:31 October 2023
Abstract Views: 913
PDF: 306
SUPPLEMENTARY MATERIAL: 27
HTML: 4
SUPPLEMENTARY MATERIAL: 27
HTML: 4
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
In order to improve the efficiency of robots picking apples in challenging orchard environments, a method for precisely detecting apples and planning the picking sequence is proposed. Firstly, the EfficientFormer network serves as the foundation for YOLOV5, which uses the EF-YOLOV5s network to locate apples in difficult situations. Meanwhile, the soft non-maximum suppression algorithm is adopted to achieve accurate identification of overlapping apples. Secondly, the adjacently identified apples are automatically divided into different picking clusters by the improved density-based spatial clustering of applications with noise. Finally, the order of apple harvest is determined to guide the robot to complete the rapid picking, according to the weight of the Gauss distance weight combined with the significance level. In the experiment, the average precision of this method is 98.84%, which is 4.3% higher than that of YOLOV5s. Meanwhile, the average picking success rate and picking time are 94.8% and 2.86 seconds, respectively. Compared with sequential and random planning, the picking success rate of the proposed method is increased by 6.8% and 13.1%, respectively. The research proves that this method can accurately detect apples in complex environments and improve picking efficiency, which can provide technical support for harvesting robots.
Bodla N., Singh B., Chellappa R. 2017. Soft-NMS—improving object detection with one line of code. Proceedings of the IEEE International Conference on Computer Vision. pp. 5561-9. DOI: https://doi.org/10.1109/ICCV.2017.593
Bu L., Chen C., Hu G. 2022. Design and evaluation of a robotic apple harvester using optimized picking patterns. Comput. Electron. Agric. 198:107092. DOI: https://doi.org/10.1016/j.compag.2022.107092
Gangammanavar H., Sen S. 2021. Stochastic dynamic linear programming: A sequential sampling algorithm for multistage stochastic linear programming. SIAM J. Optim. 31:2111-40. DOI: https://doi.org/10.1137/19M1290735
Gao F., Fu L., Zhang X. 2020. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Comput. Electron. Agric. 176:105634. DOI: https://doi.org/10.1016/j.compag.2020.105634
Han K., Xiao A., Wu E. 2021. Transformer in transformer. Advances in Neural Information Proces. Syst. 34:15908-19.
Hu G., Chen C., Chen J., Sun L. 2022. Simplified 4-DOF manipulator for rapid robotic apple harvesting. Comput. Electron. Agric. 199:107-77. DOI: https://doi.org/10.1016/j.compag.2022.107177
Ji W., Gao X., Xu B., Pan Y., Zhang Z., Zhao D. 2021. Apple target recognition method in complex environment based on improved YOLOv4. J. Food Process Eng. 44: e13866. DOI: https://doi.org/10.1111/jfpe.13866
Ji W., Pan Y., Xu B., Wang J. 2022. A real-time apple targets detection method for picking robot based on ShufflenetV2- YOLOX. Agriculture. 12:856. DOI: https://doi.org/10.3390/agriculture12060856
Ji W., Peng J., Xu B., Zhang T., 2023. Real-time detection of underwater river crab based on multi-scale pyramid fusion image enhancement and MobileCenterNet model. Comput. Electron. Agric. 204:107522. DOI: https://doi.org/10.1016/j.compag.2022.107522
Jia W., Tian Y., Luo R. 2020. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 172:105380. DOI: https://doi.org/10.1016/j.compag.2020.105380
Karcher C.J. 2022. Logspace sequential quadratic programming for design optimization. AIAA J. 60:1471-81. DOI: https://doi.org/10.2514/1.J060950
Li Y., Yuan G., Wen Y. 2022. EfficientFormer: Vision transformers at mobileNet speed. arXiv preprint arXiv. 2206:01191.
Schubert E., Sander J., Ester M. 2017. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Trans. Database Syst. 42:1-21. DOI: https://doi.org/10.1145/3068335
Swanepoel K.J. 1999. Cardinalities of k-distance sets in Minkowski spaces. Discrete Math. 197:759-67. DOI: https://doi.org/10.1016/S0012-365X(99)90143-7
Sun T., Wang H.H., He D.J. 2018. Segmentation and picking sequence planning of clustered apples. Int. Agric. Eng. J. 27:309-17.
Tang Y., Zhou H., Wang H., Zhang Y. 2023a. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision. Expert Syst. Appl. 211:118573. DOI: https://doi.org/10.1016/j.eswa.2022.118573
Tang Y., Qiu J., Zhang Y., Wu D., Cao Y., Zhao K., Zhu L. 2023b. Optimization strategies of fruit detection to overcome the challenge of unstructured background in field orchard environment: A review. Precis. Agric. 1-37. DOI: https://doi.org/10.1007/s11119-023-10009-9
Wu F., Yang Z., Mo X., Wu Z., Tang W., Duan J., Zou X., 2023. Detection and counting of banana bunches by integrating deep learning and classic image-processing algorithms. Comput. Electron. Agric. 209:107827. DOI: https://doi.org/10.1016/j.compag.2023.107827
Wu S., Li X., Wang X. 2020. IoU-aware single-stage object detector for accurate localization. Image Vis. Comput. 97:103911. DOI: https://doi.org/10.1016/j.imavis.2020.103911
Wang W., Xu Z., Lu W. 2003. Determination of the spread parameter in the Gaussian kernel for classification and regression. Neurocomputing. 55:643-63. DOI: https://doi.org/10.1016/S0925-2312(02)00632-X
Wang N., Joost W., Zhang F.S. 2016.Towards sustainable intensification of apple production in China-Yield gaps and nutrient use efficiency in apple farming systems. J. Integr. Agric 15:716-25. DOI: https://doi.org/10.1016/S2095-3119(15)61099-1
Wang D., Song H., He D. 2017. Research advance on vision system of apple picking robot. Trans. the Chin. Soc. Agric. Eng. 33:59-69.
Xu B., Cui X., Ji W., Yuan H., Wang J. 2023. Apple grading method design and implementation for automatic grader based on Improved YOLOV5. Agriculture. 13:124. DOI: https://doi.org/10.3390/agriculture13010124
Yu X.J., Fan Z.M., Wang X.D. 2021. A lab-customized autonomous humanoid apple harvesting robot. Comput. Electr. Eng. 96:107459. DOI: https://doi.org/10.1016/j.compeleceng.2021.107459
Zhang F. 2016. Design of apple picking robot based on machine vision and binocular distance measurement. Int. J. Simul. Syst. Sci. Technol. 17.
Zhang K., Lammers K., Chu P. 2021. System design and control of an apple harvesting robot. Mechatronics. 79:102644. DOI: https://doi.org/10.1016/j.mechatronics.2021.102644
Zhao D.A., Lv J., Wei J. 2011. Design and control of an apple harvesting robot. Biosyst. Eng. 110:112-22. DOI: https://doi.org/10.1016/j.biosystemseng.2011.07.005
How to Cite
Ji, W. (2023) “Apple recognition and picking sequence planning for harvesting robot in a complex environment”, Journal of Agricultural Engineering, 55(1). doi: 10.4081/jae.2024.1549.
Copyright (c) 2023 the Author(s) 
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Zhiqiang Song, Chuanxing Du, Yongcheng Chen, Dianlei Han, Xinzhong Wang, Development and test of a spring-finger roller-type hot pepper picking header , Journal of Agricultural Engineering: Vol. 55 No. 2 (2024)
- Hongbo Wang, Zhicheng Xie, Yongzheng Yang, Junmao Li, Zilu Huang, Zhihong Yu, Fast identification of tomatoes in natural environments by improved YOLOv5s , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Ziyan Fan, Lijun Li, Zicheng Gao, Fuzzy neural network PID control design of camellia fruit vibration picking manipulator , Journal of Agricultural Engineering: Vol. 54 No. 2 (2023)
- Jingtao Li, Hao Chen, Guisong Li, Yueqi Liu, Yanli Yang, Xia Liu, Chang Yi Wang, Detection method of potato leaf disease based on YOLOv5s , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Xu Xiao, Yaonan Wang, Yiming Jiang, Haotian Wu, Zhe Zhang, Rujing Wang, AC-YOLO: citrus detection in the natural environment of orchards , Journal of Agricultural Engineering: Vol. 55 No. 4 (2024)
- Juquan Ruan, Shuo Liu, Wanjing Mao, Shan Zeng, Zhuoyi Zhang, Guangsun Yin, Fine-grained recognition algorithm of crop pests based on cross-layer bilinear aggregation and multi-task learning , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Shanwen Zhang, Yongyuan Sun, Su Lu, Li Wang, Sian Liu, Zhongliu Wang, Min Dai, Jicheng Gao, Hong Miao, Design and experiment of brush-roller ginkgo leaf picker for the dwarf dense planting mode , Journal of Agricultural Engineering: Vol. 54 No. 4 (2023)
- Qingyu Chen, Rui Kang, Naishuo Wei, Yunlei Fan, Zeyu Wang, Yu Chen, Jun Chen, Design and experiment optimize of the vibration harvesting machine of Lycium barbarum L. , Journal of Agricultural Engineering: Vol. 55 No. 4 (2024)
- Guomin Gao, Hui Guo, Jing Zhang, Zhenguo Zhang, Tianlun Wu, Hao Lu, Zhaoxin Qiu, Haiyang Chen, Zhen Lingxuan, An efficient headland-turning navigation system for a safflower picking robot , Journal of Agricultural Engineering: Vol. 54 No. 3 (2023)
- Changjian Zhou, Zhenyuan Zhao, Wenzhuo Chen, Yuquan Feng, Jia Song, Wensheng Xiang, Residual attention based multi-label learning for apple leaf disease identification , Journal of Agricultural Engineering: Vol. 55 No. 4 (2024)
You may also start an advanced similarity search for this article.
