https://doi.org/10.4081/jae.2024.1549
Apple recognition and picking sequence planning for harvesting robot in a complex environment
Published: 31 October 2023
Abstract Views: 889
PDF: 267
SUPPLEMENTARY MATERIAL: 25
HTML: 4
SUPPLEMENTARY MATERIAL: 25
HTML: 4
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
In order to improve the efficiency of robots picking apples in challenging orchard environments, a method for precisely detecting apples and planning the picking sequence is proposed. Firstly, the EfficientFormer network serves as the foundation for YOLOV5, which uses the EF-YOLOV5s network to locate apples in difficult situations. Meanwhile, the soft non-maximum suppression algorithm is adopted to achieve accurate identification of overlapping apples. Secondly, the adjacently identified apples are automatically divided into different picking clusters by the improved density-based spatial clustering of applications with noise. Finally, the order of apple harvest is determined to guide the robot to complete the rapid picking, according to the weight of the Gauss distance weight combined with the significance level. In the experiment, the average precision of this method is 98.84%, which is 4.3% higher than that of YOLOV5s. Meanwhile, the average picking success rate and picking time are 94.8% and 2.86 seconds, respectively. Compared with sequential and random planning, the picking success rate of the proposed method is increased by 6.8% and 13.1%, respectively. The research proves that this method can accurately detect apples in complex environments and improve picking efficiency, which can provide technical support for harvesting robots.
Bodla N., Singh B., Chellappa R. 2017. Soft-NMS—improving object detection with one line of code. Proceedings of the IEEE International Conference on Computer Vision. pp. 5561-9.
DOI: https://doi.org/10.1109/ICCV.2017.593
Bu L., Chen C., Hu G. 2022. Design and evaluation of a robotic apple harvester using optimized picking patterns. Comput. Electron. Agric. 198:107092.
DOI: https://doi.org/10.1016/j.compag.2022.107092
Gangammanavar H., Sen S. 2021. Stochastic dynamic linear programming: A sequential sampling algorithm for multistage stochastic linear programming. SIAM J. Optim. 31:2111-40.
DOI: https://doi.org/10.1137/19M1290735
Gao F., Fu L., Zhang X. 2020. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Comput. Electron. Agric. 176:105634.
DOI: https://doi.org/10.1016/j.compag.2020.105634
Han K., Xiao A., Wu E. 2021. Transformer in transformer. Advances in Neural Information Proces. Syst. 34:15908-19.
Hu G., Chen C., Chen J., Sun L. 2022. Simplified 4-DOF manipulator for rapid robotic apple harvesting. Comput. Electron. Agric. 199:107-77.
DOI: https://doi.org/10.1016/j.compag.2022.107177
Ji W., Gao X., Xu B., Pan Y., Zhang Z., Zhao D. 2021. Apple target recognition method in complex environment based on improved YOLOv4. J. Food Process Eng. 44: e13866.
DOI: https://doi.org/10.1111/jfpe.13866
Ji W., Pan Y., Xu B., Wang J. 2022. A real-time apple targets detection method for picking robot based on ShufflenetV2- YOLOX. Agriculture. 12:856.
DOI: https://doi.org/10.3390/agriculture12060856
Ji W., Peng J., Xu B., Zhang T., 2023. Real-time detection of underwater river crab based on multi-scale pyramid fusion image enhancement and MobileCenterNet model. Comput. Electron. Agric. 204:107522.
DOI: https://doi.org/10.1016/j.compag.2022.107522
Jia W., Tian Y., Luo R. 2020. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 172:105380.
DOI: https://doi.org/10.1016/j.compag.2020.105380
Karcher C.J. 2022. Logspace sequential quadratic programming for design optimization. AIAA J. 60:1471-81.
DOI: https://doi.org/10.2514/1.J060950
Li Y., Yuan G., Wen Y. 2022. EfficientFormer: Vision transformers at mobileNet speed. arXiv preprint arXiv. 2206:01191.
Schubert E., Sander J., Ester M. 2017. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Trans. Database Syst. 42:1-21.
DOI: https://doi.org/10.1145/3068335
Swanepoel K.J. 1999. Cardinalities of k-distance sets in Minkowski spaces. Discrete Math. 197:759-67.
DOI: https://doi.org/10.1016/S0012-365X(99)90143-7
Sun T., Wang H.H., He D.J. 2018. Segmentation and picking sequence planning of clustered apples. Int. Agric. Eng. J. 27:309-17.
Tang Y., Zhou H., Wang H., Zhang Y. 2023a. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision. Expert Syst. Appl. 211:118573.
DOI: https://doi.org/10.1016/j.eswa.2022.118573
Tang Y., Qiu J., Zhang Y., Wu D., Cao Y., Zhao K., Zhu L. 2023b. Optimization strategies of fruit detection to overcome the challenge of unstructured background in field orchard environment: A review. Precis. Agric. 1-37.
DOI: https://doi.org/10.1007/s11119-023-10009-9
Wu F., Yang Z., Mo X., Wu Z., Tang W., Duan J., Zou X., 2023. Detection and counting of banana bunches by integrating deep learning and classic image-processing algorithms. Comput. Electron. Agric. 209:107827.
DOI: https://doi.org/10.1016/j.compag.2023.107827
Wu S., Li X., Wang X. 2020. IoU-aware single-stage object detector for accurate localization. Image Vis. Comput. 97:103911.
DOI: https://doi.org/10.1016/j.imavis.2020.103911
Wang W., Xu Z., Lu W. 2003. Determination of the spread parameter in the Gaussian kernel for classification and regression. Neurocomputing. 55:643-63.
DOI: https://doi.org/10.1016/S0925-2312(02)00632-X
Wang N., Joost W., Zhang F.S. 2016.Towards sustainable intensification of apple production in China-Yield gaps and nutrient use efficiency in apple farming systems. J. Integr. Agric 15:716-25.
DOI: https://doi.org/10.1016/S2095-3119(15)61099-1
Wang D., Song H., He D. 2017. Research advance on vision system of apple picking robot. Trans. the Chin. Soc. Agric. Eng. 33:59-69.
Xu B., Cui X., Ji W., Yuan H., Wang J. 2023. Apple grading method design and implementation for automatic grader based on Improved YOLOV5. Agriculture. 13:124.
DOI: https://doi.org/10.3390/agriculture13010124
Yu X.J., Fan Z.M., Wang X.D. 2021. A lab-customized autonomous humanoid apple harvesting robot. Comput. Electr. Eng. 96:107459.
DOI: https://doi.org/10.1016/j.compeleceng.2021.107459
Zhang F. 2016. Design of apple picking robot based on machine vision and binocular distance measurement. Int. J. Simul. Syst. Sci. Technol. 17.
Zhang K., Lammers K., Chu P. 2021. System design and control of an apple harvesting robot. Mechatronics. 79:102644.
DOI: https://doi.org/10.1016/j.mechatronics.2021.102644
Zhao D.A., Lv J., Wei J. 2011. Design and control of an apple harvesting robot. Biosyst. Eng. 110:112-22.
DOI: https://doi.org/10.1016/j.biosystemseng.2011.07.005
How to Cite
Ji, W. (2023) “Apple recognition and picking sequence planning for harvesting robot in a complex environment”, Journal of Agricultural Engineering, 55(1). doi: 10.4081/jae.2024.1549.
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Simone Pascuzzi, The effects of the forward speed and air volume of an air-assisted sprayer on spray deposition in tendone trained vineyards , Journal of Agricultural Engineering: Vol. 44 No. 3 (2013)
- Ossama M. M. Abdelwahab, Ronald L. Bingner, Fabio Milillo, Francesco Gentile, Effectiveness of alternative management scenarios on the sediment load in a Mediterranean agricultural watershed , Journal of Agricultural Engineering: Vol. 45 No. 3 (2014)
- Alessio Cislaghi, Gian Battista Bischetti, Best practices in post-flood surveys: The study case of Pioverna torrent , Journal of Agricultural Engineering: Vol. 53 No. 2 (2022)
- M. Grella, M. Manzone, F. Gioelli, P. Balsari, Harvesting of southern Piedmont’s orchards pruning residues: evaluations of biomass production and harvesting losses , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- Jingtao Li, Hao Chen, Guisong Li, Yueqi Liu, Yanli Yang, Xia Liu, Chang Yi Wang, Detection method of potato leaf disease based on YOLOv5s , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Juan Ãngel Mintegui Aguirre, José Carlos Robredo Sánchez, Carlos de Gonzalo Aranoa, Pablo Huelin Rueda, Jorge Fallas, Felipe Cisneros, Pedro Cisneros, Adriana Urciuolo, Rodolfo Iturraspe, Forest use strategies in watershed management and restoration: application to three small mountain watersheds in Latin America , Journal of Agricultural Engineering: Vol. 45 No. 1 (2014)
- Giuseppe Ruggiero, Stefano Dal Sasso, Rosa Viviana Loisi, Giuseppe Verdiani, Characteristics and distribution of trulli constructions in the area of the site of community importance Murgia of Trulli , Journal of Agricultural Engineering: Vol. 44 No. 2 (2013)
- Vincenzo Bagarello, Vito Ferro, Dennis Flanagan, Predicting plot soil loss by empirical and process-oriented approaches. A review , Journal of Agricultural Engineering: Vol. 49 No. 1 (2018)
- Maria Elena Menconi, Massimo Chiappini, Jan L.M. Hensen, David Grohmann, Thermal comfort optimisation of vernacular rural buildings: passive solutions to retrofit a typical farmhouse in central Italy , Journal of Agricultural Engineering: Vol. 48 No. 3 (2017)
- Alessandro Toccolini, Simone Felisari, Paolo Stefano Ferrario, Design of green spaces located below the urbanised level. Themes, problems and solutions applied to a case study , Journal of Agricultural Engineering: Vol. 46 No. 4 (2015)
<< < 1 2 3 4 5 6 7 8 9 10 > >>
You may also start an advanced similarity search for this article.
