https://doi.org/10.4081/jae.2024.1549
Apple recognition and picking sequence planning for harvesting robot in a complex environment
Published: 31 October 2023
Abstract Views: 889
PDF: 267
SUPPLEMENTARY MATERIAL: 25
HTML: 4
SUPPLEMENTARY MATERIAL: 25
HTML: 4
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
In order to improve the efficiency of robots picking apples in challenging orchard environments, a method for precisely detecting apples and planning the picking sequence is proposed. Firstly, the EfficientFormer network serves as the foundation for YOLOV5, which uses the EF-YOLOV5s network to locate apples in difficult situations. Meanwhile, the soft non-maximum suppression algorithm is adopted to achieve accurate identification of overlapping apples. Secondly, the adjacently identified apples are automatically divided into different picking clusters by the improved density-based spatial clustering of applications with noise. Finally, the order of apple harvest is determined to guide the robot to complete the rapid picking, according to the weight of the Gauss distance weight combined with the significance level. In the experiment, the average precision of this method is 98.84%, which is 4.3% higher than that of YOLOV5s. Meanwhile, the average picking success rate and picking time are 94.8% and 2.86 seconds, respectively. Compared with sequential and random planning, the picking success rate of the proposed method is increased by 6.8% and 13.1%, respectively. The research proves that this method can accurately detect apples in complex environments and improve picking efficiency, which can provide technical support for harvesting robots.
Bodla N., Singh B., Chellappa R. 2017. Soft-NMS—improving object detection with one line of code. Proceedings of the IEEE International Conference on Computer Vision. pp. 5561-9.
DOI: https://doi.org/10.1109/ICCV.2017.593
Bu L., Chen C., Hu G. 2022. Design and evaluation of a robotic apple harvester using optimized picking patterns. Comput. Electron. Agric. 198:107092.
DOI: https://doi.org/10.1016/j.compag.2022.107092
Gangammanavar H., Sen S. 2021. Stochastic dynamic linear programming: A sequential sampling algorithm for multistage stochastic linear programming. SIAM J. Optim. 31:2111-40.
DOI: https://doi.org/10.1137/19M1290735
Gao F., Fu L., Zhang X. 2020. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Comput. Electron. Agric. 176:105634.
DOI: https://doi.org/10.1016/j.compag.2020.105634
Han K., Xiao A., Wu E. 2021. Transformer in transformer. Advances in Neural Information Proces. Syst. 34:15908-19.
Hu G., Chen C., Chen J., Sun L. 2022. Simplified 4-DOF manipulator for rapid robotic apple harvesting. Comput. Electron. Agric. 199:107-77.
DOI: https://doi.org/10.1016/j.compag.2022.107177
Ji W., Gao X., Xu B., Pan Y., Zhang Z., Zhao D. 2021. Apple target recognition method in complex environment based on improved YOLOv4. J. Food Process Eng. 44: e13866.
DOI: https://doi.org/10.1111/jfpe.13866
Ji W., Pan Y., Xu B., Wang J. 2022. A real-time apple targets detection method for picking robot based on ShufflenetV2- YOLOX. Agriculture. 12:856.
DOI: https://doi.org/10.3390/agriculture12060856
Ji W., Peng J., Xu B., Zhang T., 2023. Real-time detection of underwater river crab based on multi-scale pyramid fusion image enhancement and MobileCenterNet model. Comput. Electron. Agric. 204:107522.
DOI: https://doi.org/10.1016/j.compag.2022.107522
Jia W., Tian Y., Luo R. 2020. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 172:105380.
DOI: https://doi.org/10.1016/j.compag.2020.105380
Karcher C.J. 2022. Logspace sequential quadratic programming for design optimization. AIAA J. 60:1471-81.
DOI: https://doi.org/10.2514/1.J060950
Li Y., Yuan G., Wen Y. 2022. EfficientFormer: Vision transformers at mobileNet speed. arXiv preprint arXiv. 2206:01191.
Schubert E., Sander J., Ester M. 2017. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Trans. Database Syst. 42:1-21.
DOI: https://doi.org/10.1145/3068335
Swanepoel K.J. 1999. Cardinalities of k-distance sets in Minkowski spaces. Discrete Math. 197:759-67.
DOI: https://doi.org/10.1016/S0012-365X(99)90143-7
Sun T., Wang H.H., He D.J. 2018. Segmentation and picking sequence planning of clustered apples. Int. Agric. Eng. J. 27:309-17.
Tang Y., Zhou H., Wang H., Zhang Y. 2023a. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision. Expert Syst. Appl. 211:118573.
DOI: https://doi.org/10.1016/j.eswa.2022.118573
Tang Y., Qiu J., Zhang Y., Wu D., Cao Y., Zhao K., Zhu L. 2023b. Optimization strategies of fruit detection to overcome the challenge of unstructured background in field orchard environment: A review. Precis. Agric. 1-37.
DOI: https://doi.org/10.1007/s11119-023-10009-9
Wu F., Yang Z., Mo X., Wu Z., Tang W., Duan J., Zou X., 2023. Detection and counting of banana bunches by integrating deep learning and classic image-processing algorithms. Comput. Electron. Agric. 209:107827.
DOI: https://doi.org/10.1016/j.compag.2023.107827
Wu S., Li X., Wang X. 2020. IoU-aware single-stage object detector for accurate localization. Image Vis. Comput. 97:103911.
DOI: https://doi.org/10.1016/j.imavis.2020.103911
Wang W., Xu Z., Lu W. 2003. Determination of the spread parameter in the Gaussian kernel for classification and regression. Neurocomputing. 55:643-63.
DOI: https://doi.org/10.1016/S0925-2312(02)00632-X
Wang N., Joost W., Zhang F.S. 2016.Towards sustainable intensification of apple production in China-Yield gaps and nutrient use efficiency in apple farming systems. J. Integr. Agric 15:716-25.
DOI: https://doi.org/10.1016/S2095-3119(15)61099-1
Wang D., Song H., He D. 2017. Research advance on vision system of apple picking robot. Trans. the Chin. Soc. Agric. Eng. 33:59-69.
Xu B., Cui X., Ji W., Yuan H., Wang J. 2023. Apple grading method design and implementation for automatic grader based on Improved YOLOV5. Agriculture. 13:124.
DOI: https://doi.org/10.3390/agriculture13010124
Yu X.J., Fan Z.M., Wang X.D. 2021. A lab-customized autonomous humanoid apple harvesting robot. Comput. Electr. Eng. 96:107459.
DOI: https://doi.org/10.1016/j.compeleceng.2021.107459
Zhang F. 2016. Design of apple picking robot based on machine vision and binocular distance measurement. Int. J. Simul. Syst. Sci. Technol. 17.
Zhang K., Lammers K., Chu P. 2021. System design and control of an apple harvesting robot. Mechatronics. 79:102644.
DOI: https://doi.org/10.1016/j.mechatronics.2021.102644
Zhao D.A., Lv J., Wei J. 2011. Design and control of an apple harvesting robot. Biosyst. Eng. 110:112-22.
DOI: https://doi.org/10.1016/j.biosystemseng.2011.07.005
How to Cite
Ji, W. (2023) “Apple recognition and picking sequence planning for harvesting robot in a complex environment”, Journal of Agricultural Engineering, 55(1). doi: 10.4081/jae.2024.1549.
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- M. Hellstedt, K.O. Kaustell, T. Kivinen, The occupational safety on the construction sites of the farm production buildings in Finland , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- E. Quendler, P. Pötz, W. Hagmüller, R. Kogler, J. Boxberger, Determination of the working time requirement for suckling sows in the pen of Wels , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- Maria Nicolina Ripa, Francesco Ciapanna, Goffredo Filibeck, Federica Gobattoni, Antonio Leone, Raffaele Pelorosso, Matteo Piccinno, Carlo Maria Rossi, Fabio Recanatesi, Evolution of some Mediterranean landscapes of Central Italy from historical aerial photographs , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- F. Recanatesi, M. Tolli, M.N. Ripa, R. Pelorosso, F. Gobattoni, A. Leone, Detection of Landscape patterns in airborne LIDAR data in the Nature Reserve of Castelporziano (Rome) , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- Marco Vizzari, Maurizia Sigura, Urban-rural gradient detection using multivariate spatial analysis and landscape metrics , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- F. Sarghini, G. Pergher, Numerical and experimental analysis of vertical spray control patternators , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- Giovanni Russo, Giuseppe Verdiani, Production and reuse of waste in rural area with high density of greenhouse , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- Lei Liu, Xianliang Wang, Xiaokang Zhong, Xiangcai Zhang, Yuanle Geng, Hua Zhou, Tao Chen, Design and experiment of furrow side pick-up soil blade for wheat strip-till planter using the discrete element method , Journal of Agricultural Engineering: Vol. 55 No. 1 (2024)
- Pietro Catania, Filippa Bono, Claudio De Pasquale, Mariangela Vallone, Closed tank pneumatic press application to improve Sauvignon Blanc wine quality and nutraceutical properties , Journal of Agricultural Engineering: Vol. 50 No. 4 (2019)
- Simona M.C. Porto, Claudia Arcidiacono, Umberto Anguzza, Andrea Giummarra, Giovanni Cascone, An automatic system for the detection of dairy cows lying behaviour in free-stall barns , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
<< < 7 8 9 10 11 12 13 14 15 16 > >>
You may also start an advanced similarity search for this article.
