https://doi.org/10.4081/jae.2024.1549
Apple recognition and picking sequence planning for harvesting robot in a complex environment
Published: 31 October 2023
Abstract Views: 889
PDF: 267
SUPPLEMENTARY MATERIAL: 25
HTML: 4
SUPPLEMENTARY MATERIAL: 25
HTML: 4
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
In order to improve the efficiency of robots picking apples in challenging orchard environments, a method for precisely detecting apples and planning the picking sequence is proposed. Firstly, the EfficientFormer network serves as the foundation for YOLOV5, which uses the EF-YOLOV5s network to locate apples in difficult situations. Meanwhile, the soft non-maximum suppression algorithm is adopted to achieve accurate identification of overlapping apples. Secondly, the adjacently identified apples are automatically divided into different picking clusters by the improved density-based spatial clustering of applications with noise. Finally, the order of apple harvest is determined to guide the robot to complete the rapid picking, according to the weight of the Gauss distance weight combined with the significance level. In the experiment, the average precision of this method is 98.84%, which is 4.3% higher than that of YOLOV5s. Meanwhile, the average picking success rate and picking time are 94.8% and 2.86 seconds, respectively. Compared with sequential and random planning, the picking success rate of the proposed method is increased by 6.8% and 13.1%, respectively. The research proves that this method can accurately detect apples in complex environments and improve picking efficiency, which can provide technical support for harvesting robots.
Bodla N., Singh B., Chellappa R. 2017. Soft-NMS—improving object detection with one line of code. Proceedings of the IEEE International Conference on Computer Vision. pp. 5561-9.
DOI: https://doi.org/10.1109/ICCV.2017.593
Bu L., Chen C., Hu G. 2022. Design and evaluation of a robotic apple harvester using optimized picking patterns. Comput. Electron. Agric. 198:107092.
DOI: https://doi.org/10.1016/j.compag.2022.107092
Gangammanavar H., Sen S. 2021. Stochastic dynamic linear programming: A sequential sampling algorithm for multistage stochastic linear programming. SIAM J. Optim. 31:2111-40.
DOI: https://doi.org/10.1137/19M1290735
Gao F., Fu L., Zhang X. 2020. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Comput. Electron. Agric. 176:105634.
DOI: https://doi.org/10.1016/j.compag.2020.105634
Han K., Xiao A., Wu E. 2021. Transformer in transformer. Advances in Neural Information Proces. Syst. 34:15908-19.
Hu G., Chen C., Chen J., Sun L. 2022. Simplified 4-DOF manipulator for rapid robotic apple harvesting. Comput. Electron. Agric. 199:107-77.
DOI: https://doi.org/10.1016/j.compag.2022.107177
Ji W., Gao X., Xu B., Pan Y., Zhang Z., Zhao D. 2021. Apple target recognition method in complex environment based on improved YOLOv4. J. Food Process Eng. 44: e13866.
DOI: https://doi.org/10.1111/jfpe.13866
Ji W., Pan Y., Xu B., Wang J. 2022. A real-time apple targets detection method for picking robot based on ShufflenetV2- YOLOX. Agriculture. 12:856.
DOI: https://doi.org/10.3390/agriculture12060856
Ji W., Peng J., Xu B., Zhang T., 2023. Real-time detection of underwater river crab based on multi-scale pyramid fusion image enhancement and MobileCenterNet model. Comput. Electron. Agric. 204:107522.
DOI: https://doi.org/10.1016/j.compag.2022.107522
Jia W., Tian Y., Luo R. 2020. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 172:105380.
DOI: https://doi.org/10.1016/j.compag.2020.105380
Karcher C.J. 2022. Logspace sequential quadratic programming for design optimization. AIAA J. 60:1471-81.
DOI: https://doi.org/10.2514/1.J060950
Li Y., Yuan G., Wen Y. 2022. EfficientFormer: Vision transformers at mobileNet speed. arXiv preprint arXiv. 2206:01191.
Schubert E., Sander J., Ester M. 2017. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Trans. Database Syst. 42:1-21.
DOI: https://doi.org/10.1145/3068335
Swanepoel K.J. 1999. Cardinalities of k-distance sets in Minkowski spaces. Discrete Math. 197:759-67.
DOI: https://doi.org/10.1016/S0012-365X(99)90143-7
Sun T., Wang H.H., He D.J. 2018. Segmentation and picking sequence planning of clustered apples. Int. Agric. Eng. J. 27:309-17.
Tang Y., Zhou H., Wang H., Zhang Y. 2023a. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision. Expert Syst. Appl. 211:118573.
DOI: https://doi.org/10.1016/j.eswa.2022.118573
Tang Y., Qiu J., Zhang Y., Wu D., Cao Y., Zhao K., Zhu L. 2023b. Optimization strategies of fruit detection to overcome the challenge of unstructured background in field orchard environment: A review. Precis. Agric. 1-37.
DOI: https://doi.org/10.1007/s11119-023-10009-9
Wu F., Yang Z., Mo X., Wu Z., Tang W., Duan J., Zou X., 2023. Detection and counting of banana bunches by integrating deep learning and classic image-processing algorithms. Comput. Electron. Agric. 209:107827.
DOI: https://doi.org/10.1016/j.compag.2023.107827
Wu S., Li X., Wang X. 2020. IoU-aware single-stage object detector for accurate localization. Image Vis. Comput. 97:103911.
DOI: https://doi.org/10.1016/j.imavis.2020.103911
Wang W., Xu Z., Lu W. 2003. Determination of the spread parameter in the Gaussian kernel for classification and regression. Neurocomputing. 55:643-63.
DOI: https://doi.org/10.1016/S0925-2312(02)00632-X
Wang N., Joost W., Zhang F.S. 2016.Towards sustainable intensification of apple production in China-Yield gaps and nutrient use efficiency in apple farming systems. J. Integr. Agric 15:716-25.
DOI: https://doi.org/10.1016/S2095-3119(15)61099-1
Wang D., Song H., He D. 2017. Research advance on vision system of apple picking robot. Trans. the Chin. Soc. Agric. Eng. 33:59-69.
Xu B., Cui X., Ji W., Yuan H., Wang J. 2023. Apple grading method design and implementation for automatic grader based on Improved YOLOV5. Agriculture. 13:124.
DOI: https://doi.org/10.3390/agriculture13010124
Yu X.J., Fan Z.M., Wang X.D. 2021. A lab-customized autonomous humanoid apple harvesting robot. Comput. Electr. Eng. 96:107459.
DOI: https://doi.org/10.1016/j.compeleceng.2021.107459
Zhang F. 2016. Design of apple picking robot based on machine vision and binocular distance measurement. Int. J. Simul. Syst. Sci. Technol. 17.
Zhang K., Lammers K., Chu P. 2021. System design and control of an apple harvesting robot. Mechatronics. 79:102644.
DOI: https://doi.org/10.1016/j.mechatronics.2021.102644
Zhao D.A., Lv J., Wei J. 2011. Design and control of an apple harvesting robot. Biosyst. Eng. 110:112-22.
DOI: https://doi.org/10.1016/j.biosystemseng.2011.07.005
How to Cite
Ji, W. (2023) “Apple recognition and picking sequence planning for harvesting robot in a complex environment”, Journal of Agricultural Engineering, 55(1). doi: 10.4081/jae.2024.1549.
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Fernando Ferreira Abreu, Luiz Henrique Antunes Rodrigues, Monitoring mini-tomatoes growth: A non-destructive machine vision-based alternative , Journal of Agricultural Engineering: Vol. 53 No. 3 (2022)
- Juquan Ruan, Shuo Liu, Wanjing Mao, Shan Zeng, Zhuoyi Zhang, Guangsun Yin, Fine-grained recognition algorithm of crop pests based on cross-layer bilinear aggregation and multi-task learning , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Pankaj Tyagi, Rahul Semwal, Anju Sharma, Uma Shanker Tiwary, Pritish Varadwaj, E-nose: a low-cost fruit ripeness monitoring system , Journal of Agricultural Engineering: Vol. 54 No. 1 (2023)
- Ernest Ekow Abano, Hai Le Ma, Wenjuan Qu, Thin-layer catalytic far-infrared radiation drying and flavour of tomato slices , Journal of Agricultural Engineering: Vol. 45 No. 1 (2014)
- Zhongwei Hua, Min Guan, Lightweight sandy vegetation object detection algorithm based on attention mechanism , Journal of Agricultural Engineering: Vol. 54 No. 1 (2023)
- Yevgeny Beiderman, Mark Kunin, Eli Kolberg, Ilan Halachmi, Binyamin Abramov, Rafael Amsalem, Zeev Zalevsky, Automatic solution for detection, identification and biomedical monitoring of a cow using remote sensing for optimised treatment of cattle , Journal of Agricultural Engineering: Vol. 45 No. 4 (2014)
- Pasquale Dal Sasso, Giuseppe Ruggiero, Maria Antonella Ottolino, Giuseppe Verdiani, The role of agroforestry areas of the province of Bari in the absortion of carbon dioxide , Journal of Agricultural Engineering: Vol. 43 No. 1 (2012)
- Antonio Leone, Luca Ceccarelli, M. Nicolina Ripa, Fabio Recanatesi, CONSIDERATIONS REGARDING THE PLANNING OF PROTECTED AREAS ARISING FROM A STUDY OF THE TUSCANIA NATURAL RESERVE (CENTRAL ITALY) , Journal of Agricultural Engineering: Vol. 42 No. 2 (2011)
- Cesar Augusto Terán-Chaves, Sonia Mercedes Polo-Murcia, Cropping pattern simulation-optimization model for water use efficiency and economic return , Journal of Agricultural Engineering: Vol. 52 No. 4 (2021)
- Bhola Paudel, Jayanta Kumar Basak, Seong Woo Jeon, Nibas Chandra Deb, Sijan Karki, Hyeon Tae Kim, Development and field testing of biodegradable seedling plug-tray cutting mechanism for automated vegetable transplanter , Journal of Agricultural Engineering: Vol. 55 No. 2 (2024)
<< < 1 2 3 4 5 6 7 8 9 10 > >>
You may also start an advanced similarity search for this article.
