https://doi.org/10.4081/jae.2023.1301
Cassava stalk detection for a cassava harvesting robot based on YOLO v4 and Mask R-CNN
Published: 1 August 2023
Abstract Views: 800
PDF: 416
HTML: 75
HTML: 75
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
The quality of fresh cassava roots can be increased through the use of precision equipment. As a first step towards developing an automatic cassava root cutting system, this study demonstrates the use of a computer vision system with deep learning for cassava stalk detection. An RGB image of a cassava tree mounted on a cassava-pulling machine was captured, and the YOLO v4 model and two Mask R-CNN models with ResNet 101 and ResNet 50 base architectures were employed to train the weights to predict the position of the cassava stalk. One hundred test images of stalks of various shapes and sizes were used to determine the grasping point and inclination, and the results from manual annotation were compared with the predicted results. Regarding localisation, Mask R-CNN with ResNet 101 gave a significantly higher performance than the other models, with an F1 score and a mean IoU of 0.81 and 0.70, respectively. YOLO v4 showed the highest correlation for the x- and y-coordinates for the prediction of the grasping point, with values for R2 of 0.89 and 0.53, respectively. For inclination prediction, Mask R-CNN with ResNet 101 and Mask R-CNN with ResNet 50 gave the same level of correlation, with values for R2 of 0.50 and 0.61, respectively. These results were acceptable for use as design criteria for developing a cassava rootcutting robot.
Abdulla W. 2016. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. GitHub Repository. Available from: https://github.com/matterport/Mask_RCNN
Arsawang S., Chansrakoo W., Chamsing A., Sangphanta P., Chawkongchak S. 2015. Design and development of Cassava root plucking out machine. Agric. Sci. J. 47:463-6.
Bapat K. 2018. Find the center of a blob (Centroid) using OpenCV (C++/Python). LearnOpenCV; July 19, 2018. Available from: https://www.learnopencv.com/find-center-of-blob-centroidusing-opencv-cpp-python/
Blok P.M., Barth R., van den Berg W. 2016. Machine vision for a selective broccoli harvesting robot. IFAC-PapersOnLine 49:66-71. Available from: https://doi.org/10.1016/j.ifacol.2016.10.013
DOI: https://doi.org/10.1016/j.ifacol.2016.10.013
Boonsang, S. 2020a. CiRA CORE : community. KMITL. Available from: https://www.facebook.com/groups/cira.core.comm/
Boonsang S. 2020b. KSL cira core. KLS. Available from: https://sites.google.com/site/klsrobotcenter/kls-cira-core
Brownlee J. 2021. A gentle introduction to object recognition with deep learning. Available from: https://machinelearningmastery.com/object-recognition-with-deep-learning/
Chansiri, C., & Wongpichet, S. 2011. Research and development of the digging and gathering machine for cassava harvesting. Khon Kaen University, Khon Kaen, Thailand.
Dutta, A., & Zisserman, A. 2019. The VIA annotation software for images, audio and video. MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia, 2276-2279. Available from: https://doi.org/10.1145/3343031.335053.
DOI: https://doi.org/10.1145/3343031.3350535
FAO. 2013. Cassava, a 21st century crop. In: Save and grow: cassava, a guide to sustainable production intensification. Available from: http://www.fao.org/ag/save-and-grow/cassava/en/1/index.html
Font D., Pallejà T., Tresanchez M., Runcan D., Moreno J., Martínez D., Teixidó M., Palacín J. 2014. A proposal for automatic fruit harvesting by combining a low cost stereovision camera and a robotic arm. Sensors (Switzerland) 14:11557-79.
DOI: https://doi.org/10.3390/s140711557
Fu L., Majeed Y., Zhang X., Karkee M., Zhang Q. 2020. Faster RCNN- based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosyst. Engine. 197:245-56.
DOI: https://doi.org/10.1016/j.biosystemseng.2020.07.007
Ganesh P., Volle K., Burks T.F., Mehta S.S. 2019. Deep orange: Mask R-CNN based orange detection and segmentation. IFAC-PapersOnLine 52:70-5.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.499
Ge Y., Xiong Y., From P.J. 2019. Instance segmentation and localization of strawberries in farm conditions for automatic fruit harvesting. IFAC-PapersOnLine 52:294-9.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.537
Girshick R. 2015. Fast R-CNN. pp 1440-1448 in Proceedings of the IEEE International Conference on Computer Vision 2015 Inter, 1. Available from: https://doi.org/10.1109/ICCV.2015.169
DOI: https://doi.org/10.1109/ICCV.2015.169
He K., Gkioxari G., Dollár P., Girshick R. 2020. Mask R-CNN. IEEE Trans. Pattern Anal. Machine Intell. 42:386-97.
DOI: https://doi.org/10.1109/TPAMI.2018.2844175
Jia W., Tian Y., Luo R., Zhang Z., Lian J., Zheng Y. 2020. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 172:105380.
DOI: https://doi.org/10.1016/j.compag.2020.105380
Jordan J. 2018. Evaluating image segmentation. Available from: https://www.jeremyjordan.me/evaluating-image-segmentation-models/
Koirala A., Walsh K.B., Wang Z., McCarthy C. 2019. Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO.’ Precis. Agric. 20:1107-35.
DOI: https://doi.org/10.1007/s11119-019-09642-0
KURDI. 2020. index @ www3.rdi.ku.ac.th. KU. Available from: http://www3.rdi.ku.ac.th/?p=58386
Langkapin J., Kalsirisilp R., Tantrabandit M. 2012. Design and fabrication of cassava root picking machine. Agric. Sci. J. 30:1-11.
Ling X., Zhao Y., Gong L., Liu C., Wang T. 2019. Dual-arm cooperation and implementing for robotic harvesting tomato using binocular vision. Robot. Auton. Syst. 114:134-43.
DOI: https://doi.org/10.1016/j.robot.2019.01.019
Majeed Y., Karkee M., Zhang Q., Fu L., Whiting M.D. 2019. A study on the detection of visible parts of cordons using deep learning networks for automated green shoot thinning in vineyards. IFAC-PapersOnLine 52:82-6.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.501
Manthamkan, V., Rattanasrimetha, S., & Suriwong, M. 2011. Development of cassava root lifted up by pulling stump harvester type. Karsetsart Extension Journal, 56(2), 52–60. Available from: https://kukrdb.lib.ku.ac.th/journal/ETO/search_detail/result/43687.
Mao S., Li Y., Ma Y., Zhang B., Zhou J., Kai W. 2020. Automatic cucumber recognition algorithm for harvesting robots in the natural environment using deep learning and multi-feature fusion. Comput. Electron. Agric. 170:105254.
DOI: https://doi.org/10.1016/j.compag.2020.105254
Maraphum K., Saengprachatanarug K., Wongpichet S., Phuphaphud A., Posom J. 2020. In-field measurement of starch content of cassava tubers using handheld vis-near infrared spectroscopy implemented for breeding programmes. Comput. Electron. Agric. 175:105607.
DOI: https://doi.org/10.1016/j.compag.2020.105607
Mauntumkarn W. 2010. Cassava harvesting machine. Available from: http://www.rdi.ku.ac.th/kasetresearch53/group06/wichar/index_04html
Mooney J.G., Johnson E.N. 2014. Performance evaluation of a harvesting robot for sweet pepper. J. Field Robot. 33:1-17.
Mordvintsev A., Abid K. 2013. opencv python. Available from: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_contours_begin/py_con
tours_begin.html
OAE. 2021. Agricultural economic information by commodity. Office Agric. Econ. Available from: http://www.oae.go.th/view/1/ปัจจัยการผลิต/TH-TH
Redmon J., Divvala S., Girshick R., Farhadi A. 2016. You only look once: unified, real-time object detection. pp. 779-788 in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Available from: https://doi.org/10.1109/CVPR.2016.91
DOI: https://doi.org/10.1109/CVPR.2016.91
Redmon J., Farhadi A. 2018. YOLO v.3. Tech Report, 1–6. Available from: https://pjreddie.com/media/files/papers/YOLOv3.pdf
Ren S., He K., Girshick R., Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Machine Intell. 39:1137-49.
DOI: https://doi.org/10.1109/TPAMI.2016.2577031
Sangphanta P., Chansrakoo W., Arsawang S., Chamsing A. 2011. Research and development of cassava harvesting machine. p. 12 in Proceedings of the 12th Thai Society of Agricultural Engineering Annual Academic Meeting.
Saxena S. 2021. Image augmentation techniques for training deep learning models. Analytics Vidhya. Available from: https://www.analyticsvidhya.com/blog/2021/03/image-augmentation-techniques-for-training-deep-learning-models/
Singhpoo T. 2019. Factors affecting cassava root cutting by using cylinder saw mechanism. Khon Kaen Univesity, Khon Kaen, Thailand.
Singhpoo T., Wongpichet S., Saengprachatanarug K., Posom J., Watyotha C., Yangyuen S. 2019. A study of stalk shape for designing the operational mechanism of gripping equipment for cassava tuber cut preparation process. IOP Conf. Ser. Earth Environ. Sci. 301:1-6.
DOI: https://doi.org/10.1088/1755-1315/301/1/012014
Suvanapa K., Wongpichet S. 2014. Feasibility study of cassava rhizome cutting by using square tube blade. pp. 335-342 in The 16th Thai Society of Agricultural Engineering Annual Academic Meeting.
Tian Y., Yang G., Wang Z., Wang H., Li E., Liang Z. 2019. Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput. Electron. Agricult. 157:417-26.
DOI: https://doi.org/10.1016/j.compag.2019.01.012
Vatakit K., Somphong C., Junyusen P., Arjharn W. 2014. Development of cassava harvester for cutting cassava tuber from rhizome. Agric. Sci. J. 45:353-6.
Williams H.A.M., Jones M.H., Nejati M., Seabright M.J., Bell J., Penhall N.D., Barnett J.J., Duke M.D., Scarfe A.J., Ahn H.S., Lim J.Y., MacDonald B.A. 2019. Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms. Biosyst. Engine. 181:140-56.
DOI: https://doi.org/10.1016/j.biosystemseng.2019.03.007
Yu Y., Zhang K., Yang L., Zhang D. 2019. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Comput. Electron. Agric. 163:104846.
DOI: https://doi.org/10.1016/j.compag.2019.06.001
Supporting Agencies
Khon Kaen UniversityHow to Cite
Singhpoo, T. (2023) “Cassava stalk detection for a cassava harvesting robot based on YOLO v4 and Mask R-CNN”, Journal of Agricultural Engineering, 54(2). doi: 10.4081/jae.2023.1301.
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Antonino Capra, Simona Consoli, Baldassare Scicolone, WATER MANAGEMENT STRATEGIES UNDER DEFICIT IRRIGATION , Journal of Agricultural Engineering: Vol. 39 No. 4 (2008)
- Francesca Piazzolla, Maria Luisa Amodio, Giancarlo Colelli, The use of hyperspectral imaging in the visible and near infrared region to discriminate between table grapes harvested at different times , Journal of Agricultural Engineering: Vol. 44 No. 2 (2013)
- Hui Yang, Yuhao Li, Chengguo Fu, Rongxian Zhang, Haibo Li, Yipeng Feng, Yaqi Zhang, Hongbin Cong, Fuquan Nie, Research on inspection route of hanging environmental robot based on computational fluid dynamics , Journal of Agricultural Engineering: Vol. 55 No. 2 (2024)
- Maria Luisa Amodio, Rosaria Cornacchia, Fedele Colantuono, Giancarlo Colelli, COLOR DEGRADATION KINETICS OF REHYDRATED ‘BORLOTTO’ BEANS STORED IN DIFFERENT GAS ATMOSPHERES AS MEASURED BY IMAGE ANALYSIS , Journal of Agricultural Engineering: Vol. 42 No. 4 (2011)
- Luhua Han, Haorui Ma, Menghan Mo, Francis Kumi, Jianping Hu, Hanping Mao, Design and test of an efficient seedling pick-up device with a combination of air jet ejection and mechanical action , Journal of Agricultural Engineering: Vol. 55 No. 3 (2024)
- Angelo Fabbri, Chiara Cevoli, Giuseppe Cantalupo, A method for handlebars ballast calculation in order to reduce vibrations transmissibility in walk behind tractors , Journal of Agricultural Engineering: Vol. 48 No. 2 (2017)
- Maria Elena Menconi, David Grohmann, Statistical assessment of vegetation dynamics within protected areas using remote sensing data , Journal of Agricultural Engineering: Vol. 44 No. 1 (2013)
- Silvio Kosutic, Daniele De Wrachien, Report on the 41st International Symposium Actual Tasks on Agricultural Engineering, 19-22 February 2013, Opatija, Croatia , Journal of Agricultural Engineering: Vol. 44 No. 1 (2013)
- Giuliano Vox, Pierfrancesco Losito, Fabio Valente, Rinaldo Consoletti, Giacomo Scarascia-Mugnozza, Evelia Schettini, Cristoforo Marzocca, Francesco Corsi, A wireless telecommunications network for real-time monitoring of greenhouse microclimate , Journal of Agricultural Engineering: Vol. 45 No. 2 (2014)
- Alessio Cappelli, Lucrezia Lupori, Enrico Cini, Should extra virgin olive oil production change the approach? A systematic review of challenges and opportunities to increase sustainability, productivity, and product quality , Journal of Agricultural Engineering: Vol. 54 No. 1 (2023)
<< < 19 20 21 22 23 24 25 26 27 28 > >>
You may also start an advanced similarity search for this article.
