https://doi.org/10.4081/jae.2023.1301
Cassava stalk detection for a cassava harvesting robot based on YOLO v4 and Mask R-CNN
Published: 1 August 2023
Abstract Views: 800
PDF: 416
HTML: 75
HTML: 75
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
The quality of fresh cassava roots can be increased through the use of precision equipment. As a first step towards developing an automatic cassava root cutting system, this study demonstrates the use of a computer vision system with deep learning for cassava stalk detection. An RGB image of a cassava tree mounted on a cassava-pulling machine was captured, and the YOLO v4 model and two Mask R-CNN models with ResNet 101 and ResNet 50 base architectures were employed to train the weights to predict the position of the cassava stalk. One hundred test images of stalks of various shapes and sizes were used to determine the grasping point and inclination, and the results from manual annotation were compared with the predicted results. Regarding localisation, Mask R-CNN with ResNet 101 gave a significantly higher performance than the other models, with an F1 score and a mean IoU of 0.81 and 0.70, respectively. YOLO v4 showed the highest correlation for the x- and y-coordinates for the prediction of the grasping point, with values for R2 of 0.89 and 0.53, respectively. For inclination prediction, Mask R-CNN with ResNet 101 and Mask R-CNN with ResNet 50 gave the same level of correlation, with values for R2 of 0.50 and 0.61, respectively. These results were acceptable for use as design criteria for developing a cassava rootcutting robot.
Abdulla W. 2016. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. GitHub Repository. Available from: https://github.com/matterport/Mask_RCNN
Arsawang S., Chansrakoo W., Chamsing A., Sangphanta P., Chawkongchak S. 2015. Design and development of Cassava root plucking out machine. Agric. Sci. J. 47:463-6.
Bapat K. 2018. Find the center of a blob (Centroid) using OpenCV (C++/Python). LearnOpenCV; July 19, 2018. Available from: https://www.learnopencv.com/find-center-of-blob-centroidusing-opencv-cpp-python/
Blok P.M., Barth R., van den Berg W. 2016. Machine vision for a selective broccoli harvesting robot. IFAC-PapersOnLine 49:66-71. Available from: https://doi.org/10.1016/j.ifacol.2016.10.013
DOI: https://doi.org/10.1016/j.ifacol.2016.10.013
Boonsang, S. 2020a. CiRA CORE : community. KMITL. Available from: https://www.facebook.com/groups/cira.core.comm/
Boonsang S. 2020b. KSL cira core. KLS. Available from: https://sites.google.com/site/klsrobotcenter/kls-cira-core
Brownlee J. 2021. A gentle introduction to object recognition with deep learning. Available from: https://machinelearningmastery.com/object-recognition-with-deep-learning/
Chansiri, C., & Wongpichet, S. 2011. Research and development of the digging and gathering machine for cassava harvesting. Khon Kaen University, Khon Kaen, Thailand.
Dutta, A., & Zisserman, A. 2019. The VIA annotation software for images, audio and video. MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia, 2276-2279. Available from: https://doi.org/10.1145/3343031.335053.
DOI: https://doi.org/10.1145/3343031.3350535
FAO. 2013. Cassava, a 21st century crop. In: Save and grow: cassava, a guide to sustainable production intensification. Available from: http://www.fao.org/ag/save-and-grow/cassava/en/1/index.html
Font D., Pallejà T., Tresanchez M., Runcan D., Moreno J., Martínez D., Teixidó M., Palacín J. 2014. A proposal for automatic fruit harvesting by combining a low cost stereovision camera and a robotic arm. Sensors (Switzerland) 14:11557-79.
DOI: https://doi.org/10.3390/s140711557
Fu L., Majeed Y., Zhang X., Karkee M., Zhang Q. 2020. Faster RCNN- based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosyst. Engine. 197:245-56.
DOI: https://doi.org/10.1016/j.biosystemseng.2020.07.007
Ganesh P., Volle K., Burks T.F., Mehta S.S. 2019. Deep orange: Mask R-CNN based orange detection and segmentation. IFAC-PapersOnLine 52:70-5.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.499
Ge Y., Xiong Y., From P.J. 2019. Instance segmentation and localization of strawberries in farm conditions for automatic fruit harvesting. IFAC-PapersOnLine 52:294-9.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.537
Girshick R. 2015. Fast R-CNN. pp 1440-1448 in Proceedings of the IEEE International Conference on Computer Vision 2015 Inter, 1. Available from: https://doi.org/10.1109/ICCV.2015.169
DOI: https://doi.org/10.1109/ICCV.2015.169
He K., Gkioxari G., Dollár P., Girshick R. 2020. Mask R-CNN. IEEE Trans. Pattern Anal. Machine Intell. 42:386-97.
DOI: https://doi.org/10.1109/TPAMI.2018.2844175
Jia W., Tian Y., Luo R., Zhang Z., Lian J., Zheng Y. 2020. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 172:105380.
DOI: https://doi.org/10.1016/j.compag.2020.105380
Jordan J. 2018. Evaluating image segmentation. Available from: https://www.jeremyjordan.me/evaluating-image-segmentation-models/
Koirala A., Walsh K.B., Wang Z., McCarthy C. 2019. Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO.’ Precis. Agric. 20:1107-35.
DOI: https://doi.org/10.1007/s11119-019-09642-0
KURDI. 2020. index @ www3.rdi.ku.ac.th. KU. Available from: http://www3.rdi.ku.ac.th/?p=58386
Langkapin J., Kalsirisilp R., Tantrabandit M. 2012. Design and fabrication of cassava root picking machine. Agric. Sci. J. 30:1-11.
Ling X., Zhao Y., Gong L., Liu C., Wang T. 2019. Dual-arm cooperation and implementing for robotic harvesting tomato using binocular vision. Robot. Auton. Syst. 114:134-43.
DOI: https://doi.org/10.1016/j.robot.2019.01.019
Majeed Y., Karkee M., Zhang Q., Fu L., Whiting M.D. 2019. A study on the detection of visible parts of cordons using deep learning networks for automated green shoot thinning in vineyards. IFAC-PapersOnLine 52:82-6.
DOI: https://doi.org/10.1016/j.ifacol.2019.12.501
Manthamkan, V., Rattanasrimetha, S., & Suriwong, M. 2011. Development of cassava root lifted up by pulling stump harvester type. Karsetsart Extension Journal, 56(2), 52–60. Available from: https://kukrdb.lib.ku.ac.th/journal/ETO/search_detail/result/43687.
Mao S., Li Y., Ma Y., Zhang B., Zhou J., Kai W. 2020. Automatic cucumber recognition algorithm for harvesting robots in the natural environment using deep learning and multi-feature fusion. Comput. Electron. Agric. 170:105254.
DOI: https://doi.org/10.1016/j.compag.2020.105254
Maraphum K., Saengprachatanarug K., Wongpichet S., Phuphaphud A., Posom J. 2020. In-field measurement of starch content of cassava tubers using handheld vis-near infrared spectroscopy implemented for breeding programmes. Comput. Electron. Agric. 175:105607.
DOI: https://doi.org/10.1016/j.compag.2020.105607
Mauntumkarn W. 2010. Cassava harvesting machine. Available from: http://www.rdi.ku.ac.th/kasetresearch53/group06/wichar/index_04html
Mooney J.G., Johnson E.N. 2014. Performance evaluation of a harvesting robot for sweet pepper. J. Field Robot. 33:1-17.
Mordvintsev A., Abid K. 2013. opencv python. Available from: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_contours_begin/py_con
tours_begin.html
OAE. 2021. Agricultural economic information by commodity. Office Agric. Econ. Available from: http://www.oae.go.th/view/1/ปัจจัยการผลิต/TH-TH
Redmon J., Divvala S., Girshick R., Farhadi A. 2016. You only look once: unified, real-time object detection. pp. 779-788 in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Available from: https://doi.org/10.1109/CVPR.2016.91
DOI: https://doi.org/10.1109/CVPR.2016.91
Redmon J., Farhadi A. 2018. YOLO v.3. Tech Report, 1–6. Available from: https://pjreddie.com/media/files/papers/YOLOv3.pdf
Ren S., He K., Girshick R., Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Machine Intell. 39:1137-49.
DOI: https://doi.org/10.1109/TPAMI.2016.2577031
Sangphanta P., Chansrakoo W., Arsawang S., Chamsing A. 2011. Research and development of cassava harvesting machine. p. 12 in Proceedings of the 12th Thai Society of Agricultural Engineering Annual Academic Meeting.
Saxena S. 2021. Image augmentation techniques for training deep learning models. Analytics Vidhya. Available from: https://www.analyticsvidhya.com/blog/2021/03/image-augmentation-techniques-for-training-deep-learning-models/
Singhpoo T. 2019. Factors affecting cassava root cutting by using cylinder saw mechanism. Khon Kaen Univesity, Khon Kaen, Thailand.
Singhpoo T., Wongpichet S., Saengprachatanarug K., Posom J., Watyotha C., Yangyuen S. 2019. A study of stalk shape for designing the operational mechanism of gripping equipment for cassava tuber cut preparation process. IOP Conf. Ser. Earth Environ. Sci. 301:1-6.
DOI: https://doi.org/10.1088/1755-1315/301/1/012014
Suvanapa K., Wongpichet S. 2014. Feasibility study of cassava rhizome cutting by using square tube blade. pp. 335-342 in The 16th Thai Society of Agricultural Engineering Annual Academic Meeting.
Tian Y., Yang G., Wang Z., Wang H., Li E., Liang Z. 2019. Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput. Electron. Agricult. 157:417-26.
DOI: https://doi.org/10.1016/j.compag.2019.01.012
Vatakit K., Somphong C., Junyusen P., Arjharn W. 2014. Development of cassava harvester for cutting cassava tuber from rhizome. Agric. Sci. J. 45:353-6.
Williams H.A.M., Jones M.H., Nejati M., Seabright M.J., Bell J., Penhall N.D., Barnett J.J., Duke M.D., Scarfe A.J., Ahn H.S., Lim J.Y., MacDonald B.A. 2019. Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms. Biosyst. Engine. 181:140-56.
DOI: https://doi.org/10.1016/j.biosystemseng.2019.03.007
Yu Y., Zhang K., Yang L., Zhang D. 2019. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Comput. Electron. Agric. 163:104846.
DOI: https://doi.org/10.1016/j.compag.2019.06.001
Supporting Agencies
Khon Kaen UniversityHow to Cite
Singhpoo, T. (2023) “Cassava stalk detection for a cassava harvesting robot based on YOLO v4 and Mask R-CNN”, Journal of Agricultural Engineering, 54(2). doi: 10.4081/jae.2023.1301.
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Fabrizio Mazzetto, Aldo Calcante, DEVELOPMENTAND FIRST TESTS OF AN AUTOMATIC SYSTEM FOR COMMERCIAL VINE CUTTING TRANSPLANTERS BASED ON DGPS-RTK TECHNOLOGY , Journal of Agricultural Engineering: Vol. 40 No. 2 (2009)
- Marco Vizzari, Maurizia Sigura, Urban-rural gradient detection using multivariate spatial analysis and landscape metrics , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- R. Gubiani, G. Pergher, S.R.S. Cividino, R. Lombardo, F. Blanchini, A Wii-controlled safety device for electric chainsaws , Journal of Agricultural Engineering: Vol. 44 No. s2 (2013): Proceedings of the 10th Conference of the Italian Society of Agricultural Engineering
- Sergio Castellano, Photovoltaic greenhouses: evaluation of shading effect and its influence on agricultural performances , Journal of Agricultural Engineering: Vol. 45 No. 4 (2014)
- Eutiquio Gallego, Manuel Moya, Esperanza Ayuga-Téllez, Ana I. García, Francisco Ayuga, The influence of management and construction methods in the repair costs of Spain’s low-volume road network , Journal of Agricultural Engineering: Vol. 47 No. 2 (2016)
- Fabrizio Mazzetto, Aldo Calcante, Aira Mena, Pasqualina Sacco, TEST OF GROUND-SENSING DEVICES FOR MONITORING CANOPY VIGOUR AND DOWNY MILDEW PRESENCE IN VINEYARDS: FIRST QUALITATIVE RESULTS , Journal of Agricultural Engineering: Vol. 42 No. 2 (2011)
- Giuseppe Taglioli, Angelo Fabbri, PROVE IDRAULICHE SU UN SEMOVENTE IRRIGUO , Journal of Agricultural Engineering: Vol. 38 No. 3 (2007)
- Elio Dinuccio, Fabrizio Gioelli, Dalibor Cuk, Luca Rollè, Paolo Balsari, The use of co-digested solid fraction as feedstock for biogas plants , Journal of Agricultural Engineering: Vol. 44 No. 4 (2013)
- Irene Diti, Daniele Torreggiani, Patrizia Tassinari, Rural landscape and cultural routes: a multicriteria spatial classification method tested on an Italian case study , Journal of Agricultural Engineering: Vol. 46 No. 1 (2015)
- Igor Kovacev, Daniele De Wrachien, Report on the 45th International Symposium: Actual Tasks on Agricultural Engineering, 21st-24th February 2017, Opatija, Croatia , Journal of Agricultural Engineering: Vol. 48 No. 2 (2017)
<< < 26 27 28 29 30 31 32 33 34 35 > >>
You may also start an advanced similarity search for this article.
