Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review

Arad, 2020, Development of a sweet pepper harvesting robot, J. F. Robot., 1–13

Arefi, 2013, Development of an expert system based on wavelet transform and artificial neural networks for the ripe tomato harvesting robot, Aust. J. Crop Sci., 7, 699

Bargoti, 2017, Image segmentation for fruit detection and yield estimation in apple orchards, J. F. Robot., 34, 1039, 10.1002/rob.21699

Barnea, 2016, Colour-agnostic shape-based 3D fruit detection for crop harvesting robots, Biosyst. Eng., 146, 57, 10.1016/j.biosystemseng.2016.01.013

Boyer, 1987, Color-encoded structured light for rapid active ranging, IEEE Trans. Pattern Anal. Mach. Intell., PAMI-9, 14, 10.1109/TPAMI.1987.4767869

Chen, 1997, Range data acquisition using color structured lighting and stereo vision, Image Vis. Comput., 15, 445, 10.1016/S0262-8856(96)01148-1

Choi, 2015, A machine vision system for quantification of citrus fruit dropped on the ground under the canopy, Trans. ASABE, 58, 933

Dahikar, 2014, Agricultural crop yield prediction using artificial neural network approach, Int. J. Innov. Res. Electr. Electron. Instrum. Control Eng., 2, 683

Dionisio, 2016, Using depth cameras to extract structural parameters to assess the growth state and yield of cauliflower crops, Comput. Electron. Agric., 122, 67, 10.1016/j.compag.2016.01.018

Fan, 2018, Estimating tree position, diameter at breast height, and tree height in real-time using a mobile phone with RGB-D SLAM, Remote Sens., 10, 1845, 10.3390/rs10111845

Fernández, 2014, Multisensory system for fruit harvesting robots. Experimental testing in natural scenarios and with different kinds of crops, Sensors, 14, 23885, 10.3390/s141223885

Font, 2014, A proposal for automatic fruit harvesting by combining a low cost stereovision camera and a robotic arm, Sensors, 14, 11557, 10.3390/s140711557

Fu, 2018, Kiwifruit detection in field images using Faster R-CNN with ZFNet, IFAC-PapersOnLine, 51, 45, 10.1016/j.ifacol.2018.08.059

Fu, 2020, Faster R-CNN-based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting, Biosyst. Eng., 197, 245, 10.1016/j.biosystemseng.2020.07.007

Fu, 2019, A novel image processing algorithm to separate linearly clustered kiwifruits, Biosyst. Eng., 183, 184, 10.1016/j.biosystemseng.2019.04.024

Gao, 2020, Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN, Comput. Electron. Agric., 176, 10.1016/j.compag.2020.105634

García-Luna, 2016, Towards an artificial vision-robotic system for tomato identification, IFAC-PapersOnLine, 49, 365, 10.1016/j.ifacol.2016.10.067

Ge, 2019, Fruit localization and environment perception for strawberry harvesting robots, IEEE Access, 7, 147642, 10.1109/ACCESS.2019.2946369

Gené-Mola, 2020, Fruit detection, yield prediction and canopy geometric characterization using LiDAR with forced air flow, Comput. Electron. Agric., 168, 10.1016/j.compag.2019.105121

Gené-Mola, 2019, Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities, Comput. Electron. Agric., 162, 689, 10.1016/j.compag.2019.05.016

Georg, 2019, An empirical evaluation of ten depth cameras: Bias, precision, lateral noise, different lighting conditions and materials, and multiple sensor setups in indoor environments, IEEE Robot. Autom. Mag., 26, 67, 10.1109/MRA.2018.2852795

Gongal, 2015, Sensors and systems for fruit detection and localization: A review, Comput. Electron. Agric., 116, 8, 10.1016/j.compag.2015.05.021

Gongal, 2018, Apple fruit size estimation using a 3D machine vision system, Inf. Process. Agric., 5, 498

Gongal, 2016, Apple crop-load estimation with over-the-row machine vision system, Comput. Electron. Agric., 120, 26, 10.1016/j.compag.2015.10.022

Guo, 2017, LSSA_CAU: An interactive 3d point clouds analysis software for body measurement of livestock with similar forms of cows or pigs, Comput. Electron. Agric., 138, 60, 10.1016/j.compag.2017.04.014

Haemmerle, 2018, Mobile low-cost 3D camera maize crop height measurements under field conditions, Precis. Agric., 19, 630, 10.1007/s11119-017-9544-3

Häni, 2020, A comparative study of fruit detection and counting methods for yield mapping in apple orchards, J. F. Robot., 37, 263, 10.1002/rob.21902

Harrell, 1990, Robotic picking of citrus, Robotica, 8, 269, 10.1017/S0263574700000308

Kang, 2020, Fast implementation of real-time fruit detection in apple orchards using deep learning, Comput. Electron. Agric., 168, 10.1016/j.compag.2019.105108

Kang, 2020, Fruit detection, segmentation and 3D visualisation of environments in apple orchards, Comput. Electron. Agric., 171, 10.1016/j.compag.2020.105302

Koirala, 2019, Deep learning - Method overview and review of use for fruit detection and yield estimation, Comput. Electron. Agric., 162, 219, 10.1016/j.compag.2019.04.017

Kongsro, 2014, Estimation of pig weight using a Microsoft Kinect prototype imaging system, Comput. Electron. Agric., 109, 32, 10.1016/j.compag.2014.08.008

Kuan, 2019, Comparative study of intel R200, Kinect v2, and primesense RGB-D sensors performance outdoors, IEEE Sens. J., 19, 8741, 10.1109/JSEN.2019.2920976

Kurtulmus, 2011, Green citrus detection using “eigenfruit”, color and circular Gabor texture features under natural outdoor conditions, Comput. Electron. Agric., 78, 140, 10.1016/j.compag.2011.07.001

Kusumam, 2017, 3D-vision based detection, localization, and sizing of broccoli heads in the field, J. F. Robot., 34, 1505, 10.1002/rob.21726

Lao, 2019, 3D reconstruction of maize plants based on consumer depth camera, Trans. Chinese Soc. Agric. Mach., 50, 222

Lehnert, 2017, Autonomous sweet pepper harvesting for protected cropping systems, IEEE Robot. Autom. Lett., 2, 872, 10.1109/LRA.2017.2655622

Li, 2015, Machine vision technology for detecting the external defects of fruits - A review, Imaging Sci. J., 63, 241, 10.1179/1743131X14Y.0000000088

Li, 2018, Crop recognition under weedy conditions based on 3D imaging for robotic weed control, J. F. Robot., 35, 596, 10.1002/rob.21763

Li, 2019, Maize plant 3D information acquisition system based on mobile robot platform, Trans. Chinese Soc. Agric. Mach., 50, 15

Lin, 2019, In-field citrus detection and localisation based on RGB-D image analysis, Biosyst. Eng., 186, 34, 10.1016/j.biosystemseng.2019.06.019

Lin, 2020, Color-, depth-, and shape-based 3D fruit detection, Precis. Agric., 21, 1, 10.1007/s11119-019-09654-w

Liu, 2018, Experiments and analysis of close-shot identification of on-branch citrus fruit with realsense, Sensors, 18, 1510, 10.3390/s18051510

Liu, 2017, Depth-sphere transversal method for on-branch citrus fruit recognition, Trans. Chinese Soc. Agric. Mach., 48, 32

Liu, 2020, Improved kiwifruit detection using pre-trained VGG16 with RGB and NIR information fusion, IEEE Access, 8, 2327, 10.1109/ACCESS.2019.2962513

Lv, 2019, A method to obtain the near-large fruit from apple image in orchard for single-arm apple harvesting robot, Sci. Hortic., 257, 10.1016/j.scienta.2019.108758

Ma, 2019, Calculation method for maize plant height based on depth information, Int. Agric. Eng. J., 28, 325

Mai, 2015, Research on 3D reconstruction of fruit tree and fruit recognition and location method based on RGB-D camera, Trans. Chinese Soc. Agric. Mach., 46, 35

Mehta, 2014, Vision-based control of robotic manipulator for citrus harvesting, Comput. Electron. Agric., 102, 146, 10.1016/j.compag.2014.01.003

Méndez Perez, 2017, Flexible system of multiple RGB-D sensors for measuring and classifying fruits in agri-food Industry, Comput. Electron. Agric., 139, 231, 10.1016/j.compag.2017.05.014

Milella, 2019, In-field high throughput grapevine phenotyping with a consumer-grade depth camera, Comput. Electron. Agric., 156, 293, 10.1016/j.compag.2018.11.026

Misimi, 2016, GRIBBOT - Robotic 3D vision-guided harvesting of chicken fillets, Comput. Electron. Agric., 121, 84, 10.1016/j.compag.2015.11.021

Munaro, 2016, OpenPTrack: Open source multi-camera calibration and people tracking for RGB-D camera networks, Rob. Auton. Syst., 75, 525, 10.1016/j.robot.2015.10.004

Nguyen, 2015, Structured light-based 3D reconstruction system for plants, Sensors, 15, 18587, 10.3390/s150818587

Nguyen, 2016, Detection of red and bicoloured apples on tree with an RGB-D camera, Biosyst. Eng., 146, 33, 10.1016/j.biosystemseng.2016.01.007

Nir, 2018, 3D Computer-vision system for automatically estimating heifer height and body mass, Biosyst. Eng., 173, 4, 10.1016/j.biosystemseng.2017.11.014

Pamornnak, 2017, An automatic and rapid system for grading palm bunch using a Kinect camera, Comput. Electron. Agric., 143, 227, 10.1016/j.compag.2017.10.020

Paulus, 2014, Low-cost 3D systems: Suitable tools for plant phenotyping, Sensors, 14, 3001, 10.3390/s140203001

Ramos, 2018, Measurement of the ripening rate on coffee branches by using 3D images in outdoor environments, Comput. Ind., 99, 83, 10.1016/j.compind.2018.03.024

Rosell-Polo, 2017, Kinect v2 sensor-based mobile terrestrial laser scanner for agricultural outdoor applications, IEEE/ASME Trans. Mechatronics, 22, 2420, 10.1109/TMECH.2017.2663436

Sa, 2016, Deepfruits: A fruit detection system using deep neural networks, Sensors, 16, 1222, 10.3390/s16081222

Sa, 2017, Peduncle detection of sweet pepper for autonomous crop harvesting-combined color and 3-D information, IEEE Robot. Autom. Lett., 2, 765, 10.1109/LRA.2017.2651952

Salau, 2017, Automated calculation of udder depth and rear leg angle in Holstein-Friesian cows using a multi-Kinect cow scanning system, Biosyst. Eng., 160, 154, 10.1016/j.biosystemseng.2017.06.006

Silwal, 2016, A hierarchical approach to apple identification for robotic harvesting, Trans. ASABE, 59, 1079, 10.13031/trans.59.11619

Slaughter, 1989, Discriminating fruit for robotic harvest using color in natural outdoor scenes, Trans. Am. Soc. Agric. Eng., 32, 757, 10.13031/2013.31066

Song, 2016, Segmentation and localization method of greenhouse cucumber based on image fusion technology, Int. J. Simul. Syst. Sci. Technol., 17, 11

Stajnko, 2004, Estimation of number and diameter of apple fruits in an orchard during the growing season by thermal imaging, Comput. Electron. Agric., 42, 31, 10.1016/S0168-1699(03)00086-3

Sun, 2019, Image processing algorithms for infield single cotton boll counting and yield prediction, Comput. Electron. Agric., 166, 10.1016/j.compag.2019.104976

Tang, 2020, Recognition and localization methods for vision-based fruit picking robots : A review, Front. Plant Sci., 11, 510, 10.3389/fpls.2020.00510

Tao, 2017, Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking, Comput. Electron. Agric., 142, 388, 10.1016/j.compag.2017.09.019

Tian, 2019, Fast recognition and location of target fruit based on depth information, IEEE Access, 7, 170553, 10.1109/ACCESS.2019.2955566

Tu, 2020, Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images, Precis. Agric., 10.1007/s11119-020-09709-3

Tu, 2018, Detection of passion fruits and maturity classification using Red-Green-Blue Depth images, Biosyst. Eng., 175, 156, 10.1016/j.biosystemseng.2018.09.004

Vázquez-Arellano, 2018, 3-D reconstruction of maize plants using a time-of-flight camera, Comput. Electron. Agric., 145, 235, 10.1016/j.compag.2018.01.002

Vit, 2018, Comparing RGB-D sensors for close range outdoor agricultural phenotyping, Sensors, 18, 4413, 10.3390/s18124413

Wachs, 2010, Low and high-level visual feature-based apple detection from multi-modal images, Precis. Agric., 11, 717, 10.1007/s11119-010-9198-x

Wang, 2016, The acquisition of kiwifruit feature point coordinates based on the spatial coordinates of image, IFIP Adv. Inf. Commun. Technol., 478, 399, 10.1007/978-3-319-48357-3_39

Wang, 2020, Deep Learning Approach for Apple Edge Detection to Remotely Monitor Apple Growth in Orchards, IEEE Access, 8, 26911, 10.1109/ACCESS.2020.2971524

Wang, 2012, Identification and location system of multi-operation apple robot based on vision combination, Trans. Chinese Soc. Agric. Mach., 43, 165

Wang, 2014, Size estimation of sweet onions using consumer-grade RGB-depth sensor, J. Food Eng., 142, 153, 10.1016/j.jfoodeng.2014.06.019

Wang, 2017, On-tree mango fruit size estimation using RGB-D images, Sensors, 17, 2738, 10.3390/s17122738

Wu, 2020, Using color and 3D geometry features to segment fruit point cloud and improve fruit recognition accuracy, Comput. Electron. Agric., 174, 10.1016/j.compag.2020.105475

Xiao, 2017, An intelligent precision orchard pesticide spray technique based on the depth-of-field extraction algorithm, Comput. Electron. Agric., 133, 30, 10.1016/j.compag.2016.12.002

Xiong, 2019, Development and field evaluation of a strawberry harvesting robot with a cable-driven gripper, Comput. Electron. Agric., 157, 392, 10.1016/j.compag.2019.01.009

Yamamoto, 2015, Growth measurement of a community of strawberries using three-dimensional sensor, Environ. Control Biol., 53, 49, 10.2525/ecb.53.49

Yang, 2019, Research and experiment on recognition and location system for citrus picking robot in natural environment, Trans. Chinese Soc. Agric. Mach., 50, 72

Yin, 2019, Three dimensional point cloud reconstruction and body size measurement of pigs based on multi-view depth camera, Trans. Chinese Soc. Agric. Eng., 35, 201

Zanuttigh, 2016, Operating principles of time-of-flight depth cameras, Time-of-Flight and Structured Light Depth Cameras: Technol. App., 81

Zhang, 2018, Branch detection for apple trees trained in fruiting wall architecture using depth features and Regions-Convolutional Neural Network (R-CNN), Comput. Electron. Agric., 155, 386, 10.1016/j.compag.2018.10.029

Zhang, 2020, Multi-class object detection using faster R-CNN and estimation of shaking locations for automated shake-and-catch apple harvesting, Comput. Electron. Agric., 173, 10.1016/j.compag.2020.105384

Zhang, 2019, Segmentation of apple point clouds based on ROI in RGB images, Inmateh - Agric. Eng., 59, 209, 10.35633/inmateh-59-23

Zhang, 2020, Wheat lodging detection from UAS imagery using machine learning algorithms, Remote Sens., 12, 1838, 10.3390/rs12111838

Zhang, 2016, The development of mechanical apple harvesting technology: A review, Trans. ASABE, 59, 1165, 10.13031/trans.59.11737

Zhang, 2020, Technology progress in mechanical harvest of fresh market apples, Comput. Electron. Agric., 175, 10.1016/j.compag.2020.105606

Zhang, 2018, A review of bin filling technologies for apple harvest and postharvest handling, Appl. Eng. Agric., 34, 687, 10.13031/aea.12827

Zhao, 2016, A review of key techniques of vision-based control for harvesting robot, Comput. Electron. Agric., 127, 311, 10.1016/j.compag.2016.06.022