A novel application of deep learning with image cropping: a smart city use case for flood monitoring
Tóm tắt
Event monitoring is an essential application of Smart City platforms. Real-time monitoring of gully and drainage blockage is an important part of flood monitoring applications. Building viable IoT sensors for detecting blockage is a complex task due to the limitations of deploying such sensors in situ. Image classification with deep learning is a potential alternative solution. However, there are no image datasets of gullies and drainages. We were faced with such challenges as part of developing a flood monitoring application in a European Union-funded project. To address these issues, we propose a novel image classification approach based on deep learning with an IoT-enabled camera to monitor gullies and drainages. This approach utilises deep learning to develop an effective image classification model to classify blockage images into different class labels based on the severity. In order to handle the complexity of video-based images, and subsequent poor classification accuracy of the model, we have carried out experiments with the removal of image edges by applying image cropping. The process of cropping in our proposed experimentation is aimed to concentrate only on the regions of interest within images, hence leaving out some proportion of image edges. An image dataset from crowd-sourced publicly accessible images has been curated to train and test the proposed model. For validation, model accuracies were compared considering model with and without image cropping. The cropping-based image classification showed improvement in the classification accuracy. This paper outlines the lessons from our experimentation that have a wider impact on many similar use cases involving IoT-based cameras as part of smart city event monitoring platforms.
Tài liệu tham khảo
Chen Y, Lin Z, Zhao X, Wang G, Gu Y (2014) Deep learning-based classification of hyperspectral data. IEEE J Select Top Appl Earth Obs Remote Sens 7(6):2094–2107
Lee H, Kwon H (2017) Going deeper with contextual CNN for hyperspectral image classification. IEEE Trans Image Process 26(10):4843–4855
Liu P, Zhang H, Eom KB (2017) Active deep learning for classification of hyperspectral images. IEEE J Select Top Appl Earth Obs Remote Sens 10(2):712–724
Hu W, Huang Y, Wei L, Zhang F, Li H (2015) Deep convolutional neural networks for hyperspectral image classification. J Sens 2015:12 (Art. no. 258619)
Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans Geosci Remote Sens 54(10):6232–6251
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504
Deng J, Dong W, Socher R, Li L, Kai L, Li F-F (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255
Krizhevsky ISA, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Proceedings of the advances in neural information processing systems 25 (NIPS ‘12), pp 1097–1105
Foody GM, Mathur A (2004) A relative evaluation of multiclass image classification by support vector machines. IEEE Trans Geosci Remote Sens 42(6):1335–1343
Su Y, Jiu-Fen Z, Jiu-Ling Z, Qing-Zhen L (2008) A method for image classification based on kernel PCA. In: 2008 international conference on machine learning and cybernetics, vol 2, pp 718–722
Ding C, He X (2004) K-means clustering via principal component analysis. In: Presented at the proceedings of the twenty-first international conference on Machine learning, Banff, Alberta
Wang X, Kong Y, Gao Y, Cheng Y (2017) Dimensionality reduction for hyperspectral data based on pairwise constraint discriminative analysis and nonnegative sparse divergence. IEEE J Sel Top Appl Earth Obs Remote Sens 10(4):1552–1562
Guillamet D, Vitrià J, Schiele B (2003) Introducing a weighted non-negative matrix factorization for image classification. Pattern Recognit Lett 24(14):2447–2454
Bruce LM, Koger CH, Jiang L (2002) Dimensionality reduction of hyperspectral data using discrete wavelet transform feature extraction. IEEE Trans Geosci Remote Sens 40(10):2331–2338
Samadzadegan F, Hasani H, Schenk T (2012) Simultaneous feature selection and SVM parameter determination in classification of hyperspectral imagery using ant colony optimization. Can J Remote Sens 38(2):139–156
Zhao W, Du S (2016) Spectral–spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Trans Geosci Remote Sens 54(8):4544–4554
Yang C-C et al (2003) Application of decision tree technology for image classification using remote sensing data. Agric Syst 76(3):1101–1117
Bosch A, Zisserman A, Munoz X (2007) Image classification using random forests and ferns. In: 2007 IEEE 11th international conference on computer vision, pp 1–8
Du P, Samat A, Waske B, Liu S, Li Z (2015) Random Forest and Rotation Forest for fully polarized SAR image classification using polarimetric and spatial features. ISPRS J Photogramm Remote Sens 105:38–53
Huang K, Li S, Kang X, Fang L (2015) Spectral–spatial hyperspectral image classification based on KNN. Sens Imaging 17(1):1
Amato G, Falchi F (2010) kNN based image classification relying on local feature similarity. In: Presented at the proceedings of the third international conference on SImilarity Search and APplications, Istanbul
Tzotsos A, Argialas D (2008) Support vector machine classification for object-based image analysis. In: Blaschke T, Lang S, Hay GJ (eds) Object-based image analysis: spatial concepts for knowledge-driven remote sensing applications. Springer, Berlin, pp 663–677
Maulik U, Chakraborty D (2017) Remote sensing image classification: a survey of support-vector-machine-based advanced techniques. IEEE Geosci Remote Sens Mag 5(1):33–52
Millard K, Richardson M (2015) On the importance of training data sample selection in random forest image classification: a case study in Peatland ecosystem mapping. Remote Sens 7(7):8489–8515
Foody GM, Mathur A (2006) The use of small training sets containing mixed pixels for accurate hard image classification: training on mixed spectral responses for classification by a SVM. Remote Sens Environ 103(2):179–189
Blaschke T (2010) Object based image analysis for remote sensing. ISPRS J Photogramm Remote Sens 65(1):2–16
Yan G, Mas JF, Maathuis BHP, Xiangmin Z, Van Dijk PM (2006) Comparison of pixel-based and object-oriented image classification approaches—a case study in a coal fire area, Wuda, Inner Mongolia, China. Int J Remote Sens 27(18):4039–4055
Duro DC, Franklin SE, Dubé MG (2012) A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery. Remote Sens Environ 118:259–272
Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771
Cheng G, Han J, Zhou P, Guo L (2014) Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS J Photogramm Remote Sens 98:119–132
Blaschke T (2003) Object-based contextual image classification built on image segmentation. In: IEEE workshop on advances in techniques for analysis of remotely sensed data, pp 113–119
Li L-J, Su H, Lim Y, Fei-Fei L (2012) Objects as attributes for scene classification. Springer, Berlin, pp 57–69
Zhong P, Gong Z, Li S, Schönlieb C (2017) Learning to diversify deep belief networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55(6):3516–3530
Giacinto G, Roli F (2001) Design of effective neural network ensembles for image classification purposes. Image Vis Comput 19(9):699–707
Chen Y, Zhao X, Jia X (2015) Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE J Sel Top Appl Earth Obs Remote Sens 8(6):2381–2392
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
Li T, Zhang J, Zhang Y (2014) Classification of hyperspectral image based on deep belief networks. In: 2014 IEEE international conference on image processing (ICIP), pp 5132–5136
Mortensen TM (2014) Blurry and centered or clear and balanced? Journal Pract 8(6):704–725
Orabona F, Metta G, Sandini G (2005) Object-based visual attention: a model for a behaving robot. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05)-workshops, pp 89–89