Transfer Learning for Binary Classification of Thermal Images

Main Article Content

Daniel Alexis Pérez-Aguilar https://orcid.org/0000-0003-4514-2873
Redy Henry Risco-Ramos https://orcid.org/0000-0002-3491-6020
Luis Casaverde-Pacherrez https://orcid.org/0000-0003-4037-1802

Abstract

The classification of thermal images is a key aspect in the industrial sector, since it is usually the starting point for the detection of faults in electrical equipment. In some cases, this task is automated through the use of traditional artificial intelligence techniques, while in others, it is performed manually, which can lead to high rates of human error. This paper presents a comparative analysis between eleven transfer learning architectures (AlexNet, VGG16, VGG19, ResNet, DenseNet, MobileNet v2, GoogLeNet, ResNeXt, Wide ResNet, MNASNet and ShuffleNet) through the use of fine-tuning, in order to perform a binary classification of thermal images in an electrical distribution network. For this, a database with 815 images is available, divided using the 60-20-20 hold-out technique and cross-validation with 5-Folds, to finally analyze their performance using Friedman test. After the experiments, satisfactory results were obtained with accuracies above 85 % in 10 of the previously trained architectures. However, the architecture that was not previously trained had low accuracy; with this, it is concluded that the application of transfer learning through the use of previously trained architectures is a proper mechanism in the classification of this type of images, and represents a reliable alternative to traditional artificial intelligence techniques.
Abstract 338 | PDF (Español (España)) Downloads 160 PDF Downloads 65 EPUB (Español (España)) Downloads 2

References

[1] M. Haenlein and A. Kaplan, “A brief history of artificial intelligence: On the past, present, and future of artificial intelligence,” California Management Review, vol. 61, no. 4, pp. 5–14, 2019. [Online]. Available: https://doi.org/10.1177/0008125619864925
[2] M. Flasinski, Introduction to artificial intelligence. Springer International Publishing, 2016. [Online]. Available: http://doi.org/10.1007/978-3-319-40022-8
[3] M.-H. Huang and R. T. Rust, “Artificial intelligence in service,” Journal of Service Research, vol. 21, no. 2, pp. 155–172, 2018. [Online]. Available: https://doi.org/10.1177/1094670517752459
[4] Z. Aung, I. S. Mikhaylov, and Y. T. Aung, “Artificial intelligence methods application in oil industry,” in 2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), 2020, pp. 563–567. [Online]. Available: https://doi.org/10.1109/EIConRus49466.2020.9039330
[5] T. P. Carvalho, F. A. A. M. N. Soares, R. Vita, R. da P. Francisco, J. ao P. Basto, and S. G. S. Alcalá, “A systematic literature review of machine learning methods applied to predictive maintenance,” Computers & Industrial Engineering, vol. 137, p. 106024, 2019. [Online]. Available: https://doi.org/10.1016/j.cie.2019.106024
[6] S. Wan, L. Qi, X. Xu, C. Tong, and Z. Gu, “Deep learning models for real-time human activity recognition with smartphones,” Mobile Networks and Applications, vol. 25, no. 2, pp. 743–755, Apr. 2020. [Online]. Available: https://doi.org/10.1007/s11036-019-01445-x
[7] V. Golodov, A. Zavei-Boroda, S. Ivanov, and K. Nikolskaya, “Development of a deep learning neural network for human movements analysis,” in 2017 Second Russia and Pacific Conference on Computer Technology and Applications (RPC), 2017, pp. 72–74. [Online]. Available: https://doi.org/10.1109/RPC.2017.8168071
[8] E. A. Galindo, J. A. Perdomo, and J. C. Figueroa-García, “Estudio comparativo entre máquinas de soporte vectorial multiclase, redes neuronales artificiales y sistema de inferencia neuro-difuso autoorganizado para problemas de clasificación,” Información tecnológica, vol. 31, pp. 273–286, 02 2020. [Online]. Available: http://dx.doi.org/10.4067/S0718-07642020000100273
[9] A. Brunetti, D. Buongiorno, G. F. Trotta, and V. Bevilacqua, “Computer vision and deep learning techniques for pedestrian detection and tracking: A survey,” Neurocomputing, vol. 300, pp. 17–33, 2018. [Online]. Available: https://doi.org/10.1016/j.neucom.2018.01.092
[10] I. Yildiz, P. Tian, J. Dy, D. Erdogmus, J. Brown, J. Kalpathy-Cramer, S. Ostmo, J. Peter Campbell, M. F. Chiang, and S. Ioannidis, “Classification and comparison via neural networks,” Neural Networks, vol. 118, pp. 65–80, 2019. [Online]. Available: https://doi.org/10.1016/j.neunet.2019.06.004
[11] Y. Jung, “Multiple predicting k-fold crossvalidation for model selection,” Journal of Nonparametric Statistics, vol. 30, no. 1, pp. 197–215, 2018. [Online]. Available: https://doi.org/10.1080/10485252.2017.1404598
[12] F. Pacheco, J. Valente de Oliveira, R.-V. Sénchez, M. Cerrada, D. Cabrera, C. Li, G. Zurita, and M. Artés, “A statistical comparison of neuroclassifiers and feature selection methods for gearbox fault diagnosis under realistic conditions,” Neurocomputing, vol. 194, pp. 192–206, 2016. [Online]. Available: https://doi.org/10.1016/j.neucom.2016.02.028
[13] D. W. Zimmerman and B. D. Zumbo, “Relative power of the Wilcoxon test, the Friedman test, and repeated-measures ANOVA on ranks,” The Journal of Experimental Education, vol. 62, no. 1, pp. 75–86, 1993. [Online]. Available: https://doi.org/10.1080/00220973.1993.9943832
[14] C. Lile and L. Yiqun, “Anomaly detection in thermal images using deep neural networks,” in 2017 IEEE International Conference on Image Processing (ICIP), 2017, pp. 2299–2303. [Online]. Available: https://doi.org/10.1109/ICIP.2017.8296692
[15] A. Dragomir, M. Adam, M. Andruçcâ, A. Munteanu, and E. Boghiu, “Considerations regarding infrared thermal stresses monitoring of electrical equipment,” in 2017 International Conference on Electromechanical and Power Systems (SIELMEN), 2017, pp. 100–103. [Online]. Available: https://doi.org/10.1109/SIELMEN.2017.8123307
[16] F. Fambrini, Y. Iano, D. G. Caetano, A. A. D. Rodríguez, C. Moya, E. Carrara, R. Arthur, F. C. Cabello, J. V. Zubem, L. M. Del Val Cura, J. a. B. Destro Filho, J. R. Campos, and J. H. Saito, “Gpu cuda jseg segmentation algorithm associated with deep learning classifier for electrical network images identification,” Procedia Computer Science, vol. 126, pp. 557–565, 2018, knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 22nd International Conference, KES-2018, Belgrade, Serbia. [Online]. Available: https://doi.org/10.1016/j.procs.2018.07.290
[17] X. W. X. L. Z. J. Wenzhen Yang, Jiali Luo and Z. Pan, “Image tactile perception with an improved jseg algorithm,” International Journal of Performability Engineering, vol. 14, no. 1, p. 77, 2018. [Online]. Available: https://doi.org/10.23940/ijpe.18.01.p9.7788
[18] I. Ullah, F. Yang, R. Khan, L. Liu, H. Yang, B. Gao, and K. Sun, “Predictive maintenance of power substation equipment by infrared thermography using a machine-learning approach,” Energies, vol. 10, no. 12, 2017. [Online]. Available: https://doi.org/10.3390/en10121987
[19] H. Ramchoun, Y. Ghanou, M. Ettaouil, and M. A. J. Idrissi, “Multilayer perceptron: Architecture optimization and training,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, no. 1, 2016. [Online]. Available: http://doi.org/10.9781/ijimai.2016.415
[20] A. S. Nazmul Huda, S. Taib, M. S. Jadin, and D. Ishak, “A semi-automatic approach for thermographic inspection of electrical installations within buildings,” Energy and Buildings, vol. 55, pp. 585–591, 2012. [Online]. Available: https://doi.org/10.1016/j.enbuild.2012.09.014
[21] C. Yuan, X. Sun, and R. Lv, “Fingerprint liveness detection based on multi-scale 0PQ and PCA,” China Communications, vol. 13, no. 7, pp. 60–65, 2016. [Online]. Available: https://doi.org/10.1109/CC.2016.7559076
[22] H. Zou and F. Huang, “A novel intelligent fault diagnosis method for electrical equipment using infrared thermography,” Infrared Physics & Technology, vol. 73, pp. 29–35, 2015. [Online]. Available: https://doi.org/10.1016/j.infrared.2015.08.019
[23] S.-S. Yu, S.-W. Chu, C.-M. Wang, Y.-K. Chan, and T.-C. Chang, “Two improved kmeans algorithms,” Applied Soft Computing, vol. 68, pp. 747–755, 2018. [Online]. Available: https://doi.org/10.1016/j.asoc.2017.08.032
[24] T. V. Phan, S. Sultana, T. G. Nguyen, and T. Bauschert, “Q - transfer: A novel framework for efficient deep transfer learning in networking,” in 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), 2020, pp. 146–151. [Online]. Available: https://doi.org/10.1109/ICAIIC48513.2020.9065240
[25] M. Ebrahim, M. Al-Ayyoub, and M. A. Alsmirat, “Will transfer learning enhance imagenet classification accuracy using imagenet-pretrained models?” in 2019 10th International Conference on Information and Communication Systems (ICICS), 2019, pp. 211–216. [Online]. Available: https://doi.org/10.1109/IACS.2019.8809114
[26] T. Alshalali and D. Josyula, “Fine-tuning of pre-trained deep learning models with extreme learning machine,” in 2018 International Conference on Computational Science and Computational Intelligence (CSCI), 2018, pp. 469–473. [Online]. Available: https://doi.org/10.1109/CSCI46756.2018.00096
[27] G. Vrbançiç and V. Podgorelec, “Transfer learning with adaptive fine-tuning,” IEEE Access, vol. 8, pp. 196 197–196 211, 2020. [Online]. Available: https://doi.org/10.1109/ACCESS.2020.3034343
[28] T. Kaur and T. K. Gandhi, “Automated brain image classification based on vgg-16 and transfer learning,” in 2019 International Conference on Information Technology (ICIT), 2019, pp. 94–98. [Online]. Available: https://doi.org/10.1109/ICIT48102.2019.00023
[29] R. L. Gálvez, E. P. Dadios, A. A. Bandala, and R. R. P. Vicerra, “Threat object classification in X-ray images using transfer learning,” in 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology,Communication and Control, Environment and Management (HNICEM), 2018, pp. 1–5. [Online]. Available: https://doi.org/10.1109/HNICEM.2018.8666344
[30] D. Xue, X. Zhou, C. Li, Y. Yao, M. M. Rahaman, J. Zhang, H. Chen, J. Zhang, S. Qi, and H. Sun, “An application of transfer learning and ensemble learning techniques for cervical histopathology image classification,” IEEE Access, vol. 8, pp. 104 603–104 618, 2020. [Online]. Available: https://doi.org/10.1109/ACCESS.2020.2999816
[31] E. Cengil and A. Çinar, “Multiple classification of flower images using transfer learning,” in 2019 International Artificial Intelligence and Data Processing Symposium (IDAP), 2019, pp. 1–6. [Online]. Available: https://doi.org/10.1109/IDAP.2019.8875953
[32] J. R. Rajayogi, G. Manjunath, and G. Shobha, “Indian food image classification with transfer learning,” in 2019 4th International Conference on Computational Systems and Information Technology for Sustainable Solution (CSITSS), vol. 4, 2019, pp. 1–4. [Online]. Available: https://doi.org/10.1109/CSITSS47250.2019.9031051
[33] H. Shao, M. Xia, G. Han, Y. Zhang, and J. Wan, “Intelligent fault diagnosis of rotorbearing system under varying working conditions with modified transfer convolutional neural network and thermal images,” IEEE Transactions on Industrial Informatics, vol. 17, no. 5, pp. 3488–3496, 2021. [Online]. Available: https://doi.org/10.1109/TII.2020.3005965
[34] O. Janssens, R. Van de Walle, M. Loccufier, and S. Van Hoecke, “Deep learning for infrared thermal image based machine health monitoring,” IEEE/ASME Transactions on Mechatronics, vol. 23, no. 1, pp. 151–159, 2018. [Online]. Available: https://doi.org/10.1109/TMECH.2017.2722479
[35] T. Carneiro, R. V. Medeiros Da NóBrega, T. Nepomuceno, G.-B. Bian, V. H. C. De Albuquerque, and P. P. R. Filho, “Performance analysis of google colaboratory as a tool for accelerating deep learning applications,” IEEE Access, vol. 6, pp. 61 677–61 685, 2018. [Online]. Available: https://doi.org/10.1109/ACCESS.2018.2874767
[36] A. S. N. Huda and S. Taib, “Suitable features selection for monitoring thermal condition of electrical equipment using infrared thermography,” Infrared Physics & Technology, vol. 61, pp. 184–191, 2013. [Online]. Available: https://doi.org/10.1016/j.infrared.2013.04.012
[37] M. S. Jadin, S. Taib, and K. H. Ghazali, “Feature extraction and classification for detecting the thermal faults in electrical installations,” Measurement, vol. 57, pp. 15–24, 2014. [Online]. Available: https://doi.org/10.1016/j.measurement.2014.07.010
[38] W. I. Technology, ThermoProTP8S™ IR Thermal Camera. User Manual. Wuhan Guide Infrared Technology Co., Ltd., 2007. [Online]. Available: https://bit.ly/3bVGd0u
[39] L. Sandjakoska and F. Stojanovska, “How initialization is related to deep neural networks generalization capability: Experimental study,” in 2020 55th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), 2020, pp. 163–166. [Online]. Available: https://doi.org/10.1109/ICEST49890.2020.9232882
[40] C. Heghedus, A. Chakravorty, and C. Rong, “Neural network frameworks. comparison on public transportation prediction,” in 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2019, pp. 842–849. [Online]. Available: https://doi.org/10.1109/IPDPSW.2019.00138
[41] A. A. Almisreb, N. Jamil, and N. M. Din, “Utilizing alexnet deep transfer learning for ear recognition,” in 2018 Fourth International Conference on Information Retrieval and Knowledge Management (CAMP), 2018, pp. 1–5. [Online]. Available: https://doi.org/10.1109/INFRKM.2018.8464769
[42] S. Liu and W. Deng, “Very deep convolutional neural network based image classification using small training sample size,” in 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), 2015, pp. 730–734. [Online]. Available: https://doi.org/10.1109/ACPR.2015.7486599
[43] J. Xiao, J. Wang, S. Cao, and B. Li, “Application of a novel and improved VGG-19 network in the detection of workers wearing masks,” Journal of Physics: Conference Series, vol. 1518, p. 012041, apr 2020. [Online]. Available: https://doi.org/10.1088/1742-6596/1518/1/012041
[44] A. Budhiman, S. Suyanto, and A. Arifianto, “Melanoma cancer classification using resnet with data augmentation,” in 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), 2019, pp. 17–20. [Online]. Available: https://doi.org/10.1109/ISRITI48646.2019.9034624
[45] K. Zhang, Y. Guo, X. Wang, J. Yuan, and Q. Ding, “Multiple feature reweight densenet for image classification,” IEEE Access, vol. 7, pp. 9872–p880, 2019. [Online]. Available: https://doi.org/10.1109/ACCESS.2018.2890127
[46] T. Fang, “A novel computer-aided lung cancer detection method based on transfer learning from googlenet and median intensity projections,” in 2018 IEEE International Conference on Computer and Communication Engineering Technology (CCET), 2018, pp. 286–290. [Online]. Available: https://doi.org/10.1109/CCET.2018.8542189
[47] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1–9. [Online]. Available: https://doi.org/10.1109/CVPR.2015.7298594
[48] GeeksforGeeks. (2020) Understanding googlenet model - CNN architecture. [Online]. Available: https://bit.ly/2RLmiuc
[49] K. Fu, L. Sun, X. Kang, and F. Ren, “Text detection for natural scene based on mobilenet V2 and U-net,” in 2019 IEEE International Conference on Mechatronics and Automation (ICMA), 2019, pp. 1560–1564. [Online]. Available: https://doi.org/10.1109/ICMA.2019.8816384
[50] C. Qiu, M. Schmitt, H. Taubenböck, and X. X. Zhu, “Mapping human settlements with multiseasonal sentinel-2 imagery and attention-based resnext,” in 2019 Joint Urban Remote Sensing Event (JURSE), 2019, pp. 1–4. [Online]. Available: https://doi.org/10.1109/JURSE.2019.8809009
[51] S. Zagoruyko and N. Komodakis, “Wide residual networks,” in Proceedings of the British Machine Vision Conference (BMVC), E. R. H. Richard C. Wilson and W. A. P. Smith, Eds. BMVA Press, September 2016, pp. 87.1–87.12. [Online]. Available: https://dx.doi.org/10.5244/C.30.87
[52] M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, and Q. V. Le, “Mnasnet: Platform-aware neural architecture search for mobile,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2815–2823. [Online]. Available: https://doi.org/10.1109/CVPR.2019.00293
[53] Y. Li and C. Lv, “Ss-yolo: An object detection algorithm based on YOLOv3 and shufflenet,” in 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), vol. 1, 2020, pp. 769–772. [Online]. Available: https://doi.org/10.1109/ITNEC48623.2020.9085091
[54] PyTorch. (2019) TORCHVISION.MODELS. [Online]. Available: https://bit.ly/2QSClGe
[55] X. Song, Y. Du, and J. Jackson, “An empirical study on hyperparameters and their interdependence for RL generalization,” arXiv preprint arXiv, vol. abs/1906.00431, 2019. [Online]. Available: https://bit.ly/3ulY3zZ
[56] J. N. van Rijn and F. Hutter, “Hyperparameter importance across datasets,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’18. New York, NY, USA: Association for Computing Machinery, 2018, pp. 2367–2376. [Online]. Available: https://doi.org/10.1145/3219819.3220058
[57] A. Aravkin, J. V. Burke, A. Chiuso, and G. Pillonetto, “On the estimation of hyperparameters for empirical bayes estimators: Maximum marginal likelihood vs minimum MSE,” IFAC Proceedings Volumes, vol. 45, no. 16, pp. 125–130, 2012, 16th IFAC Symposium on System Identification. [Online]. Available: https://doi.org/10.3182/20120711-3-BE-2027.00353
[58] A. Mikolajczyk and M. Grochowski, “Data augmentation for improving deep learning in image classification problem,” in 2018 International Interdisciplinary PhD Workshop (IIPhDW), 2018, pp. 117–122. [Online]. Available: https://doi.org/10.1109/IIPHDW.2018.8388338
[59] C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” Journal of Big Data, vol. 6, no. 1, p. 60, Jul. 2019. [Online]. Available: https://doi.org/10.1186/s40537-019-0197-0
[60] D. Avola, L. Cinque, G. L. Foresti, F. Lamacchia, M. R. Marini, L. Perini, K. Qorraj, and G. Telesca, “A shape comparison reinforcement method based on feature extractors and f1-score,” in 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019, pp. 2155–2159. [Online]. Available: https://doi.org/10.1109/SMC.2019.8914601
[61] J. Amat Rodrigo. (2020) Validación de modelos predictivos: Cross-validation, oneleaveout, bootstraping. [Online]. Available: https://bit.ly/3bYgPHk