Automated Aircraft Structural Defect Detection Using Deep Learning and Computer Vision
DOI:
https://doi.org/10.15282/mekatronika.v7i2.11787Keywords:
1Computer Vision, Deep Learning, Defect detection, Mask R-CNN, Instance Segmentation, Object DetectionAbstract
Manual aircraft inspections are labor-intensive and susceptible to human error, potentially compromising safety and accuracy. This study presents an automated defect detection framework based on the Mask R-CNN instance segmentation model for identifying cracks and dents in aircraft structures. A dataset of 2,000 annotated images was generated using augmentation techniques and used to train a ResNet-101-based Mask R-CNN model. The system achieved high detection performance, with crack detection reaching a precision of 92.8%, recall of 88.7%, and F1 score of 90.75%; dent detection achieved 91.2% precision, 88.1% recall, and an F1 score of 89.62%. Evaluation using mean Intersection over Union (IoU) and Average Precision (AP@[IoU=0.50:0.95]) confirmed accurate defect localization and segmentation. These findings demonstrate the model's potential to improve inspection reliability and operational efficiency, contributing to safer, more consistent aircraft maintenance practices.
References
[1] C. V. Oster, J. S. Strong, and C. K. Zorn, “Analyzing aviation safety: Problems, challenges, opportunities,” Research in Transportation Economics, vol. 43, no. 1, pp. 148–164, 2013.
[2] I. Savage, “Comparing the fatality risks in United States transportation across modes and over time,” Research in Transportation Economics, vol. 43, no. 1, pp. 9–22, 2013.
[3] A. Barnett, “Aviation safety: A whole new world?” Transportation Science, vol. 54, no. 1, pp. 84–96, 2020.
[4] N. E. Andenyangtso, R. D. Enyinna, and M. U. Bonet, “Effective Maintenance Of Aircraft Antiskid Brake System,” Mekatronika: Journal of Intelligent Manufacturing and Mechatronics, vol. 6, no. 1, pp. 19–29, 2024.
[5] Y. Li, Z. Han, H. Xu, L. Liu, X. Li, and K. Zhang, “Applied sciences YOLOv3-Lite : A lightweight crack detection network for aircraft structure based on depthwise separable convolutions,” Applied Sciences, vol. 9, no. 18, p. 3781, 2019.
[6] B. F. Spencer, V. Hoskere, and Y. Narazaki, “Advances in computer vision-based civil infrastructure inspection and monitoring,” Engineering, vol. 5, no. 2, pp. 199–222, 2019.
[7] Y. D. V. Yasuda, F. A. M. Cappabianco, L. E. G. Martins, and J. A. B. Gripp, “Aircraft visual inspection: A systematic literature review,” Computers in Industry, vol. 141, p. 103695, 2022.
[8] W. Chen and S. Huang, “Human reliability analysis for visual inspection in aviation maintenance by a Bayesian network approach,” Transportation Research Record, vol. 2449, pp. 105–113, 2014.
[9] C. G. Drury and A. K. Gramopadhye, “Human factors in aviation maintenance: how we got to where we are,” International Journal of Industrial Ergonomics, vol. 26, no. 2, pp. 125–151, 2000.
[10] W. Wang and C. Su, “Semi-supervised semantic segmentation network for surface crack detection,” Automation in Construction, vol. 128, p. 103786, 2021.
[11] S. Teng, Z. Liu, G. Chen, and L. Cheng, “Concrete crack detection based on well-known feature extractor model and the YOLO_v2 network,” Applied Sciences, vol. 11, no. 2, pp. 1–13, 2021.
[12] D. Medak, L. Posilovic, M. Subasic, M. Budimir, and S. Loncaric, “Automated defect detection from ultrasonic images using deep learning,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 68, no. 10, pp. 3126–3134, 2021.
[13] I. Abdel-Qader, O. Abudayyeh, and M. E. Kelly, “Analysis of edge-detection techniques for crack identification in bridges,” Journal of Computing in Civil Engineering, vol. 17, no. 4, pp. 255–263, 2003.
[14] D. Lattanzi and G. R. Miller, “Robust automated concrete damage detection algorithms for field applications,” J. Journal of Computing in Civil Engineering, vol. 28, no. 2, pp. 253–262, 2014.
[15] Y. Tomoyuki and H. Shuji, “Improved percolation-based method for crack detection in concrete surface images,” in 2008 19th International Conference on Pattern Recognition, pp. 1–4, 2008.
[16] H. Kim, E. Ahn, S. Cho, M. Shin, and S. H. Sim, “Comparative analysis of image binarization methods for crack identification in concrete structures,” Cement and Concrete Research, vol. 99, pp. 53–61, 2017.
[17] Z. Shen, X. Wan, F. Ye, X. Guan, and S. Liu, “Deep learning based framework for automatic damage detection in aircraft engine borescope inspection,” in 2019 International Conference on Computing, Networking and Communications, pp. 1005–1010, 2019.
[18] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research., vol. 15, pp. 1929–1958, 2014.
[19] S. Wu and J. I. E. Fang, “Sample and structure-guided network for road crack detection,” IEEE Access, vol. 7, pp. 130032–130043, 2019.
[20] X. Tang, “DeepID3: Face recognition with very deep neural networks,” arXiv preprint arXiv:1502.00873, 2015.
[21] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.
[22] C. Wang, X. Wang, X. Zhou, and Z. Li, “The aircraft skin crack inspection based on different-source sensors and support vector machines,” Journal of Nondestructive Evaluation, vol. 35, no. 3, p. 46, 2016.
[23] T. George, “Crack analysis of aircraft slat and flap sections using NDT techniques,” International Advanced Research Journal in Science, Engineering and Technology, vol. 3, no. 5, pp. 48–51, 2016.
[24] T. Malekzadeh, M. Abdollahzadeh, H. Nejati, and N.-M. Cheung, “Aircraft fuselage defect detection using deep neural networks,” arXiv preprint arXiv:1712.09213, 2017.
[25] L. Yang, B. Li, W. Li, Z. Liu, G. Yang, and J. Xiao, “Deep concrete inspection using unmanned aerial vehicle towards CSSC database,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 24-28, 2017.
[26] K. Gopalakrishnan, H. Gholami, A. Vidyadharan, and A. Agrawal, “Crack damage detection in unmanned aerial vehicle images of civil infrastructure using pre-trained deep learning model,” International Journal for Traffic and Transport Engineering, vol. 8, no. 1, pp. 1–14, 2018.
[27] S. Bouarfa, A. Doğru, R. Arizar, R. Aydoğan, and J. Serafico, “Towards automated aircraft maintenance inspection. A use case of detecting aircraft dents using mask r-cnn,” AIAA Scitech 2020 Forum, p. 0389, 2020.
[28] A. Doğru, S. Bouarfa, R. Arizar, and R. Aydoğan, “Using convolutional neural networks to automate aircraft maintenance visual inspection,” Aerospace, vol. 7, no. 12, pp. 1–22, 2020.
[29] H. Huang, S. Zhao, D. Zhang, and J. Chen, “Deep learning-based instance segmentation of cracks from shield tunnel lining images,” Structure and Infrastructure Engineering, vol. 18, no. 2, pp. 183–196, 2022.
[30] R. Vorobev, I. Vasilev, and I. Kremnev, “Tomography of materials and structures segmentation of structural defects in polymer composite computed tomography images with deep learning models,” Tomography of Materials and Structures, vol. 3, p. 100014, 2023.
[31] D. Denhof, B. Staar, M. Lütjen, and M. Freitag, “Automatic optical surface inspection of wind turbine rotor blades using convolutional neural networks,” Procedia CIRP, vol. 81, pp. 1166–1170, 2023.
[32] X. Xu, M. Zhao, P. Shi, R. Ren, X. He, X. Wei, et al., “Crack detection and comparison study based on faster R-CNN and mask R-CNN,” Sensors, vol. 22, no. 3, p. 1215, 2022.
[33] H. Fujita, M. Itagaki, K. Ichikawa, Y. K. Hooi, K. Kawano, and R. Yamamoto, “Fine-tuned pre-trained mask R-CNN models for surface object detection,” arXiv preprint arXiv:2010.11464, 2020.
[34] L. Jiao et al., “A survey of deep learning-based object detection,” IEEE Access, vol. 7, pp. 128837–128868, 2019.
[35] K. Zhang, H. D. Cheng, B. Zhang, and D. Ph, “Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning,” Journal of Computing in Civil Engineering, vol. 32, no. 2012, pp. 1–12, 2018.
[36] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, 2017.
[37] A. N. Azhar and M. L. Khodra, “Fine-tuning pretrained multilingual BERT model for Indonesian aspect-based sentiment analysis,” in 2020 7th International Conference on Advance Informatics: Concepts, Theory and Applications, pp. 2980–2988, 2020.
[38] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 386–397, 2020.
[39] Z. Zhou, M. Zhang, J. Chen, and X. Wu, “Detection and classification of multi-magnetic targets using mask-RCNN,” IEEE Access, vol. 8, pp. 187202–187207, 2020.
[40] L. Zhang, Z. Wang, L. Wang, Z. Zhang, X. Chen, and L. Meng, “Machine learning-based real-time visible fatigue crack growth detection,” Digital Communications and Networks, vol. 7, no. 4, pp. 551–558, 2021.
[41] M. M. Rosso, G. Marasco, S. Aiello, A. Aloisio, B. Chiaia, and G. C. Marano, “Convolutional networks and transformers for intelligent road tunnel investigations,” Computers & Structures, vol. 275, p. 106918, 2023.
[42] O. Yaman, F. Yol, and A. Altinors, “A fault detection method based on embedded feature extraction and SVM classification for UAV motors,” Microprocessors and Microsystems, vol. 94, p. 104683, 2022.
[43] N. Hütten, M. A. Gomes, T. Meisen, F. Hölken, K. Andricevic, and R. Meyes, “Deep learning for automated visual inspection in manufacturing and maintenance: A survey of open-access papers,” Applied System Innovation, vol. 7, pp. 1–38, 2024.
[44] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN : Towards real-time object detection with region proposal networks,” Advances in Neural Information Processing Systems, vol. 28, pp. 1–14, 2016.
[45] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” European Conference on Computer Vision, pp. 213-229, 2020.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 The Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
