OPTIMIZING SUPPORT VECTOR MACHINE FOR IMBALANCED DATASETS BY COMBINING POSTERIOR PROBABILITY AND CORRELATION METHODS
DOI:
https://doi.org/10.15282/ijsecs.11.1.2025.2.0134Keywords:
Supervised classification, Imbalanced dataset, Posterior Probability, CorrelationAbstract
The challenge of classifying imbalanced data persists in machine learning, particularly in critical applications such as medical diagnosis, fraud detection, and anomaly identification, where detecting the minority class is essential. Conventional classifiers, such as support vector machines (SVM), tend to favor the majority class, leading to reduced sensitivity in identifying minority instances. This study introduces posterior probability and correlation-support vector machine (PC-SVM), a novel approach that integrates posterior probability estimation with correlation analysis to enhance the performance of SVMs on imbalanced datasets. Unlike traditional SVM models, which struggle with class imbalance and require additional data-balancing techniques, PC-SVM dynamically adjusts classification thresholds using posterior probability values and correlation-weighted features, simplifying the classification process while improving its effectiveness. The effectiveness of PC-SVM was evaluated using multiple imbalanced datasets from KEEL, UCI, and Kaggle repositories. The results demonstrate that PC-SVM achieves 100% recall for the minority class, significantly outperforming the traditional SVM, which attained only 80% recall on average. This 20% improvement in recall underscores PC-SVM’s ability to mitigate the imbalance issue without relying on oversampling or cost-sensitive adjustments. Furthermore, PC-SVM exhibits consistent performance across various evaluation metrics, including accuracy, precision, recall, and F1-score, ensuring robust classification results. By improving the detection of minority classes, PC-SVM offers a transformative solution for real-world applications that demand high sensitivity in identifying rare but crucial instances. Its ability to maintain classification integrity without additional balancing techniques positions it as a valuable model for industries such as healthcare, finance, and cybersecurity, where accurate minority class recognition is critical.
Downloads
References
[1] J. Alcaraz, M. Labbé, and M. Landete, “Support vector machine with feature selection: A multiobjective approach,” Expert Systems with Applications, vol. 204, Art. no. 117485, 2022, https://doi.org/10.1016/j.eswa.2022.117485
[2] J. Liu, “Fuzzy support vector machine for imbalanced data with borderline noise,” Fuzzy Sets and Systems, vol. 413, pp. 64–73, 2021, https://doi.org/10.1016/j.fss.2020.07.018
[3] C. Wu, N. Wang, and Y. Wang, “Increasing minority recall support vector machine model for imbalanced data classification,” Discrete Dynamics in Nature and Society, vol. 2021, Art. no. 6647557, 2021, https://doi.org/10.1155/2021/6647557
[4] H. Liu, Z. Liu, W. Jia, D. Zhang, and J. Tan, “A novel imbalanced data classification method based on weakly supervised learning for fault diagnosis,” IEEE Transactions on Industrial Informatics, vol. 18, no. 3, pp. 1583–1593, 2022, https://doi.org/10.1109/TII.2021.3084132
[5] S. Shaikh, S. M. Daudpota, A. S. Imran, and Z. Kastrati, “Towards improved classification accuracy on highly imbalanced text dataset using deep neural language models,” Applied Sciences, vol. 11, no. 2, Art. no. 869, 2021, https://doi.org/10.3390/app11020869
[6] R. A. Hamad, M. Kimura, and J. Lundström, “Efficacy of imbalanced data handling methods on deep learning for smart homes environments,” SN Computer Science, vol. 1, no. 4, Art. no. 205, 2020, https://doi.org/10.1007/s42979-020-00211-1
[7] V. Rupapara, F. Rustam, H. F. Shahzad, A. Mehmood, I. Ashraf, and G. S. Choi, “Impact of SMOTE on imbalanced text features for toxic comments classification using RVVC model,” IEEE Access, vol. 9, pp. 78621–78634, 2021, https://doi.org/10.1109/ACCESS.2021.3083638
[8] H. Qin, H. Zhou, and J. Cao, “Imbalanced learning algorithm based intelligent abnormal electricity consumption detection,” Neurocomputing, vol. 402, pp. 112–123, 2020, https://doi.org/10.1016/j.neucom.2020.03.085
[9] S. S. Mullick, S. Datta, S. S. Dhekane, and S. Das, “Appropriateness of performance indices for imbalanced data classification: An analysis,” Pattern Recognition, vol. 102, Art. no. 107197, 2020, https://doi.org/10.1016/j.patcog.2020.107197
[10] H. Shamsudin, U. K. Yusof, A. Jayalakshmi, and M. N. A. Khalid, “Combining oversampling and undersampling techniques for imbalanced classification: A comparative study using credit card fraudulent transaction dataset,” in Proceedings of the 2020 16th IEEE International Conference on Control & Automation (ICCA), Singapore, 2020, pp. 803–808, https://doi.org/10.1109/ICCA51439.2020.9264517
[11] K. H. Kim and S. Y. Sohn, “Hybrid neural network with cost-sensitive support vector machine for class-imbalanced multimodal data,” Neural Networks, vol. 130, pp. 176–184, 2020, https://doi.org/10.1016/j.neunet.2020.06.026
[12] X. Tao, Q. Li, C. Ren, et al., “Affinity and class probability-based fuzzy support vector machine for imbalanced data sets,” Neural Networks, vol. 122, pp. 289–307, 2020, https://doi.org/10.1016/j.neunet.2019.10.016
[13] C. Jimenez-Castaño, A. Alvarez-Meza, and A. Orozco-Gutierrez, “Enhanced automatic twin support vector machine for imbalanced data classification,” Pattern Recognition, vol. 107, Art. no. 107442, 2020, https://doi.org/10.1016/j.patcog.2020.107442
[14] R. Abo Zidan and G. Karraz, “Gaussian pyramid for nonlinear support vector machine,” Applied Computational Intelligence and Soft Computing, vol. 2022, Art. no. 5255346, 2022, https://doi.org/10.1155/2022/5255346
[15] Y. S. Solanki, P. Chakrabarti, M. Jasinski, et al., “A hybrid supervised machine learning classifier system for breast cancer prognosis using feature selection and data imbalance handling approaches,” Electronics, vol. 10, no. 6, Art. no. 699, 2021, https://doi.org/10.3390/electronics10060699
[16] R. Ramesh Kumar, K. Karthik, P. V. Elumalai, et al., “Investigation of nano composite heat exchanger annular pipeline flow using CFD analysis for crude oil and water characteristics,” Case Studies in Thermal Engineering, vol. 49, Art. no. 103297, 2023, https://doi.org/10.1016/j.csite.2023.103297
[17] B. Richhariya and M. Tanveer, “A reduced universum twin support vector machine for class imbalance learning,” Pattern Recognition, vol. 102, Art. no. 107150, 2020, https://doi.org/10.1016/j.patcog.2019.107150
[18] M. Li, A. Xiong, L. Wang, S. Deng, and J. Ye, “ACO resampling: Enhancing the performance of oversampling methods for class imbalance classification,” Knowledge-Based Systems, vol. 196, Art. no. 105818, 2020, https://doi.org/10.1016/j.knosys.2020.105818
[19] A. S. Hussein, T. Li, C. W. Yohannese, and K. Bashir, “A-SMOTE: A new preprocessing approach for highly imbalanced datasets by improving SMOTE,” International Journal of Computational Intelligence Systems, vol. 12, no. 2, pp. 1412–1422, 2019, https://doi.org/10.2991/ijcis.d.191114.002
[20] P. Gnip, L. Vokorokos, and P. Drotár, “Selective oversampling approach for strongly imbalanced data,” PeerJ Computer Science, vol. 7, pp. 1–22, 2021, https://doi.org/10.7717/peerj-cs.604
[21] X. W. Liang, A. P. Jiang, T. Li, Y. Y. Xue, and G. T. Wang, “LR-SMOTE — An improved unbalanced data set oversampling based on K-means and SVM,” Knowledge-Based Systems, vol. 196, Art. no. 105845, 2020, https://doi.org/10.1016/j.knosys.2020.105845
[22] A. S. Desuky and S. Hussain, “An improved hybrid approach for handling class imbalance problem,” Arabian Journal for Science and Engineering, vol. 46, no. 4, pp. 3853–3864, 2021, https://doi.org/10.1007/s13369-021-05347-7
[23] K. Qi, H. Yang, Q. Hu, and D. Yang, “A new adaptive weighted imbalanced data classifier via improved support vector machines with high-dimension nature,” Knowledge-Based Systems, vol. 185, Art. no. 104933, 2019, https://doi.org/10.1016/j.knosys.2019.104933
[24] H. Shamsudin, U. K. Yusof, H. Yang, and I. S. Isa, “An optimized support vector machine with genetic algorithm for imbalanced data classification,” Jurnal Teknologi, vol. 85, no. 4, pp. 67–74, 2023, https://doi.org/10.11113/jurnalteknologi.v85.19695
[25] Y.-S. Park and J.-S. Lee, “A learning objective controllable sphere-based method for balanced and imbalanced data classification,” IEEE Access, vol. 9, pp. 158010–158026, 2021, https://doi.org/10.1109/ACCESS.2021.3130272
[26] H. Patel, D. S. Rajput, G. T. Reddy, et al., “A review on classification of imbalanced data for wireless sensor networks,” International Journal of Distributed Sensor Networks, vol. 16, no. 4, 2020, https://doi.org/10.1177/1550147720916404
[27] S. Strasser and M. Klettke, “Transparent data preprocessing for machine learning,” in Proceedings of the Workshop on Human-In-the-Loop Data Analytics (HILDA ’24), Santiago, Chile, 2024, https://doi.org/10.1145/3665939.3665960
[28] J. Nalic and A. Svraka, “Importance of data pre-processing in credit scoring models based on data mining approaches,” in Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 2018, pp. 1046–1051, https://doi.org/10.23919/MIPRO.2018.8400191
[29] H. F. Tayeb, M. Karabatak, and C. Varol, “Time series database preprocessing for data mining using Python,” in Proceedings of the 2020 8th International Symposium on Digital Forensics and Security (ISDFS), Beirut, Lebanon, 2020, pp. 20–23, https://doi.org/10.1109/ISDFS49300.2020.9116260
[30] S. Albahra, T. Gorbett, S. Robertson, et al., “Artificial intelligence and machine learning overview in pathology & laboratory medicine: A general review of data preprocessing and basic supervised concepts,” Seminars in Diagnostic Pathology, vol. 40, no. 2, pp. 71–87, 2023, https://doi.org/10.1053/j.semdp.2023.02.002
[31] Z. Liu, “Research on data preprocessing method for artificial intelligence algorithm based on user online behavior,” Journal of Computing and Electronic Information Management, vol. 12, no. 3, pp. 74–78, 2024, https://doi.org/10.54097/qf6fv8j1
[32] A. Q. Md, S. Kulkarni, C. J. Joshua, et al., “Enhanced preprocessing approach using ensemble machine learning algorithms for detecting liver disease,” Biomedicines, vol. 11, no. 2, Art. no. 581, 2023, https://doi.org/10.3390/biomedicines11020581
[33] K. Maharana, B. Mondal, and B. Nemade, “A review: Data pre-processing and data augmentation techniques,” Global Transitions Proceedings, vol. 3, no. 1, pp. 91–99, 2022, https://doi.org/10.1016/j.gltp.2022.04.020
[34] T. H. Duong and T. A. Nguyen-Thi, “A review: Preprocessing techniques and data augmentation for sentiment analysis,” Computational Social Networks, vol. 8, no. 1, pp. 1–16, 2021, https://doi.org/10.1186/s40649-020-00080-x
[35] C. Fan, M. Chen, X. Wang, J. Wang, and B. Huang, “A review on data preprocessing techniques toward efficient and reliable knowledge discovery from building operational data,” Frontiers in Energy Research, vol. 9, pp. 1–17, 2021, https://doi.org/10.3389/fenrg.2021.652801
[36] V. Chernykh, A. Stepnov, and B. O. Lukyanova, “Data preprocessing for machine learning in seismology,” CEUR Workshop Proceedings, vol. 2930, pp. 119–123, 2021. [Online]. Available: http://ceur-ws.org/Vol-2930/
[37] A. J. Mohammed, “Improving classification performance for a novel imbalanced medical dataset using SMOTE method,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 3, pp. 3161–3172, 2020, https://doi.org/10.30534/ijatcse/2020/104932020
[38] A. Kulkarni, D. Chong, and F. A. Batarseh, “Foundations of data imbalance and solutions for a data democracy,” in Data Democracy: At the Frontier of Medical Informatics, Healthcare, and Cybersecurity, F. A. Batarseh and R. Niu, Eds. London, U.K.: Academic Press, 2020, pp. 83–106, https://doi.org/10.1016/B978-0-12-818366-3.00005-8
[39] D. Makowski, M. S. Ben-Shachar, I. Patil, and D. Lüdecke, “Methods and algorithms for correlation analysis in R,” Journal of Open Source Software, vol. 5, no. 51, Art. no. 2306, 2020, https://doi.org/10.21105/joss.02306
[40] M. S. Vural and M. Telceken, “Modification of posterior probability variable with frequency factor according to Bayes theorem,” Journal of Intelligent Systems with Applications, vol. 5, no. 1, pp. 19–26, 2022, https://doi.org/10.54856/jiswa.202205195
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Canggih Ajika Pamungkas, Megat Farez Zuhairi
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
