Sign Language Recognition using Deep Learning through LSTM and CNN

Authors

  • Kiran Pandian Faculty of Manufacturing and Mechatronic Engineering Technology, Universiti Malaysia Pahang, 26600 Pahang, Malaysia.
  • Mohd Azraai Mohd Razman Faculty of Manufacturing and Mechatronic Engineering Technology, Universiti Malaysia Pahang, 26600 Pahang, Malaysia.
  • Ismail Mohd Khairuddin Faculty of Manufacturing and Mechatronic Engineering Technology, Universiti Malaysia Pahang, 26600 Pahang, Malaysia.
  • Muhammad Amirul Abdullah Faculty of Manufacturing and Mechatronic Engineering Technology, Universiti Malaysia Pahang, 26600 Pahang, Malaysia.
  • Ahmad Fakhri Ab Nasir Faculty of Computing, Universiti Malaysia Pahang, 26600 Pekan Pahang, Malaysia.
  • Wan Hasbullah Mat Isa Faculty of Manufacturing and Mechatronic Engineering Technology, Universiti Malaysia Pahang, 26600 Pahang, Malaysia.

DOI:

https://doi.org/10.15282/mekatronika.v5i1.9410

Keywords:

Sign Language, Deaf, CNN, LSTM

Abstract

This study presents the application of using deep learning to detect, recognize and translate sign language. Understanding sign language is crucial for communication between the deaf and mute people and the general society. This helps sign language users to easily communicate with others, thus eliminating the differences between both parties. The objectives of this thesis are to extract features from the dataset for sign language recognition model and the formulation of deep learning models and the classification performance to carry out the sign language recognition. First, we develop methodology for an efficient recognition of sign language. Next is to develop multiple system using three different model which is LSTM, CNN and YOLOv5 and compare the real time test result to choose the best model with the highest accuracy. We used same datasets for all algorithms to determine the best algorithm. The YOLOv5 has achieved the highest accuracy of 97% followed by LSTM and CNN with 94% and 66.67%.

Downloads

Published

2023-04-21

How to Cite

[1]
K. Pandian, M. A. Mohd Razman, I. Mohd Khairuddin, M. A. . Abdullah, A. F. Ab Nasir, and W. H. Mat Isa, “Sign Language Recognition using Deep Learning through LSTM and CNN”, Mekatronika: J. Intell. Manuf. Mechatron., vol. 5, no. 1, pp. 67–71, Apr. 2023.

Issue

Section

Original Article

Most read articles by the same author(s)

1 2 3 4 > >>