Araştırma Makalesi
BibTex RIS Kaynak Göster

Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods

Yıl 2021, Cilt: 4 Sayı: 1, 17 - 23, 24.03.2021

Ali Öztürk Melih Karatekin İsa Alperen Saylar Nazım Bahadır Bardakcı

https://doi.org/10.38016/jista.808458
Cited By: 1

Öz

In order for people to be able to communicate with each other, they must be able to agree mutually. Communication is quite difficult for individuals with hearing problems. Such individuals make their lives much more difficult by isolating themselves from society. The people living with hearing loss can understand the contact person with often lip-reading method, but it is quite difficult for them to express themselves to the people. Since the use of sign language has not become widespread around the world, the number of people who know sign language is very low, except for individuals with hearing disabilities. In this study, it was achieved to dynamically recognize the movements of the sign language finger alphabet via image processing using deep learning methods and to translate it into writing. Accordingly, it is aimed to facilitate communication between people who do not know the sign language in daily life and people with hearing loss. The input given to the system is an image of the hand showing any letter from the alphabet. The image of the hand is interpreted by deep learning methods in the system, and it is compared to one of the letters in the alphabet and an output with the similarity ratio to this letter is displayed on the screen. The system has been tested with a total of 1300 images. The overall accuracy rate of the system was calculated as 88% where true positive rate was 87% and false negative rate was 13%.

Anahtar Kelimeler

Sign Language Image Processing Deep Learning Image Analysis Object Detection

Kaynakça

  • [1]- Ashish S. N., Aarti G. A., 2016. Sign language recognition using image based hand gesture recognition techniques. 2016 Online International Conference on Green Engineering and Technologies (IC-GET), Coimbatore/India, 19 November 2016, 1-5. Accessed: 04.02.2020
  • [2]- Sandhya, A., Ananya R., 2018. Recognition of sign language using image processing. International Journal of Business Intelligence and Data Mining, 13(1/2/3):163. doi:10.1504/IJBIDM.2018.088428.
  • [3]- Gaikwad, S., Shetty, A., Satam, A., Rathod, M., Shah, P., 2019. Recognition of American Sign Language using image processing and machine learning. International Journal of Computer Science and Mobile Computing, 8(3), 352-357.
  • [4]- Tamura, S., Kawasaki, S., 1988. Recognition of sign language motion images. Pattern Recognition, 21(4), 343-357.
  • [5]- Kumarage, D., Fernando, S., Fernando, P., Madushanka, D., Samarasinghe, R., 2011. Real-time sign language gesture recognition using still-image comparison & motion recognition. 2011 6th International Conference on Industrial and Information Systems, Kandy/Sri Lanka, 10 October 2011, 169-174. Accessed: 10.01.2020
  • [6]- Tan Y.S., Lim K.M., Tee C., Lee C., 2020. Convolutional neural network with spatial pyramid pooling for hand gesture recognition. Neural Computing and Applications, Published online: 15 September 2020, doi:10.1007/s00521-020-05337-0.
  • [7]- Pramada S., Saylee D., Pranita N., Samiksha N., Vaidya S., 2013. Intelligent sign language recognition using image processing. IOSR Journal of Engineering (IOSRJEN), 3(2), 45-51.
  • [8]- Shivashankara S., Srinath S., 2018. An American sign language recognition system using bounding box and palm features extraction techniques. International Journal of Recent Technology and Engineering (IJRTE), 7(4s): 492-505.
  • [9]- Vargas L.P., Barba L., Torres C.O., Mattos L., 2011. Sign Language Recognition System using Neural Network for Digital Hardware Implementation. Journal of Physics: Conference Series, 274 (2011) 012051, 1-7, doi: 10.1088/1742-6596/274/1/012051.
  • [10]- Shaoqing R., Kaiming H., Ross G., Jian S., 2017. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137-1149.

Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods

Yıl 2021, Cilt: 4 Sayı: 1, 17 - 23, 24.03.2021

Ali Öztürk Melih Karatekin İsa Alperen Saylar Nazım Bahadır Bardakcı

https://doi.org/10.38016/jista.808458
Cited By: 1

Öz

In order for people to be able to communicate with each other, they must be able to agree mutually. Communication is quite difficult for individuals with hearing problems. Such individuals make their lives much more difficult by isolating themselves from society. The people living with hearing loss can understand the contact person with often lip-reading method, but it is quite difficult for them to express themselves to the people. Since the use of sign language has not become widespread around the world, the number of people who know sign language is very low, except for individuals with hearing disabilities. In this study, it was achieved to dynamically recognize the movements of the sign language finger alphabet via image processing using deep learning methods and to translate it into writing. Accordingly, it is aimed to facilitate communication between people who do not know the sign language in daily life and people with hearing loss. The input given to the system is an image of the hand showing any letter from the alphabet. The image of the hand is interpreted by deep learning methods in the system, and it is compared to one of the letters in the alphabet and an output with the similarity ratio to this letter is displayed on the screen. The system has been tested with a total of 1300 images. The overall accuracy rate of the system was calculated as 88% where true positive rate was 87% and false negative rate was 13%.

Anahtar Kelimeler

Sign Language Image Processing Deep Learning Image Analysis Object Detection

Kaynakça

  • [1]- Ashish S. N., Aarti G. A., 2016. Sign language recognition using image based hand gesture recognition techniques. 2016 Online International Conference on Green Engineering and Technologies (IC-GET), Coimbatore/India, 19 November 2016, 1-5. Accessed: 04.02.2020
  • [2]- Sandhya, A., Ananya R., 2018. Recognition of sign language using image processing. International Journal of Business Intelligence and Data Mining, 13(1/2/3):163. doi:10.1504/IJBIDM.2018.088428.
  • [3]- Gaikwad, S., Shetty, A., Satam, A., Rathod, M., Shah, P., 2019. Recognition of American Sign Language using image processing and machine learning. International Journal of Computer Science and Mobile Computing, 8(3), 352-357.
  • [4]- Tamura, S., Kawasaki, S., 1988. Recognition of sign language motion images. Pattern Recognition, 21(4), 343-357.
  • [5]- Kumarage, D., Fernando, S., Fernando, P., Madushanka, D., Samarasinghe, R., 2011. Real-time sign language gesture recognition using still-image comparison & motion recognition. 2011 6th International Conference on Industrial and Information Systems, Kandy/Sri Lanka, 10 October 2011, 169-174. Accessed: 10.01.2020
  • [6]- Tan Y.S., Lim K.M., Tee C., Lee C., 2020. Convolutional neural network with spatial pyramid pooling for hand gesture recognition. Neural Computing and Applications, Published online: 15 September 2020, doi:10.1007/s00521-020-05337-0.
  • [7]- Pramada S., Saylee D., Pranita N., Samiksha N., Vaidya S., 2013. Intelligent sign language recognition using image processing. IOSR Journal of Engineering (IOSRJEN), 3(2), 45-51.
  • [8]- Shivashankara S., Srinath S., 2018. An American sign language recognition system using bounding box and palm features extraction techniques. International Journal of Recent Technology and Engineering (IJRTE), 7(4s): 492-505.
  • [9]- Vargas L.P., Barba L., Torres C.O., Mattos L., 2011. Sign Language Recognition System using Neural Network for Digital Hardware Implementation. Journal of Physics: Conference Series, 274 (2011) 012051, 1-7, doi: 10.1088/1742-6596/274/1/012051.
  • [10]- Shaoqing R., Kaiming H., Ross G., Jian S., 2017. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137-1149.
Toplam 10 adet kaynakça vardır.

Ayrıntılar

Birincil Dil İngilizce
Konular Yapay Zeka
Bölüm Araştırma Makalesi
Yazarlar

Ali Öztürk 0000-0002-1797-2039

Melih Karatekin 0000-0001-5794-6233

İsa Alperen Saylar 0000-0003-4557-5329

Nazım Bahadır Bardakcı 0000-0002-3912-8621

Yayımlanma Tarihi 24 Mart 2021
Gönderilme Tarihi 9 Ekim 2020
Yayımlandığı Sayı Yıl 2021 Cilt: 4 Sayı: 1

Kaynak Göster

APA Öztürk, A., Karatekin, M., Saylar, İ. A., Bardakcı, N. B. (2021). Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods. Journal of Intelligent Systems: Theory and Applications, 4(1), 17-23. https://doi.org/10.38016/jista.808458
AMA Öztürk A, Karatekin M, Saylar İA, Bardakcı NB. Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods. jista. Mart 2021;4(1):17-23. doi:10.38016/jista.808458
Chicago Öztürk, Ali, Melih Karatekin, İsa Alperen Saylar, ve Nazım Bahadır Bardakcı. “Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods”. Journal of Intelligent Systems: Theory and Applications 4, sy. 1 (Mart 2021): 17-23. https://doi.org/10.38016/jista.808458.
EndNote Öztürk A, Karatekin M, Saylar İA, Bardakcı NB (01 Mart 2021) Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods. Journal of Intelligent Systems: Theory and Applications 4 1 17–23.
IEEE A. Öztürk, M. Karatekin, İ. A. Saylar, ve N. B. Bardakcı, “Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods”, jista, c. 4, sy. 1, ss. 17–23, 2021, doi: 10.38016/jista.808458.
ISNAD Öztürk, Ali vd. “Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods”. Journal of Intelligent Systems: Theory and Applications 4/1 (Mart 2021), 17-23. https://doi.org/10.38016/jista.808458.
JAMA Öztürk A, Karatekin M, Saylar İA, Bardakcı NB. Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods. jista. 2021;4:17–23.
MLA Öztürk, Ali vd. “Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods”. Journal of Intelligent Systems: Theory and Applications, c. 4, sy. 1, 2021, ss. 17-23, doi:10.38016/jista.808458.
Vancouver Öztürk A, Karatekin M, Saylar İA, Bardakcı NB. Recognition of Sign Language Letters Using Image Processing and Deep Learning Methods. jista. 2021;4(1):17-23.

Cited By

Validity and Reliability Study of a Turkish Form of the Machine Learning Attitude Scale
Research on Education and Psychology
https://doi.org/10.54535/rep.1017070
 Kapak Resmi İndir

Zeki Sistemler Teori ve Uygulamaları Dergisi