A comparison of convolutional neural networks for Kazakh sign language recognition
For people with disabilities, sign language is the most important means of communication. Therefore, more and more authors of various papers and scientists around the world are proposing solutions to use intelligent hand gesture recognition systems. Such a system is aimed not only for those who wish...
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | article |
| Lenguaje: | EN RU UK |
| Publicado: |
PC Technology Center
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/bbffd941b98c492e9d9606919a0e9b09 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:bbffd941b98c492e9d9606919a0e9b09 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:bbffd941b98c492e9d9606919a0e9b092021-11-04T14:06:13ZA comparison of convolutional neural networks for Kazakh sign language recognition1729-37741729-406110.15587/1729-4061.2021.241535https://doaj.org/article/bbffd941b98c492e9d9606919a0e9b092021-10-01T00:00:00Zhttp://journals.uran.ua/eejet/article/view/241535https://doaj.org/toc/1729-3774https://doaj.org/toc/1729-4061For people with disabilities, sign language is the most important means of communication. Therefore, more and more authors of various papers and scientists around the world are proposing solutions to use intelligent hand gesture recognition systems. Such a system is aimed not only for those who wish to understand a sign language, but also speak using gesture recognition software. In this paper, a new benchmark dataset for Kazakh fingerspelling, able to train deep neural networks, is introduced. The dataset contains more than 10122 gesture samples for 42 alphabets. The alphabet has its own peculiarities as some characters are shown in motion, which may influence sign recognition. Research and analysis of convolutional neural networks, comparison, testing, results and analysis of LeNet, AlexNet, ResNet and EffectiveNet – EfficientNetB7 methods are described in the paper. EffectiveNet architecture is state-of-the-art (SOTA) and is supposed to be a new one compared to other architectures under consideration. On this dataset, we showed that the LeNet and EffectiveNet networks outperform other competing algorithms. Moreover, EffectiveNet can achieve state-of-the-art performance on nother hand gesture datasets. The architecture and operation principle of these algorithms reflect the effectiveness of their application in sign language recognition. The evaluation of the CNN model score is conducted by using the accuracy and penalty matrix. During training epochs, LeNet and EffectiveNet showed better results: accuracy and loss function had similar and close trends. The results of EffectiveNet were explained by the tools of the SHapley Additive exPlanations (SHAP) framework. SHAP explored the model to detect complex relationships between features in the images. Focusing on the SHAP tool may help to further improve the accuracy of the modelChingiz KenshimovSamat MukhanovTimur MerembayevDidar YedilkhanPC Technology Centerarticlehand gesture recognitionsign language recognitionconvolutional neural network (cnn)deep learningTechnology (General)T1-995IndustryHD2321-4730.9ENRUUKEastern-European Journal of Enterprise Technologies, Vol 5, Iss 2 (113), Pp 44-54 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN RU UK |
| topic |
hand gesture recognition sign language recognition convolutional neural network (cnn) deep learning Technology (General) T1-995 Industry HD2321-4730.9 |
| spellingShingle |
hand gesture recognition sign language recognition convolutional neural network (cnn) deep learning Technology (General) T1-995 Industry HD2321-4730.9 Chingiz Kenshimov Samat Mukhanov Timur Merembayev Didar Yedilkhan A comparison of convolutional neural networks for Kazakh sign language recognition |
| description |
For people with disabilities, sign language is the most important means of communication. Therefore, more and more authors of various papers and scientists around the world are proposing solutions to use intelligent hand gesture recognition systems. Such a system is aimed not only for those who wish to understand a sign language, but also speak using gesture recognition software. In this paper, a new benchmark dataset for Kazakh fingerspelling, able to train deep neural networks, is introduced. The dataset contains more than 10122 gesture samples for 42 alphabets. The alphabet has its own peculiarities as some characters are shown in motion, which may influence sign recognition.
Research and analysis of convolutional neural networks, comparison, testing, results and analysis of LeNet, AlexNet, ResNet and EffectiveNet – EfficientNetB7 methods are described in the paper. EffectiveNet architecture is state-of-the-art (SOTA) and is supposed to be a new one compared to other architectures under consideration. On this dataset, we showed that the LeNet and EffectiveNet networks outperform other competing algorithms. Moreover, EffectiveNet can achieve state-of-the-art performance on nother hand gesture datasets.
The architecture and operation principle of these algorithms reflect the effectiveness of their application in sign language recognition. The evaluation of the CNN model score is conducted by using the accuracy and penalty matrix. During training epochs, LeNet and EffectiveNet showed better results: accuracy and loss function had similar and close trends. The results of EffectiveNet were explained by the tools of the SHapley Additive exPlanations (SHAP) framework. SHAP explored the model to detect complex relationships between features in the images. Focusing on the SHAP tool may help to further improve the accuracy of the model |
| format |
article |
| author |
Chingiz Kenshimov Samat Mukhanov Timur Merembayev Didar Yedilkhan |
| author_facet |
Chingiz Kenshimov Samat Mukhanov Timur Merembayev Didar Yedilkhan |
| author_sort |
Chingiz Kenshimov |
| title |
A comparison of convolutional neural networks for Kazakh sign language recognition |
| title_short |
A comparison of convolutional neural networks for Kazakh sign language recognition |
| title_full |
A comparison of convolutional neural networks for Kazakh sign language recognition |
| title_fullStr |
A comparison of convolutional neural networks for Kazakh sign language recognition |
| title_full_unstemmed |
A comparison of convolutional neural networks for Kazakh sign language recognition |
| title_sort |
comparison of convolutional neural networks for kazakh sign language recognition |
| publisher |
PC Technology Center |
| publishDate |
2021 |
| url |
https://doaj.org/article/bbffd941b98c492e9d9606919a0e9b09 |
| work_keys_str_mv |
AT chingizkenshimov acomparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition AT samatmukhanov acomparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition AT timurmerembayev acomparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition AT didaryedilkhan acomparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition AT chingizkenshimov comparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition AT samatmukhanov comparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition AT timurmerembayev comparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition AT didaryedilkhan comparisonofconvolutionalneuralnetworksforkazakhsignlanguagerecognition |
| _version_ |
1718444865022328832 |