Kenyan Sign Language Recognition Using Ensemble Method

Abstract

The interaction between two or more people is called communication. It may be performed by word, by a written paper, by gestures like hand and head movements, by facial expression, by lips movement. Social inclusion heavily relies on communication, but there still exist severe obstacles to it as very little is known or translated to the Kenyan Sign Language (KSL). Unlike the American or Indian Sign Language, KSL has its own linguistic and cultural systems that cannot be identified by generic recognition systems. This work counters such issues by creating an ensemble machine-learning approach to KSL recognition that integrates the feature-extraction capability of Convolutional Neural Networks (CNN) with the classification strength of k-Nearest Neighbors (KNN). The model was preprocessed, augmented, and annotated using a curated dataset of 8,898 labeled KSL images obtained via Kaggle, in order to increase diversity and decrease noise. Gesture images were fed to the CNN component that extracted high-level spatial features, and then the KNN classifier used the same embeddings to make similarity-based decisions. In order to improve accuracy and reliability in relation to misclassification, a stacking ensemble method was used to combine the two models. Performance on the test set was evaluated using evaluation metrics such as precision, recall, F1-score and confusion matrices. The ensemble model was more accurate (70.32) than standalone classifiers and it was also found to have better recognition of the more complicated KSL gestures. These findings highlight how ensemble learning can be used to overcome communicative barriers between the Deaf and hearing populations in Kenya. The paper offers a technological solution to real-time KSL recognition, as well as provides a base to conduct further studies on larger and more varied datasets and gesture recognition in dynamic time. Finally, the work leads to the social inclusion process because it allows the use of convenient communication means that empower Deaf individuals and facilitate equal access to education, health, and everyday life.