Development of the Multilingual Sign Language Learning Game for Interactive Education

Arifa Ashrafi, Yuriy Philippovich, Viktor Mokhnachev, Ali Makhmud

Abstract


This paper presents a fun and engaging educational game designed to help people learn sign languages. It is built using web technologies like HTML5, CSS3, and JavaScript, making it accessible online. Players go on a first-person journey through three sign languages: Bengali, Russian, and American Sign Language. Each language has five different levels to keep the learning interesting. A special learning cycle called 'Hunt-Revelation-Reinforcement,' has been used. It helps players learn and remember signs more effectively. High-quality graphics and a child friendly interface of the game adjusts the difficulty based on how well the player is doing. For alpha testing, 50 participants from different ages and backgrounds volunteered. The results were very positive: players improved their ability to recognize signs by 35%. Most (85%) volunteers stayed motivated to keep learning. It also received a high usability score of 90 out of 100. These promising results suggest that this game has potential for further development and wider distribution.

Full Text:

PDF

References


Ashrafi, A., Mokhnachev, V.S., Philippovich, Y.N., Tsilenko, L.P. (2020). Development of Image Dataset Using Hand Gesture Recognition System for Progression of Sign Language Translator. In: Silhavy, R., Silhavy, P., Prokopova, Z. (eds) Software Engineering Perspectives in Intelligent Systems. CoMeSySo 2020. Advances in Intelligent Systems and Computing, vol 1294. Springer, Cham. https://doi.org/10.1007/978-3-030-63322-6_56

Fu, FL., Yu, SC. (2008). Three Layered Thinking Model for Designing Web-Based Educational Games. In: Li, F., Zhao, J., Shih, T.K., Lau, R., Li, Q., McLeod, D. (eds) Advances in Web Based Learning - ICWL 2008. ICWL 2008. Lecture Notes in Computer Science, vol 5145. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85033-5_26

Ma, X. (2023). Introduction to Digital Content. In: Social Influence on Digital Content Contribution and Consumption. Management for Professionals. Springer, Singapore. https://doi.org/10.1007/978-981-99-6737-7_1

Davidoff, J. (2023). Cross-cultural Communication. In: Shamey, R. (eds) Encyclopedia of Color Science and Technology. Springer, Cham. https://doi.org/10.1007/978-3-030-89862-5_408

Roux, L., Nodenot, T., Etcheverry, P., Dagorret, P., Marquesuzaa, C., Lopisteguy, P. (2023). A Learner’s Behavior Model for an E-Learning Hybrid Recommender System. In: Ifenthaler, D., Sampson, D.G., Isaías, P. (eds) Open and Inclusive Educational Practice in the Digital World. Cognition and Exploratory Learning in the Digital Age. Springer, Cham. https://doi.org/10.1007/978-3-031-18512-0_5

Hai-Jew, S. (2019). Alpha Testing, Beta Testing, and Customized Testing. In: Designing Instruction For Open Sharing. Springer, Cham. https://doi.org/10.1007/978-3-030-02713-1_9

Zechner, M., Green, R. (2012). An Android Game Development Framework. In: Beginning Android Games. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4302-4678-7_5

Slootman, M. (2018). A Mixed-Methods Approach. In: Ethnic Identity, Social Mobility and the Role of Soulmates. IMISCOE Research Series. Springer, Cham. https://doi.org/10.1007/978-3-319-99596-0_3

Drew, M.R., Falcone, B., Baccus, W.L. (2018). What Does the System Usability Scale (SUS) Measure?. In: Marcus, A., Wang, W. (eds) Design, User Experience, and Usability: Theory and Practice. DUXU 2018. Lecture Notes in Computer Science(), vol 10918. Springer, Cham. https://doi.org/10.1007/978-3-319-91797-9_25

Xu, N., Lin, W., Lu, X., Wei, Y. (2024). Tracking. In: Video Object Tracking. Synthesis Lectures on Computer Vision. Springer, Cham. https://doi.org/10.1007/978-3-031-44660-3_2

A. Ashrafi, V. S. Mokhnachev and A. E. Harlamenkov, "Improving Sign Language Recognition with Machine Learning and Artificial Intelligence," 2024 6th International Youth Conference on Radio Electronics, Electrical and Power Engineering (REEPE), Moscow, Russian Federation, 2024, pp. 1-6, doi: 10.1109/REEPE60449.2024.10479844.

Ashrafi, A., Mokhnachev, V.S. (2023) Designing User-Friendly Interfaces For A Multilingual Sign Language Dictionary, Artificial Intelligence in Automated Control and Data Processing Systems, Collection of Articles of the 2nd All-Russian Scientific Conference: In 5 Volumes, April 27–28, Moscow.

Oellermann, W.L. (2001). Web Services Architecture. In: Architecting Web Services. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4302-1140-2_2

Weber, G. (2012). Adaptive Learning Systems. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_534

A. Ashrafi, V. S. Mokhnachev, Y. N. Philippovich, and L. P. Tsilenko, “Video classification using CNN-LSTM architecture for Bengali sign language” in Fundamental and applied sciences today XXVIII, p. 140-146. Pothi.com, Bengaluru, Karnataka, India (2022).

A. Ashrafi, V. S. Mokhnachev, Y. N. Philippovich, A. Y. Harlamenkov, and L. P. Tsilenko, “Comparative Study Of Different Sign Language Recognition Methods”, Natrazvitie, St. Petersburg, 2022, pp. 28-31.

Ashrafi, A., Mokhnachev, V., Philippovich, Y., Harlamenkov, A., Chernenko, S. (2023). Russian Sign Language Recognition Using MediaPipe. In: Dolinina, O., et al. Artificial Intelligence in Models, Methods and Applications. AIES 2022. Studies in Systems, Decision and Control, vol 457. Springer, Cham. https://doi.org/10.1007/978-3-031-22938-1_21

Napier, J., Leeson, L. (2016). Learning and Teaching Sign Languages. In: Sign Language in Action. Research and Practice in Applied Linguistics. Palgrave Macmillan, London. https://doi.org/10.1057/9781137309778_4

Ashrafi, A., Mokhnachev, V.S., Philippovich, Y.N. Converting Video into Animation Using Video Tracking Technology for Developing a Multilingual Dictionary of Sign Language. IIASU'24 3rd All-Russian scientific conference "Artificial Intelligence in Automated Control Systems and Data Processing", within the framework of the congress "Russian Engineer" at Bauman Moscow State Technical University, 2024.

Miller, O., Averbuch, A. & Navon, E. Tracking of Moving Objects in Video Through Invariant Features in Their Graph Representation. J Image Video Proc 2008, 328052 (2008). https://doi.org/10.1155/2008/328052

Ashrafi A., Mokhnachev, V.S. Conversion Of Indian Sign Language Into Text Using Gesture Recognition Method. Theory and Practice of Modern Science: the View of Youth. Materialy Vserossijskoj nauchno-prakticheskoj konferencii na anglijskom jazyke. Nauchnoe izdanie. Sost. E.N. Lashina, M.S. Lipatov. Pod obshhej redakciej V.V. Kirillovoj . Sankt-Peterburg, 2022. S. 204-211.

(2008). Client–Server Architecture. In: Furht, B. (eds) Encyclopedia of Multimedia. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-78414-4_187

Ashrafi, Arifa, and Yuriy Philippovich. "Creating the First Bengali-Russian Sign Language Dictionary for Inclusive Multilingual Communication." International Journal of Open Information Technologies, vol. 13, no. 7, 2025


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИТ конгресс СНЭ

ISSN: 2307-8162