Bidirectional Braille-speech Communication System for Deafblind Students


  • Ana Paula Pérez-Aguirre School of Engineering and Sciences, Tecnologico de Monterrey
  • Iván Arturo Morales-Pérez School of Engineering and Sciences, Tecnologico de Monterrey
  • Jorge Allan Gómez-Mercado School of Engineering and Sciences, Tecnologico de Monterrey
  • Rodrigo Alberto Gutiérrez-Martínez School of Engineering and Sciences, Tecnologico de Monterrey
  • Iván Matehuala-Moran School of Engineering and Sciences, Tecnologico de Monterrey
  • Rubén Fuentes-Alvarez School of Engineering and Sciences, Tecnologico de Monterrey



Accessibility, Assistive Technology, Braille, Convolutional Neural Networks, Deafblindness, Education


Deafblindness is a type of dual disability wherein both visual and auditory capabilities are significantly impaired. Special communication methods have been developed for the deafblind community, yet these methods require that both people involved have prior knowledge and training to successfully communicate; limiting deafblind people’s social interactions, particularly in academic settings. This paper describes the development of a device that enables two-way communication between a severe to profoundly deafblind user and a hearing person with no prior knowledge of Braille and no additional intermediaries. A Convolutional Neural Network (CNN) scheme for speech recognition was designed and implemented along with the development of an algorithm capable of developing both text-to-speech and Finger-Braille-to-text conversion. Lastly, a system integration via 3D modelling and additive manufacturing was carried out to deliver a functional prototype. The resulting device aims to allow deafblind students to send and receive information entirely in finger Braille, using buttons and vibrotactile feedback, while the hearing tutor receives auditory messages and speaks to reply, making the educational experience as familiar as possible for both parties. Users testing the device achieved an average typing accuracy of over 95% and demonstrated understanding of commands transmitted through the device’s components. 


Ask Larsen, F. and Damen, S. (2014) ‘Definitions of deaf-blindness and congenital deaf-blindness’, Research in Developmental Disabilities, Vol. 35, No. 10, pp. 2568–2576.

Ayton, L., Galvin, K., Johansen, L., O’Hare, F. and Shepard, E. (2023) ‘Awareness of Usher syndrome and the need for multidisciplinary care: A cross-occupational survey of Allied Health Clinicians’, Journal of Multidisciplinary Healthcare, Vol. 16, pp. 1927–1936.

Banerjee, K., Prasad, V., Gupta, R., Vyat, K., Anushree H. and Mishra, B. (2021) ‘Exploring alternatives to Softmax function’, Proceedings of the 2nd International Conference on Deep Learning Theory and Applications, (DeLTA, 2021), Paris, Vol. 1, pp. 81–86.

Blenkhorn, P. (1995) ‘A system for converting braille into print’, IEEE Transactions on Rehabilitation Engineering, Vol. 3, No. 2, pp. 215–221.

Bowen, S.K. and Probst, K.M. (2023) ‘Deaf and hard of hearing students with disabilities: An evolving landscape’, Education Sciences, Vol. 13, No. 7, pp. 752–767.

Castiglione, A. and Möller, C. (2022) ‘Usher syndrome’, Audiology Research, Vol. 12, No. 1, pp. 42–65.

CDHCM (2020) It is necessary to consider measures and strategies that include deaf-blind people in the new normal | CDHCM, [Online], Available: [19 Feb 2024].

CONADIS (2018) Deaf-blindness in México: Facts for June the 27th, International Deaf-blindness Day | Government of Mexico, [Online], Available at: [9 Jul 2023].

CONAPRED (2008) ‘Una Esperanza Para Ángel’ [Hope for Ángel] | National Council for Preventing Discrimination, [Online], Available: [10 Jul 2023].

Ding, X. (ed.) (2012) Advances in Character Recognition: Finger braille recognition system, Beijing: IntechOpen.

Dogbe, D.S.Q. and Anku, F.K. (2024) ‘Prospects and challenges of educating a deaf-blind student in a university in Ghana’, European Journal of Education Studies, Vol. 11, No. 1, pp. 118–134.

Dongare, A., Kharde, R. and Kachare, A. (2012) ‘Introduction to artificial neural network’, International Journal of Engineering and Innovative Technology, Vol. 2, No. 1, pp. 189–194.

He, J., Li, H., Xu, J. and Zheng, C. (2018) ‘Relu deep neural networks and linear finite elements’, Journal of Computational Mathematics, Vol. 38, No. 3, pp. 502–527.

Hersh, M. (2013) ‘Deaf-blind people, Communication, independence, and isolation’, Journal of Deaf Studies and Deaf Education, Vol. 18, No. 4, pp. 446–463.

INEGI (2020) [Population with disability, with limitations for daily activity and some mental problem or condition, by federated state and quinquennial age groups according to sex | Interactive-generic tabulations, [Online], Available: [9 Jul 2023].

Kassem, K., Caramazza, P., Mitchell, K., Miller, M., Emadi, A. and Faccio, D. (2022) ‘Real-time scene monitoring for deaf-blind people’, Sensors, Vol. 22, No. 19, p. 7136.

Kingma, D.P. and Ba, J. (2015) ‘ADAM: A method for stochastic optimization’, Proceedings of the 3rd International Conference for Learning Representations (ICLR, 2015), San Diego.

Li, J., Cheng, J.-H., Shi, J.-Y. and Huang, F. (2012) ‘Brief introduction of back propagation (bp) neural network algorithm and its improvement’, Advances in Computer Science and Information Engineering, Vol. 169, pp. 553–558.

Liu, W., Yu, W., Li, K., Zhou, S., Wang, Q. and Yu, H. (2023) ‘Enhancing blind-dumb assistance through a self-powered tactile sensor-based braille typing system’, Nano Energy, Vol. 116, p. 108795.

Manga, T. and Masuku, K.P. (2020) ‘Challenges of teaching the deaf-blind learner in an education setting in Johannesburg: Experiences of educators and assistant educators’, South African Journal of Communication Disorders, Vol. 67, No. 1.

Mazouch, P., Ptáčková, V., Fischer, J. and Hulík, V. (2018) ‘Students who have unsuccessfully studied in the past – analysis of causes’, Journal on Efficiency and Responsibility in Education and Science, Vol. 11, No. 3, pp. 65–71. https://dx.doi:10.7160/eriesj.2018.110303

McFee, B., McVicar, M., Raffel, C., Liang, D., Nieto, O., Moore, J., Ellis, D., Repetto, D., Viktorin, P., Santos, J.F. and Holovaty, A. (2015) ‘Librosa: v0.4.0.’, Zenodo.

McKinney, E. (2022) ‘Meeting the Needs of a Student with Deaf-blindness in the Academic Classroom: A Model That Works’, Visual Impairment and Deaf-blind Education Quarterly, Vol. 67, No. 4, pp. 17–43.

Montgomery, D.C. (1991) Introduction to Statistical Quality Control, 2nd Edition, New York: John Wiley and Sons.

Moore, C. and Murray, I. (2001) ‘An electronic design of a low-cost braille typewriter’, Proceedings of The Seventh Australian and New Zealand Intelligent Information Systems Conference (ANZIIS, 2001), Adelaide, pp. 153–157.

Morgan, D.S., Carhart, D.A., Dmitriyev, S., Gunderson, B.J., Patadia, S., Pulik, L. and Yanz, L.A. (2011) Braille Writer: US Patent 20090274505A1, [Online], Available: [16 Sep 2023].

NCDB (2013) State Deaf-Blind projects, State Deaf-Blind Projects | National Center on Deaf-Blindness, [Online], Available: [11 Jul 2023].

NFB (2021) Deaf-blind Resources, Deaf-blind Resources | National Federation of the Blind, [Online], Available: [11 Jul 2023].

Olayi, J.E., Igba, I.U. and Ewa, J.A. (2023) ‘Communication strategies for teaching deaf-blind learners in an inclusive classroom’, Special needs education from the lens of interdisciplinary dialogue: A Festschrift in honour of Prof. Emeka D. Ozoji, Vol. 1, No. 1, pp. 269–278.

Riccobono, D. and Morrow, S.M. (2022) ‘State Deaf-blind Projects: National Survey Results’, Visual Impairment and Deaf-blind Education Quarterly, Vol. 67, No. 4, pp. 109–125.

Rodríguez-Jiménez, M., Pérez-Jorge, D., Puerta-Araña, I. and Ariño-Mateo, E. (2022) ‘Quality of life in Deaf-blind people and its effect on the processes of educational adaptation and social inclusion in Canary Islands, Spain’, Education Sciences, Vol. 12, No. 7, p. 490.

Rusiecki, A. (2019), ‘Trimmed categorical cross-entropy for deep learning with label noise’, Electronic Letters, Vol. 55, No. 6, pp. 319–320

Schulz, S., Harzheim, L., Hübner, C., Lorke, M., Jünger, S. and Woopen, C. (2023) ‘Patient-centered empirical research on ethically relevant psychosocial and cultural aspects of cochlear, glaucoma and cardiovascular implants – a scoping review’, BMC Medical Ethics, Vol. 24, No. 1.

Smolen, E.R. and Paul, P.V. (2023) ‘Perspectives on identity and d/deaf and hard-of-hearing students’, Education Sciences, Vol. 13, No. 8, p. 782.

Traer, J. and McDermott, J.H. (2016) Statistics of natural reverberation enable perceptual separation of sound and space | IR Survey (Traer and McDermott), [Online], Available: [17 Sep 2023].

Turunen-Taheri, S., Hagerman, A., Hellström, S., Skjönsberg, Å. and Backenroth, G. (2023) ‘Combined severe-to-profound hearing and vision impairment—experiences of daily life and need of support, an interview study’, PLOS ONE, Vol. 18, No. 6, p. e0280709.

Vincent, C., Wittich, W., Bergeron, F., Hotton, M. and Achou, B. (2021) ‘Shopping when you are Deaf-blind: A pre-technology test of new methods for face-to-face communication—deaf-blindness and face-to-face communication’, Societies, Vol. 11, No. 4, p. 131.

WFDB (2024) World Federation of the Deaf-blind | Personas con sordoceguera y educación, [Online], Available: [19 February 2024].

WFDB (2023) Second global report on the situation of persons with deaf-blindness | World Federation of the Deaf-blind, [Online], Available: [19 Feb 2024].

Zdravkova, K., Krasniqi, V., Dalipi, F. and Ferati, M. (2022) ‘Cutting-edge communication and learning assistive technologies for disabled children: An artificial intelligence perspective’, Frontiers in Artificial Intelligence, Vol. 5, p. 970430.

Additional Files



How to Cite

Pérez-Aguirre, A. P., Morales-Pérez, I. A., Gómez-Mercado, J. A., Gutiérrez-Martínez, R. A., Matehuala-Moran, I. and Fuentes-Alvarez, R. (2024) ’Bidirectional Braille-speech Communication System for Deafblind Students’, Journal on Efficiency and Responsibility in Education and Science, vol. 17, no. 2, pp. 164–177.



Research Paper