HR4RS stamp
Nora Fernandez Fernandez
The High Performance in Materials Science and Engineering (CIM) research group has developed the project entitled 'Detection and decoding of labiofacial language based on high-sensitivity sensors and artificial intelligence'.
Under the name SENLAB-IA, this project consists of the development of a system for detection and decoding of spoken language and face, through sensors made of polymers with nanoparticles. The final objective is the development of a prototype, type face mask, that allows to integrate the complete system and achieve a translation of the spoken language, through some type of App.
As Alejandro Ureña, principal investigator of the project and professor of Materials Science and Metallurgical Engineering at the URJC, explains, “we have developed these sensors to detect the movements that occur in and around the face when speaking. The proposal also includes Artificial Intelligence tools to decode these signals and transform them into a spoken or written language”.
The sensors developed are characterized by their simple design, low cost and contact. The CIM group is an expert in these materials and has previously applied them in the detection of human movement or in the rhythm of respiration through the smart masks.
The project also has the participation of the Inclusive Education, People with Disabilities and Accessibility (DIVERSIA) group, led by Professor Ricardo Moreno Rodríguez, who is responsible for designing the test protocol to detect the signals collected by the sensors, which respond to labiofacial movements associated with vowels, phonemes, words, phrases... In addition, "several undergraduate and master's students will participate, including those from the URJC's Degree in Spanish Sign Language and the Deaf Community and end users of the applied technology," he comments. Urena.
One of the potential beneficiaries of this technology would be hearing impaired children. “95% of deaf children are born into hearing families, so their mother tongue is the spoken language of their environment. For them, lip reading becomes an essential support tool for their communication, with the help of hearing aids or cochlear implants. This system can become a transitional element for those families that progressively approach sign language as a priority language”, explains the main researcher of the project.
Regarding the use of Artificial Intelligence, the URJC Biomedical Engineering and Data Science (BigMed+) research group, coordinated by Professor José Luis Rojo Álvarez, will be responsible for developing an automatic learning model (Machine Learning). that allows the translation of the movements detected by the sensors into oral or written language.
The tool that would be developed in this project would favor the social and economic integration of people with hearing disabilities, with accessibility problems in many environments, such as training, leisure, transport, work, etc. All this is possible, as Ureña says, "by making available adapted technologies and support devices that would improve their access to communication, correcting the deficiencies they have in the provision of services, and even helping to eliminate social discrimination." that they experience in their day to day.
In this VII Call for Grants for Research Projects, a total of 27 projects have competed. Apart from this, developed by the URJC, the Polytechnic University of Cartagena and the University of Alicante have also obtained aid for their respective projects.