EBU Technology & Innovation – Rai Virtual LIS: an avatar for sign language applications

Mauro Rossini, Carmen Marino and Andrea del Principe, Radiotelevisione Italiana

The Rai Virtual LIS platform enables the production of Italian Sign Language (LIS) content using avatars to create new opportunities and services for deaf people in contexts where LIS human interpreters are not provided .

The platform is based on a real-time graphics engine while the avatars and animations are developed using the latest 3D computer graphics technologies. The results obtained by the platform in terms of fluidity and intelligibility have been positively evaluated by many deaf experts and signatories.

Using avatars in this way opens up a whole range of new services for deaf people: descriptions in sign language of monuments, works of art, architectural contexts, cultural and tourist information; the translation into sign language, through a virtual actor, of educational content, conferences, training courses and books; providing information to travelers at stations and airports; and on-demand translation services for content which, due to subject specificity and lexical complexity, may be difficult to understand.

New modules

Over the past year, the Rai Virtual LIS platform has been enhanced with two software modules: Education and Weather forecast.

Education is a web application that allows students to learn sign language and generate new sign language content through virtual actors. The application is divided into two sections: the “Dictionary” section allows you to view all the LIS signs available in the database, animated by the avatar, while the “Translation” section is dedicated to the generation of new content in sign language by displaying the Italian text and the corresponding sequence of LIS signs. Currently, around 1000 signs have been digitized and the body of signs can be gradually extended.

The Weather forecast The app downloads the weather forecast from the national Rai teletext service daily and performs an automatic sign language translation. The output of the system is an Italian sign language video. The translation is based on a deterministic algorithm invented and patented by the Rai Center for Research, Technological Innovation and Experimentation (CRITS).

Social advantages

The Rai Virtual LIS platform aims to extend the use of Italian Sign Language for better social inclusion of deaf people in cultural and didactic contexts, to provide free tools to experiment with new forms of communication and to facilitate relationships between children, including those who communicate. difficulties.

The platform is currently based on Italian Sign Language but it can be extended to include other sign languages ​​to create an international sign language platform.

In accordance with its public service mandate, Rai plans to make the Virtual LIS platform available free of charge for deaf communities, associations, museums, schools, universities, public and private institutions wishing to provide content and services. non-profit sign language for deaf people. .

This article first appeared in issue 49 of tech-i magazine.