Period: 1/1/2019 – 31/12/2021
Research project supported by the Spanish Ministry of Science under reference no. RTI2018-094879-B-I00.
Open Education (OE) is being fostered by governments and international organizations as the best way of modernizing education. The relevance of Open Educational Resources (OER) in OE is stated in the Paris OER Declaration 2012 and in the Ljubljana OER Action Plan 2017. The development of multilingual OER appears as an important social challenge to promote a wider access to education at all levels, contributing to social inclusion and allowing inter-cultural knowledge sharing.
Similarly, Parliamentary Openness (PO) is also being promoted by parliamentary monitoring organizations (PMO) in order to increase the openness of parliamentary work and enhancing citizen participation. The Declaration on Parliamentary Openness (2012) also highlights the relevance of multilingualism in PO for countries with more than one official language.
Universities worldwide are beginning to see benefits in producing OER as recording lectures of classrooms to support teaching and learning activities. These recordings are typically published in online learning platforms and they are becoming even more popular among students. In the same way, parliaments are using even more video as a powerful way to bring the attention of society towards the work of parliaments. In particular, they are increasingly making live streams and recordings available allowing to search for parts of plenary sessions or speeches. The Spanish Law 19/2013 on Transparency establishes legal obligations for the Public Administrations to enable easy access to information in websites using comprehensible and reusable formats and in accordance with the principle of universal accessibility. As a consequence, Transparency Portals have been created as a meeting and dissemination point. Based on this evidence, the development of multilingual video content appears as the key enabler to dramatically encourage OE and PO. However, multilingual subtitles in these video recordings are scarce and they are generally produced manually using non-scalable solutions.
The main aim of the project is to further improve the state of the art in Automatic Speech Recognition (ASR) and Statistical Machine Translation (SMT) to deal with the kind of audiovisual collections we are considering. Thus, different open research challenges in ASR and SMT will be addressed throughout the project: i) channel variability, noise and reverberation; ii) far-field speech recognition; iii) speaker diarization; iv) multispeaker recognition; v) on-line speech recognition; vi) neural machine translation; vii) translation of sentences containing ASR errors.
The technology developed will be tested in two use cases. On the one hand, a repository of classroom video recordings called “Videoapunts” developed at the Universitat Politècnica de València. The classes are recorded using microphone arrays and the open source Opencast platform which supports the management of educational audio and video content. On the other hand, transparency portals of public administrations will be considered to create a a repository of video recordings of plenary sessions.
The project will have a dramatic impact in OE and PO, overcoming communication barriers and enabling an uninhibited exchange of ideas across languages and cultures. The technologies developed will bring important advances in the hot topic research fields of ASR and SMT. The project is also aligned with market and user needs, providing plausible scenarios for products or services in which research results can be applied beyond the educational and parliamentary contexts.
Project goals
In accordance with the main purpose of the project, the following project objectives have been defined:
- To provide high-quality automatic transcriptions of classroom and plenary session video recordings.
- To provide high-quality automatic translations of classroom and plenary session video recordings.
- To provide multilingual on-line ASR of classroom and plenary live streams.
- To integrate the technology developed into Opencast and Transparency Portal platforms to enable real-life impact.