License

Licensed under the Apache License, Version 2.0.

The transLectures-UPV Platform (TLP) includes software developed at the Universitat Politècnica de València (UPV) by the MLLP research group as part of the transLectures EU project until TLP Version 1.0.1.

1. Introduction

The transLectures-UPV Platform (TLP) consists of a set of software tools that enable the integration of advanced Automatic Speech Recognition (ASR), Machine Translation (MT) and Text-To-Speech Systhesis (TTS) technologies into large video repositories and Massive Open Online Courses (MOOCs) platforms. These technologies are used to generate automatic multilingual subtitles and (synthesized) audiotracks of audio and video files, as well as to generate automatic translations of text/HTML documents. TLP also includes web-based tools to view and easily correct transcription and translation errors in order to generate perfect outputs. In addition, TLP also brings support to multilingual web translation using MT technology.

It was developed up to Version 1.0.1 by the MLLP research group from the Universitat Politècnica de València (UPV) as part of the EU research project transLectures. After the end of transLectures, TLP is still being maintained by the MLLP research group, with the release of TLP versions 1.1, 1.2, 2.0, 2.1, 3.0, 3.1, 3.2, 3.3 and 3.4.

2. Getting Started

In this section we will give a brief overview of the transLectures-UPV Platform (TLP), describing the workflows involved when integrating Automatic Speech Recognition (ASR), Machine Translation (MT) and Text-To-Speech Systhesis (TTS) technologies into large media repositories, MOOC platforms and websites with the aid of this platform.

2.1. TLP Overview

TLP is a self-contained piece of software that includes everything that is needed in order to integrate transcription, translation and speech synthesis technologies into large media repositories, MOOC platforms and/or websites.

The main features of TLP are the following:

Easy integration of ASR, MT and TTS technologies: TLP provides a framework via the Ingest Service in which custom ASR, MT and TTS systems can be easily integrated into the platform to provide automatic subtitles, document translations, and synthesised text-to-speech audio tracks.
Exploitation of media-related resources: Topic adaptation techniques for ASR are supported by TLP (i.e. exploitation of text extracted from suplied media slides, related documents, etc.)
Support to grid computing: TLP has been developed expressly to take advantage of computer clusters in order to parallelize at maximum transcription, translation, and text-to-speech processes.
Ergonomic solution for subtitle and translation editing: TLP includes two web-based software tools, the TLP Transcription Editor and the TLP Media Player, that can be used to review transcriptions and subtitles respectively for audio/video objects with ease and comfort; as well as a third web-based tool called the TLP Translation Editor to review text document translations.
Support to collaborative subtitling/translation: TLP enables collaborative video subtitling and document translation by managing edit sessions and keeping track of all user editions made on subtitle and translation files. Changes can be postponed to be reviewed by authors, who can later review these changes to accept or reject them.
Automated postprocess triggered by user interactions: Whenever a transcription or translation is edited, time alignments at the word level for transcriptions, automatic translations, and automatic synthesised audio tracks are re-generated; among other actions. This feature is directly managed by the Postprocessor module.
Support to multilingual web translation: MT technology can be used used as to translate static and dynamic web pages or web sites in a very straightforward manner trough the Web Translation Plugin and the Web Translation Editor.
Extensive and powerful public API: TLP offers a complete API that makes very easy the integration process between TLP and remote video repositories, MOOC platforms, institutions, companies, etc.
Based on open-source tools and libraries: All software tools included in TLP are entirely based on open-source languages and libraries.
Publicly released as open-source software: Successive versions of TLP have been publicly released under the open-source Apache License 2.0.

Its main components are the Database, the Web Service, the Ingest Service, the Transcription Editor, the Media Player, the Translation Editor, the Web Translation Plugin and the Web Translation Editor, each of which are described in their corresponding sections. In addition, TLP features the Postprocessor module to perform automated actions on the basis of user interaction, as well as several Client Tools.

The figure below shows the main components of TLP and a simplification of the interactions between them.

TLP Components

2.2. Use Cases

In this section we show basic use cases of TLP in three different scenarios: on large media repositories, on MOOC platforms, and on on-line websites.

2.2.1. TLP for large media repositories

We have defined three use cases to illustrate the main ways a large media repository and its users can interact with TLP:

A new recording from the media repository is uploaded to TLP for the generation of automatic subtitles and audio tracks.
A user plays a media file with subtitles from the media repository’s website.
A user corrects subtitle errors (transcription or translation).

Use Case 1: A new recording from the media repository is uploaded to TLP for the generation of automatic subtitles and audio tracks

A lecturer/speaker records a new lecture/media in a recording studio, in a classroom, or during a conference. To get this video transcribed and translated into several languages the recorded media file plus other metadata are sent to the TLP Web Service via the /speech/ingest interface.

The TLP Ingest Service reads this upload and launches the required transcription, translation and/or speech synthesis processes. During this stage, the client (in this case, the remote media repository) can check at any time the progress of the upload using the /speech/status endpoint of the Web Service.

Finally, the Ingest Service creates a new media record in the Database and stores all media, subtitles, and synthesized audiotrack files.

Use Case 2: A user plays a media file with subtitles from the media repository’s website

A user browses the media repository’s catalogue and selects the media he or she wants to watch/listen using the repository’s media player. The user can watch the selected media with subtitles in different languages, or even listen to it in another language using automatically synthesized audio tracks where available.

To get the list of all subtitle languages available, the repository’s media player sends a request to the /speech/langs interface of the TLP’s Web Service, displaying to the user the language availability.

As the user selects the desired subtitle language, the repository’s media player calls the /speech/get endpoint to download the corresponding subtitle file in the required format (srt, vtt, dfxp, etc.), which is immediately processed and displayed in the media player.

A similar procedure is followed when the user requests a synthesized audio track, but in this case the media player makes use of the /speech/audiotrack interface.

Use Case 3: A user or lecturer corrects subtitle errors (transcription or translation)

A user (or a lecturer), while playing a media file with subtitles (as shown above in use case 2), notices that the displayed subtitles contain some errors and decides to correct them. The repository’s media player redirects somehow the user to the TLP Media Player, that offers an ergonomic and efficient interface for subtitle editing.

Any corrections made by the user are sent back to the Web Service via the /speech/mod interface and appended to the subtitles file, or left for revision, depending on the user’s confidence. All subtitle edits are kept so that at any time a user with special permissions can browse the edition history and recover previous editions.

In case the subtitle file is updated, automatic translations and synthesized audio tracks are automatically re-generated using user corrections.

2.2.2. TLP for MOOC platforms

We have defined three use cases to illustrate the main ways a MOOC platform can interact with TLP:

A new MOOC is added to the platform, and automatic subtitles for videos as well as automatic translations for text contents are requested.
MOOC teachers review video subtitles and translations of text contents.
Video subtitles and translated text contents are imported into the MOOC platform.

Use Case 1: A new MOOC is added to the platform, and automatic subtitles for videos as well as automatic translations for text contents are requested.

A MOOC teacher or manager adds a new MOOC to the platform’s catalogue, and requests to translate all MOOC contents into several languages. Automatically, the MOOC Platform identifies all video and text contents and uploads them to the TLP server using the /speech/ingest and /text/ingest API interfaces respectively from the Web Service.

The Ingest Service reads these uploads and launches the required transcription, translation and/or speech synthesis processes. During this stage, the client (in this case, the remote media repository) can check at any time the progress of the upload using the /speech/status and /text/status endpoints.

Finally, the Ingest Service creates a new media record in the Database and stores all media, subtitles, synthesized audiotrack, documents and translations.

Use Case 2: MOOC teachers review video subtitles and translations of text contents.

After a successful generation of automatic subtitles and translations, the MOOC teacher wants to review these outputs, in order to amend possible transcription and/or translation errors.

The MOOC platform allows the teacher to use the TLP Media Player to review video subtitles and the TLP Translation Editor to amend translation errors of text contents. This can be done by embedding them in the MOOC platform or by redirecting the teacher to a specific URL.

Any corrections made by the teacher are sent back to the Web Service via the /speech/mod or /text/mod interfaces. All editions are kept so that at any time a user with special permissions can browse the edition history and recover previous editions. In case a subtitle or translations file is updated, automatic translations and synthesized audio tracks, when applicable, are automatically re-generated by the Ingest Service using user corrections.

It must be noted that TLP brings to MOOC platforms the possibility of setting up a subtitle and translation editing framework under a collaborative scenario in which MOOC participants can carry out the process of revision with or without admin supervision.

Use Case 3: Video subtitles and translated text contents are imported into the MOOC platform.

The MOOC teacher/manager wants to import subtitles and translations for all course contents to the MOOC platform in order to bring them to students. Automatically, the MOOC platform retrieves all video subtitles and translations from text contents using the /speech/get and /text/get API interfaces respectively. Subtitles and translations are stored in the MOOC platform database in order to make them available for the students. This operation could be executed every time any subtitles and/or translations from the MOOC course are modified.

2.2.3. TLP for multilingual web translation

We have defined three use cases to illustrate the main ways a website can interact with TLP for multilingual web translation:

An API client registers a new website into TLP to make it multilingual.
A user enters into the website and requests a translated version.
The website admin decides to correct translation errors.

Use Case 1: An API client registers a new website into TLP to make it multilingual.

An API client wants to make multilingual a new webpage. To do so, first, the client registers the webpage into TLP by using the /web/manage/add API interface of the TLP Web Service. This interface returns a unique website key, used to identify the website in further transactions. Then, the API client integrates the TLP’s Web Translation plugin into the website by just adding a proper <script> HTML5 element in the header of all website’s pages. The <script> tag basically commands to the web browser to call the /web/getjs interface using the website key as parameter to get and load the specific Web Translation plugin generated for that website.

Use Case 2: A user enters into the website and requests a translated version.

A user visits the website using his/her web browser and requests a translated version of the webpage in a given language. First of all, the web browser calls the /web/getjs interface to get and load the specific Web Translation plugin generated for that website. Then, as soon as the webpage contents are rendered by the browser, the plugin parses the whole HTML document, locating all text contents and sending them to the /web/translate interface in order to retrieve their translations into the language requested by the user. The Web Service then returns to the browser the available translations for the given texts, displaying them to the user. It might be that some text contents have not been translated yet, for instance, when a webpage is parsed for the first time by the web translation plugin, or when the original text contents have been modified. In both cases, the untranslated source texts are sent to the Ingest Service to generate their translations. After a while, these translations are stored into the Database in order to make them available in future requests.

Use Case 3: The website’s admin user decides to correct translation errors.

The website’s admin, after visiting a web page (as depicted in use case 2), notices a translation error and decides to amend it. After pressing a specific combination of keys, the Web Translation plugin shows the translation editor interface. Then, the admin user clicks to the corresponding text node, amends the translation, and confirms the correction. The plugin then sends back to the server the corrected text by calling the /web/amend interface. The amended text is finally stored into the Database, making it available in future requests.

Alternatively, admins can review the whole website’s translations anywhere using the embbedable TLP Web Translation Editor.

3. Database

The TLP Database is a SQL-based relational database which stores all the data required for the Web Service and the Ingest Service. The main entities stored in the Database are the following:

Videos/Audios: All the information related to a specific audio or video is stored in the database, including language, duration, title, keywords and category. An external ID, provided by the client is used to identify the media object in all transactions performed between the client and the API.
Speakers: Information about the speaker/lecturer can be used by the ASR system to adapt the underlying models to the unique characteristics of the given speaker and, therefore, improve the quality of the resulting subtitles.
Subtitles: All subtitles automatically generated by the Ingest Service are stored in DFXP format into the database and retrieved by the client via the Web Service.
Audiotracks: As in the case of subtitles, automatically synthesized audio tracks from translated subtitles are also stored in the database.
Documents: Similarly to video and audio entities, text documents are stored as well along with metadata such as document format, sentences, words, language, etc.
Translations: Again, similarly to subtitles objects, translations in the original document format, as well as in an internal XML format (DTLX format), are stored in the database.
Websites: Metadata of every website or webpage to be translated, as well as all source and translated texts.
Uploads: Every time an /ingest operation is performed, a new upload entry is stored in the database to track its progress.

TLP also needs to store data outside the relational database (in the filesystem of the TLP Server). In TLP are defined several data sources, and their root directories or mount points have to be registered in the mount_points table of the database:

Language Processing modules: Root directory of all language processing modules to integrated into the Ingest Service (systems key).
Video/audio objects:
- Media files: Root directory of media files, such as video, audio, slides, thumbnail files (media key).
- Subtitle files: Root directory of subtitle files (transcriptions key).
- Uploads: Root directory for the uploaded objects and temporal files (uploads key).
- Alignments: Root directory for the temporal files of text-to-speech alignment processes (valign_uploads key).
Text objects:
- Documents: Root directory of all text document files (text_documents key).
- Uploads: Root directory for the uploaded objects and temporal files (text_uploads key).
Web objects:
- Uploads: Root directory for temporal files (web_uploads key).

4. Web Service

The Web Service is the API interface for exchanging information and data between the client and the transLectures-UPV Platform. It also enables the viewing and editing capabilities of the TLP Media Player and the TLP Translation Editor, as well as the website translation features. The Web Service defines a wide set of API HTTP interfaces to allow for the full integration between TLP and the remote media repository, MOOC Platform, or website.

The Web Service implements thre sub-APIs for the three main classes of objects than can be processed by TLP:

/speech sub-API: is aimed to deal with video and audio files.
/text sub-API: is intended to deal with text documents and contents.
/web sub-API: is intended to specifically deal with text contents extracted from html documents from the client (web browser) side.

/speech and /text sub-APIs feature very similar interfaces and specifications.

The Web Service requires an specific scheme for user authentication, please refer to API User Authentication for more information.

In addition, TLP offers several tools and libraries to interact with this API; you will find more information about them in the Client Tools Section.

If you decide to develop you own API client, we recommend you to read our API Technical Preface. Also, a detailed description of all API interfaces can be found in here for the /speech sub-API, here for the /text sub-API, and here for the /web sub-API.

4.1. /speech: API Interfaces for audio/video transcription and translation

Please note that all the following interfaces will share a common prefix: /speech.

Interfaces for media upload and management:

/ingest	Upload media (audio/video) files and any attachments and metadata to the TLP Server for automatic multilingual subtitling and speech synthesis.
/uploadslist	Get a list of all the user’s uploads.
/status	Check the current status of a specific upload ID.
/systems	Get a list of all available Speech Recognition, Machine Translation, and Text-To-Speech Systems that can be applied to transcribe, translate, and synthesize a media file.
/list	Get a list of all existing audio and videos in the Database.

Interfaces for downloading media and subtitle files:

/metadata	Get metadata and media file locations for a given media ID.
/langs	Get a list of all subtitle and audiotrack languages available for a given media ID.
/get	Download the current subtitle file for a given media ID and language.
/audiotrack	Download an audiotrack file for a given media ID and language.

Interfaces for editing subtitles:

/start_session	Starts an edition session to send and commit modifications of a subtitles file.
/session_status	Returns the current status of the given session ID.
/mod	Send and commit subtitle corrections under an edit session.
/end_session	Ends an open edition session, and depending on the confidence of the user, editions are directly stored in the corresponding subtitles files or left for revision.

Interfaces for managing subtitles' user revisions:

/lock	Allow/disallow regular users to send subtitles modifications for an specific Media ID.
/edit_history	Returns a list of all edit sessions that involved an specific media ID.
/revisions	Returns a list of all edit sessions for all API user’s media files that are pending to be revised.
/mark_revised	Mark/unmark as revised an specific edit session ID, typically from another Session ID on the TLP Media Player.
/accept	Accept modifications of one or more pending edit sessions without having to revise them. Modifications are commited into the corresponding subtitles files.
/reject	Reject modifications of one or more pending edit sessions without having to revise them.

4.2. /text: API Interfaces for text/document translation

Please note that all the following interfaces will share a common prefix: /text.

Interfaces for document upload and management:

/ingest	Upload document (text/document) files and metadata to the TLP Server to be automatically translated.
/uploadslist	Get a list of all the user’s uploads.
/status	Check the current status of a specific upload ID.
/systems	Get a list of all available Machine Translation systems that can be applied to translate a document file.
/list	Get a list of all existing documents in the Database.

Interfaces for downloading translated document files:

/metadata	Get metadata for a given document ID.
/langs	Get a list of all language translations available for a given document ID.
/get	Download the latest version of the translated document file for a given document ID and language.

Interfaces for editing translations:

/start_session	Starts an edition session to send and commit modifications of a translation file.
/session_status	Returns the current status of the given session ID.
/mod	Send and commit translation corrections under an edit session.
/end_session	Ends an open edition session, and depending on the confidence of the user, editions are directly stored in the corresponding translation files or left for revision.

Interfaces for managing translations' user revisions:

/lock	Allow/disallow regular users to send modifications for an specific document ID.
/edit_history	Returns a list of all edit sessions that involved an specific document ID.
/revisions	Returns a list of all edit sessions for all API user’s document files that are pending to be revised.
/mark_revised	Mark/unmark as revised an specific edit session ID, typically from another Session ID on the TLP Translation Editor.
/accept	Accept modifications of one or more pending edit sessions without having to revise them. Modifications are commited into the corresponding translation files.
/reject	Reject modifications of one or more pending edit sessions without having to revise them.

4.3. /web: API Interfaces for multilingual web translation

Please note that all the following interfaces will share a common prefix: /web.

Interfaces to be used by the API client:

/manage	Features four sub-interfaces to list, add, edit and delete websites.
/systems	Get a list of all available Machine Translation systems that can be applied to translate a document file.

Interfaces to be used by the client web browser:

/getjs	Returns a JavaScript plugin that enables web translation on web browsers.
/translate	Requests translations of a given set of texts, sending part or all of them to translate if no translations are available yet.
/amend	Send user corrections of translated texts.

Interfaces to be used by the TLP Web Translation Editor:

/editor

Features three sub-interfaces: /langs, /get and /mod.

4.4. API User Authentication

The Web Service implements a custom API user authentication system based on authentication tokens. Every API call (except for those of the /web sub-API to be used by web browsers) must include a valid authentication token in order to authenticate the API user. TLP offers two different authentication methods:

Secret Key: An API user authentication token, associated to the user account, is provided to the Web Service. This token is valid for user authentication on all API interfaces. This is the recommended authentication method for direct client to server API calls.
Request Key: A lifetime-limited request-dependent authentication token is provided to the Web Service. This token is valid for user authentication only on a reduced set of API interfaces and for a limited period of time. This authentication method should be used in case the use of the secret key as an authentication token could be exposed or revealed to third-parties, for instance when a user belonging to the API client organisation is using the TLP Media Player or the TLP Translation Editor to edit a subtitle file or translation file, respectively.

The figure above shows a typical integration scenario between TLP and the remote media repository, in which the Secret Key authentication method is used for all direct API calls between both parts, whilst the alternative Request Key method is used to generate TLP Media Player URLs that will be followed by the repository’s users to review media subtitles. In this latter case, the Request Key is the authentication token used in all API calls between the Player and the Web Service.

For further information and technical details, please refer to the Preface of the Web Service’s API Documentation.

5. Transcription Editor

The TLP Transcription Editor is a web-based tool which allows users to review the transcription of a video/audio file, without having to deal with time alignments. Changes committed by the user are automatically aligned with the speech signal, so that afterwards, in a second step, the user can review and adjust media subtitles with the Media Player, if necessary.

The Transcription editor can be called externally/embedded using a valid URL. For further technical information, please refer to Calling the TLP Transcription Editor.

6. Media Player

The TLP Media Player is an HTML5 media player which allows users to review and modify media subtitles of different languages with ease. The Player provides a highly ergonomic editing interface, optimized to reduce user effort.

The TLP Media Player can be called externally/embedded using a valid URL. For further technical information, please refer to the Calling the TLP Media Player Annex.

7. Translation Editor

The TLP Translation Editor is an HTML5 software which allows users to review and modify document translations with ease. The Editor provides a highly ergonomic editing interface, optimized to reduce user effort.

The TLP Translation Editor can be called externally/embedded using a valid URL. For further technical information, please refer to the Calling the TLP Translation Editor Annex.

8. Web Translation Plugin

TLP features a web translation plugin that allows API users to automatically translate their websites directly from the web browser, that is, from the client side.

To integrate the TLP’s web translation plugin into a website:

Include the TLP’s Javascript plugin in the website header using the /getjs interface. I.e:

 <header>
   <script src="http://my-tlp-server.com/api/web/getjs?key=MY_WEBSITE_KEY"></script>
 </header>

Translated contents for any website page will be accessible by adding #lang=LANG_CODE (i.e. #lang=es) to the URL.
Pressing Ctrl+Alt+E will enable the translation editor to amend automatic translations. The plugin will request the API user’s password to grant such permission.

Alternatively, admins can review the whole website’s translations anywhere using the embbedable TLP Web Translation Editor.

9. Web Translation Editor

The TLP Web Translation Editor is an HTML5 software which allows users to review and modify all translations for a single website with ease. It can be used as an alternative to the Web Translation Plugin. The main differences to the Web Translation Plugin are:

All text translations are accessed at once, avoiding the navigation through the translated website.
Can be integrated as an external tool. This for example can be used by web masters to enable other users to review and modify the translations.

The TLP Web Translation Editor can be called externally/embedded using a valid URL. For further technical information, please refer to the Calling the TLP Web Translation Editor Annex.

10. Ingest Service

The Ingest Service is the service devoted to process video/audio and text/document files uploaded via the /speech/ingest and the /text/ingest interfaces of the Web Service, respectively. It also carries out the process of text-to-speech alignment to recover time aligments at the word level for manually reviewed transcripts.

The Ingest Service checks periodically (typically every minute) whether new uploads are awaiting to be processed, also checking if the ongoing uploads are progressing correctly or have failed. The uploads table of the Database is used to keep track of the status of every upload.

Please note that in the case of multilingual web translation, text contents to be translated are automatically gathered and sent to the Ingest Service by the Postprocessor module. Similarly for the case of text-to-speech alignment, the Postprocessor checks whether there exist user interactions that require such processes, requesting them to the Ingest Service.

The figure above shows the internal structure of the Ingest Service, which is split in two layers:

Upper Layer

The Upper Layer implements the main logic of the Ingest Service using a modular design. It has a central node, the Core, which the logic of all possible workflows that can be followed by an upload, leaving data processing tasks to external modules. This means that the functionalities of the Ingest Service can be easily modified, replaced or extended by swapping these external modules with others, e.g., other Automatic Speech Recognition (ASR) and Machine Translation (MT) modules.

Actually, there exist 4 instances of the Ingest Service’s Core:

IS-Align: To manage text-to-speech alignment processes.
IS-Speech: To process video/audio objects.
IS-Text: To translate text objects.
IS-Web: To translate web pages.

External modules can be divided in two categories:

Base Modules: Modules that implement APIs for basic operations used by the Core.
- Media Module: Module that offers several methods of media format conversion.
- Text: Module to process text input data.
- Format Adapters: Module that deals with the TLP’s internal data formats: <X10000, DFXP>> (for media subtitles) and DTLX (for text document translations).
- Mailer Module: Module with routines used to send e-mail notifications regarding upload status updates.
- URL Downloader: Module that allows for the download media files from a given URL address. It also offers the possibility of downloading obfuscated URLs such as YouTube or Vimeo using external plug-ins, called URL decoders.
Language Processing Modules: Modules that integrate text-to-speech alignment, transcription, translation and speech synthesis technologies into the Ingest Service. These modules have to be properly registered in the table systems of the Database.

Please note that propietary modules are not distributed with the public release of TLP, but module templates that you have to implement to integrate your own LP technologies.
- Align Modules: Automatic Text-to-Speech alignment modules, used to recover time alignments at the word level for input texts.
- ASR Modules: Automatic Speech Recognition Modules, used to generate transcription subtitle files.
- MT Modules: Machine Translation Modules, used to generate translated subtitle files, or to translate text documents.
- TTS Modules: Text-To-Speech Modules, used to generate synthesized audiotracks in a specific language.
- Text Retrieval Module: Extracts plain text information from the different file resources included in the MPF. It also downloads related text documents from the web. This text data can be used by ASR Modules to enhance transcription quality by adapting the underlying ASR System to the topic of the media file.

Lower layer

The Lower Layer satisfies all local installation dependencies related to data storage and job scheduling. It is split into two parallel sublayers:

Scheduler layer: Implements an API for launching and scheduling the transcription and translation processes, typically in a Grid Engine/Job Management System.
Storage layer: Implements an API that allows access to the data stored in the Database and in the TLP Server’s hard drive.

10.1. Uploads Workflow

In this section we explain the different steps an upload can follow from the moment it is ingested into the transLectures-UPV Platform until its processing finishes.

10.1.1. Video/Audio transcription and translation workflow

First we must distinguish between four types of operations:

New Media: This operation is requested when a newly-recorded, non-existing media is uploaded to TLP for the first time. In this operation, a new Media object is created in the Database.
Update Media: This operation is requested when updates are applied to an existing media. For instance, new text resources such as slides might be added to the Media Package File (MPF) to improve the automatic transcription and translations of the existing media, or to update the existing media file with a re-recording.
Delete Media: This operation is requested when a media is deleted from the remote repository.
Cancel Upload: This operation is requested to cancel an ongoing upload for whatever reason.

Depending on the type of operation and the input data, the steps an upload follows in the Ingest Service may vary. The figure below illustrates the standard Ingest Service workflow:

Media Package Files are uploaded to the transLectures-UPV Platform via the Web Service's /ingest interface and stored in the Database. The Ingest Service reads the uploads table of the database and starts processing the uploaded MPF. An upload will typically follow the following sequential steps, with some exceptions (some steps might be skipped depending on the input data):

Media Package Processing: The MPF is processed for the first time, performing several security, data integrity and data format checks, and, if all checks are correct, the upload status moves to the next processing step.
Transcription Generation: In this step, a transcription file in DFXP format is generated from the main media file (video, audio) using an Automatic Speech Recognition (ASR) Module.

This step is skipped in the following cases:
1. The Ingest Service does not feature a suitable ASR Module for the source language of the main media file.
2. Subtitles in the source language were provided in the MPF.
3. The client has explicitly not requested this step.
4. In update operations that do not involve re-transcribing the lecture.
5. In delete or cancel operations.
Translation(s) Generation: In this step, one or more translation files in DFXP format are generated from a transcription file (either automatically generated in the previous step, or provided in the MPF), using the appropriate Machine Translation Modules.

This step is skipped in the following cases:
1. The Ingest Service does not have suitable MT Modules for the source language of the main media file.
2. Subtitles in all requested translation languages offered by the Ingest Service are already provided in the MPF.
3. The client has explicitly not requested this step.
4. In update operations that do not involve re-translating the lecture.
5. In delete or cancel operations.
Text-To-Speech Track Generation: In this step one or more synthesized audiotrack files are generated from a translation file (either automatically generated in the previous step or provided in the MPF), using the appropriate Text-To-Speech Modules. This step is skipped in the following cases:
1. The Ingest Service does not have suitable TTS Modules for the target language of any translation files.
2. Audiotracks in all requested languages offered by the Ingest Service are already provided in the MPF.
3. The client has explicitly not requested this step.
4. In update operations that do not involve re-translating the lecture.
5. In delete or cancel operations.
Media Conversion: In this step, the main media file is converted into the media formats required by the TLP Player in order to maximize browser compatibility. This step is skipped in the following cases:
1. All required media files were attached in the MPF.
2. In update operations where the main media file has not changed.
3. In delete or cancel operations.
Store Data: This is the final step. For New Media and Update Media operations, the data contained in the MPF and the data automatically generated by the Ingest Service are stored in the Database. For Delete Media operations, all previously stored media files and data are deleted.

In every execution of the Ingest Service, the Core reviews which uploads are being processed, checking whether the related processes are:

Queued: Processes are queued when they are waiting to be executed. No action is performed.
Running: Processes are being executed. No action is performed.
Finished: All processes finished successfully. The Core changes the upload status to the next processing step.
Failed: Some processes failed. The Core changes the upload status to an error state.

Detailed information about the Ingest Service workflows and behaviour can be found in this Appendix.

10.1.2. Text/Document translation workflow

First we must distinguish between four types of operations:

New Document(s): This operation is requested when one or more new text documents are uploaded to TLP for the first time. In this operation, new Document objects are created in the Database.
Update Document(s): This operation is requested to update existing document files, either the source document or to regenerate automatic translations.
Delete Document(s): This operation is requested to delete existing document files.
Cancel Upload: This operation is requested to cancel an ongoing upload for whatever reason.

Depending on the type of operation and the input data, the steps an upload follows in the Ingest Service may vary. The figure below illustrates the standard Ingest Service workflow:

Media Package Files are uploaded to the transLectures-UPV Platform via the Web Service's /text/ingest interface and stored in the Database. The Ingest Service reads the uploads table of the database and starts processing the uploaded MPF. An upload will typically follow the following sequential steps, with some exceptions (some steps might be skipped depending on the input data):

Media Package Processing: The MPF is processed for the first time, performing several security, data integrity and data format checks, and, if all checks are correct, the upload status moves to the next processing step.
Translation(s) Generation: In this step, one or more translation files in are generated for every text document file using the appropriate Machine Translation Modules.

This step is skipped in the following cases:
1. The Ingest Service does not feature suitable MT Modules for the source language.
2. The client has explicitly not requested this step.
3. In update operations that do not involve re-translating the lecture.
4. In delete or cancel operations.
Store Data: This is the final step. For New Document(s) and Update Document(s) operations, the data contained in the MPF and the data automatically generated by the Ingest Service are stored in the Database. For Delete Document(s) operations, all previously stored document files and data are deleted.

In every execution of the Ingest Service, the Core reviews which uploads are being processed, checking whether the related processes are:

Queued: Processes are queued when they are waiting to be executed. No action is performed.
Running: Processes are being executed. No action is performed.
Finished: All processes finished successfully. The Core changes the upload status to the next processing step.
Failed: Some processes failed. The Core changes the upload status to an error state.

Detailed information about the Ingest Service workflows and behaviour can be found in this Appendix.

10.2. User Quota

Each API client account has an upload quota. This quota represents the remaining number of videos and media time for video objects, or the remaining number of document files, sentences and/or words for text objects, that the user can upload. Once a new media file or document or text contents are uploaded, the Ingest Service checks whether the client has enough quota to process that/these particular objects, updating accordingly the user’s quota after processing them. Automatic re-transcriptions and re-translations do not decrease the user’s quota.

The Ingest Service features a Test Mode that allows the client to perform integration tests without consuming quota, and obtaining fast responses.

11. Postprocessor

The postprocessor is the module devoted to react to different type of user interactions with TLP. Its main tasks are the following:

Close/finish user edit sessions created on the Transcription Editor, Media Player or the Translation Editor in which no user activity nor web browser alive messages have been logged for a defined period of time.
Compute user interaction statistics as well as transcription and/or translation quality measures on closed edit sessions.
Request automatic time alignments at the word level of supervised transcriptions to the Ingest Service.
Request the re-generation of translations and/or synthesized audiotracks to the Ingest Service, when applicable.
Look for untranslated texts from websites and send them to the Ingest Service to be translated.
Delete source and translated texts from websites no longer accessed for a defined period of time.

The postprocessor module is located under the misc/postprocessor folder.

12. Client tools

The transLectures-UPV Platform offers several libraries and command-line utilities in order to facilitate the client’s interaction with the Web Service API, the TLP Media Player and the TLP Translation Editor. These tools are located under the misc/client-tools folder.

Libraries:

Python → libtlp.py (see Documentation)
PHP → libtlp.php (see Documentation)

Command-line utilities:

Python-based (using libtlp.py):
- ws-client.py: Script to call all Web Service interfaces.
- tx-editor-url-generator.py: Generates valid URLs to the TLP Transcription Editor.
- player-url-generator.py: Generates valid URLs to the TLP Media Player.
- tl-editor-url-generator.py: Generates valid URLs to the TLP Translation Editor.
- web-tl-editor-url-generator.py: Generates valid URLs to the TLP Web Translation Editor.

Appendices

Appendix A: Installation

In this section we provide intallation instructions to properly set up TLP.

Despite the distributed nature of TLP, that is, that each of its components can in theory be installed on different machines, for the sake of simplicity we recommend they be installed on a single machine to create what we have called a TLP Server.

We have tested a TLP installation on Ubuntu 14.04 LTS Desktop and Server versions. It has not been tested on other versions or distributions. All installation notes contained in this documentation are based on Debian/Ubuntu-based distributions and may not, therefore, be applicable to other distributions.

Requirements

What follows are the minimum and recommended hardware requirements for installing TLP on a single machine.

Minimum hardware requirements

Intel Core i5 processor.
8 GB RAM.
500 GB of free hard disk space.
Linux-based operating system.

Recommended hardware requirements (TLP + in-house grid engine for state-of-the art ASR/MT/TTS technologies)

High-end Intel Core i7 processor.
2 x high-end GPUs with Nvidia CUDA support.
64 GB RAM.
3 TB of free hard disk space.
Ubuntu Server 14.04 LTS operating system.

Automatic/Guided Installation

This TLP release includes an installation script (install.sh) which automatically installs the Database, Web Service, Ingest Service and Player components on a single machine, ensuring that all software dependencies are satisfied. So, to automatically install TLP, simply do:

chmod +x ./install.sh
sudo ./install.sh

and follow the instructions given by the script.

Please note that this script was conceived to be executed on a fresh installation of the Ubuntu 14.04 LTS Server Operating System. If this is not your case, please refer to the Manual Installation instructions below, or execute it at your own risk.

Manual Installation

First, we recommend the creation of a TLP user on your machine for running all tasks and processess related to TLP workflows. In our installation examples, we will assume that a tluser user and tlgroup group have been created, as well as a home directory for this user (/home/tluser) where all media, subtitles and uploaded files will be stored. In Debian/Ubuntu systems this can be done as follows:

sudo useradd -d /home/tluser -m -s /bin/bash tluser
sudo groupadd tlgroup
sudo usermod -a -G tlgroup tluser

Manual installation and configuration guides for each TLP component:

Database
Web Service
Transcription Editor
Media Player
Translation Editor
Web Translation Editor
Ingest Service
Postprocessor module

Database

Requirements

PostgreSQL server and client (version >= 9.3).

Installation Steps

The following steps take you through the process of creating a transLectures Repository, including the creation of a system user, the Database and the required directory structure.

Install the PostgreSQL server and client packages. On Ubuntu 14.04 LTS, this is easy to do using the following command line:
```
sudo apt-get install postgresql
```
Create a new database user, the transLectures user (tluser), which will be used when connecting to the TLP Database. Remember to set a password for this user (e.g. tlpass). Note that, in order to execute the following commands, we must be operating as the default database superuser, postgres.
```
sudo -u postgres createuser -s tluser
sudo -u postgres psql -c "ALTER USER tluser WITH PASSWORD 'tlpass'"
```
Create a new database with the name tldb, and insert the Database schema and static data (located at db/sql/schema.sql and db/sql/static_data.sql, respectively). Note that, in order to execute the following commands, we must be operating as the transLectures user (tluser, see Manual Installation).
```
sudo -u tluser createdb tldb
sudo -u tluser psql -f "db/sql/schema.sql" tldb
sudo -u tluser psql -f "db/sql/static_data.sql" tldb
```

Create speech, text and web root directories and the required subdirectories, and set proper directory permissions.

sudo mkdir -p /home/tluser/tlp-repo/speech/media
sudo mkdir -p /home/tluser/tlp-repo/speech/trans
sudo mkdir -p /home/tluser/tlp-repo/speech/uploads
sudo mkdir -p /home/tluser/tlp-repo/speech/align
sudo mkdir -p /home/tluser/tlp-repo/text/trans
sudo mkdir -p /home/tluser/tlp-repo/text/uploads
sudo mkdir -p /home/tluser/tlp-repo/web/uploads
sudo chown tluser:tlgroup /home/tluser/tlp-repo/*
sudo chmod 775 /home/tluser/tlp-repo/*
sudo chmod g+s /home/tluser/tlp-repo/*

Insert a new row in the machines table and add a machine named localhost, with IP 127.0.0.1 and ID 0:

sudo -u tluser psql tldb -c "INSERT INTO machines (id, hostname, ip) VALUES (0, 'localhost', '127.0.0.1');"

Insert mount points of the required subdirectories for the machine ID 0 created in step 5.

sudo -u tluser psql tldb -c "
 INSERT INTO mount_points (name, machine_id, path) VALUES
     ('media', 0, '/home/tluser/tlp-repo/speech/media'),
     ('transcriptions', 0, '/home/tluser/tlp-repo/speech/trans'),
     ('uploads', 0, '/home/tluser/tlp-repo/speech/uploads'),
     ('valign_uploads', 0, '/home/tluser/tlp-repo/speech/align'),
     ('text_documents', 0, '/home/tluser/tlp-repo/text/trans'),
     ('text_uploads', 0, '/home/tluser/tlp-repo/text/uploads'),
     ('web_uploads', 0, '/home/tluser/tlp-repo/web/uploads');"

(optional) Check the connection to the Database using the following command line:
```
sudo -u tluser psql tldb
```

Web Service

Requirements

Apache HTTP Server (version 2.0 or above).
Apache Mod modwsgi library for Apache (version 3.3 or above).
Python 2.
Psycopg Python library (version 2.4 or above).
Paste Python library (version 1.7 or above).
TLP Database.

Installation Steps

Install the required software dependencies. On Ubuntu 14.04 LTS, this is easy to do using the following command line:

sudo apt-get install apache2 apache2-utils libapache2-mod-wsgi python-dev postgresql-server-dev-all
sudo pip install psycopg2 Paste

Copy the web-service and lib directories into the desired installation directory. In our example we use /home/tluser/tlp.
```
sudo mkdir -p /home/tluser/tlp
sudo cp -r web-service lib /home/tluser/tlp/
```
Configure your Apache sites-enabled file so that a relative address of your HTTP Server points to the Web Service's WSGI script (/home/tluser/tlp/web-service/ws.py). Here we configure it so that the Web Service is accessible from the relative path /api (e.g. http://myserver.com/api). In Ubuntu 14.04 LTS, the Apache configuration file is located at /etc/apache2/sites-enabled/000-default.conf. You have to add the following command line to your <VirtualHost> directive(s):
```
<VirtualHost *:80>
 ...
 WSGIScriptAlias /api/speech /home/tluser/tlp/web-service/ws-speech.py
 WSGIScriptAlias /api/text /home/tluser/tlp/web-service/ws-text.py
 WSGIScriptAlias /api/web /home/tluser/tlp/web-service/ws-web.py
 <Directory /home/tluser/tlp/web-service>
   Require all granted
 </Directory>
 ...
</VirtualHost>
```
Add www-data user to the tlgroup group.
```
sudo usermod -a -G tlgroup www-data
```
Restart Apache server. On Ubuntu 14.04 LTS:
```
sudo service apache2 restart
```

Configuration

The Web Service comes with a configuration file that indicates its root directory and database connection parameters, among other information. This configuration file must be given the name config.ini and be located in the Web Service installation dir (i.e. /home/tluser/tlp/web-service/config.ini). The specification of the configuration file is as follows:

[authentication]

key_generator = <string>: Web Service’s Secret Key generator.

[storage]

db_name = <string>: Database name.
db_user = <string>: Database user name.
db_host = <string>: Database hostname or IP address.
db_passwd = <string>: Database user password. You can leave this field empty if Database SSL Auth is enabled.

[media_urls]

use_urls = <boolean>: Return URLs instead of absolute file paths when returning URIs of media files.
base_url = <boolean>: URL prefix used to create full media URLs.

[mailing]

enabled = <boolean>: Send e-mail alerts whenever the Web Service fails for whatever reason.
smtp_server = <string>: SMTP server to send e-mails.
from_address = <string>: From address.
to_address = <string>: Comma-separated recipient e-mails.

[misc]

html_msg = <string>: HTML code to be returned by the Web Service when accessing to unexisting API endpoints.

Below is a real example of a Web Service configuration file.

Example of a Web Service’s configuration file

[authentication]
key_generator = 12345

[storage]
db_name = tldb
db_user = tluser
db_host = my-tlp-server.com
# You may leave this empty if SSL auth is enabled:
db_passwd =

[media_urls]
use_urls = yes
base_url = http://my-tlp-server.com/data

[mailing]
enabled = yes
smtp_server = smtp.my-tlp.server.com
from_address = noreply@my-tlp-server.com
to_address = admin@my-tlp-server.com

[misc]
html_msg = <html><head><meta charset="UTF-8"></head><body><p>Hi there!</p></body></html>

Transcription Editor

Requirements

HTTP Server (i.e. Apache HTTP Server)
PHP5 with cURL support
TLP Web Service

Installation steps

Install the required external software dependencies. In our installation example, we use the open source Apache HTTP Server. On Ubuntu 14.04 LTS, this is easy to do using the following command line:
```
sudo apt-get install apache2 php5 libapache2-mod-php5 php5-curl
```
Move all of the files inside the player folder of the TLP package into any folder of your HTTP Server. In our example we will use the directory /home/tluser/tlp/transedit:
```
sudo mkdir -p /home/tluser/tlp
sudo cp -r transedit /home/tluser/tlp/
```
You will need to add the following lines in your <VirtualHost> directive(s) of the Apache sites-enabled file (located at /etc/apache2/sites-enabled/000-default).
```
<VirtualHost>
 ...
   Alias /transedit /home/tluser/tlp/transedit
    <Directory /home/tluser/tlp/transedit>
      Options Indexes FollowSymLinks MultiViews
      Require all granted
    </Directory>
 ...
</VirtualHost>
```
Restart your HTTP Server where necessary (in our example, it is):
```
sudo service apache2 restart
```
(Optional) Check whether the Transcription Editor can be accessed (note that it will show you an error message - this is normal):
```
curl http://localhost/transedit/
```

Configuration

The Transcription Editor makes use of the TLP PHP library. You need to open the file tlplib.php in /misc/client-tools/php/lib and change the $API_URL variable to point to your TLP API URL.

Media Player

Requirements

HTTP Server (i.e. Apache HTTP Server)
PHP5 with cURL support
TLP Web Service

Installation steps

Install the required external software dependencies. In our installation example, we use the open source Apache HTTP Server. On Ubuntu 14.04 LTS, this is easy to do using the following command line:
```
sudo apt-get install apache2 php5 libapache2-mod-php5 php5-curl
```
Move all of the files inside the player folder of the TLP package into any folder of your HTTP Server. In our example we will use the directory /home/tluser/tlp/player:
```
sudo mkdir -p /home/tluser/tlp
sudo cp -r player /home/tluser/tlp/
```
You will need to add the following lines in your <VirtualHost> directive(s) of the Apache sites-enabled file (located at /etc/apache2/sites-enabled/000-default).
```
<VirtualHost>
 ...
   Alias /player /home/tluser/tlp/player
    <Directory /home/tluser/tlp/player>
      Options Indexes FollowSymLinks MultiViews
      Require all granted
    </Directory>
 ...
</VirtualHost>
```
Set ownership of the player/translectures/config.json file to the HTTP Server user (in Apache it is www-data), and disable its read and write permissions for groups and others:
```
sudo chown www-data /home/tluser/tlp/player/translectures/config.json
sudo chmod 600 /home/tluser/tlp/player/translectures/config.json
```
(Optional) Create a symbolic link inside the player directory pointing to the media repository directory (/home/tluser/tlp-repo/media, see the section on installation of the Database), for instance player/data:
```
sudo ln -s /home/tluser/tlp-repo/media /home/tluser/tlp/player/data
```
Note that this symbolic link, when in the form of a URL (i.e. http://localhost/player/data), will become the tlbaseurl parameter when calling the Player (please see this Appendix).
Restart your HTTP Server where necessary (in our example, it is):
```
sudo service apache2 restart
```
(Optional) Check whether the Player can be accessed (note that it will show you an error message - this is normal):
```
curl http://localhost/player/
```

Configuration

The configuration file player/translectures/config.json must be edited to in order to grant the Player access to the Web Service.

JSON config file specification:

 {
  "ws_url" : <str> ,
  "data_url" : <str>
 }

Configuration parameters:

ws_url: Web Service URL.
data_url: Media storage data URL (to stream local videos).

JSON config file example:

 {
  "ws_url" : "http://my.server.com/api/speech/",
  "data_url" : "http://my.server.com/player/data/"
 }

Translation Editor

Requirements

HTTP Server (i.e. Apache HTTP Server)
PHP5 with cURL support
TLP Web Service

Installation steps

Install the required external software dependencies. In our installation example, we use the open source Apache HTTP Server. On Ubuntu 14.04 LTS, this is easy to do using the following command line:
```
sudo apt-get install apache2 php5 libapache2-mod-php5 php5-curl
```
Move all of the files inside the docedit folder of the TLP package into any folder of your HTTP Server. In our example we will use the directory /home/tluser/tlp/docedit:
```
sudo mkdir -p /home/tluser/tlp
sudo cp -r docedit /home/tluser/tlp/
```
You will need to add the following lines in your <VirtualHost> directive(s) of the Apache sites-enabled file (located at /etc/apache2/sites-enabled/000-default).
```
<VirtualHost>
 ...
   Alias /docedit /home/tluser/tlp/docedit
    <Directory /home/tluser/tlp/docedit>
      Options Indexes FollowSymLinks MultiViews
      Require all granted
    </Directory>
 ...
</VirtualHost>
```
Set ownership of the docedit/config.json file to the HTTP Server user (in Apache it is www-data), and disable its read and write permissions for groups and others:
```
sudo chown www-data /home/tluser/tlp/docedit/config.json
sudo chmod 600 /home/tluser/tlp/docedit/config.json
```
Restart your HTTP Server where necessary (in our example, it is):
```
sudo service apache2 restart
```
(Optional) Check whether the Translation Editor can be accessed (note that it will show you an error message - this is normal):
```
curl http://localhost/docedit/
```

Configuration

The configuration file docedit/config.json must be edited to in order to grant the Translation Editor access to the Web Service.

JSON config file specification:

 {
  "ws_url" : <str>
 }

Configuration parameters:

ws_url: Web Service URL.

JSON config file example:

 {
  "ws_url" : "http://my.server.com/api/text/"
 }

Web Translation Editor

Requirements

HTTP Server (i.e. Apache HTTP Server)
PHP5 with cURL support
TLP Web Service

Installation steps

Install the required external software dependencies. In our installation example, we use the open source Apache HTTP Server. On Ubuntu 14.04 LTS, this is easy to do using the following command line:
```
sudo apt-get install apache2 php5 libapache2-mod-php5 php5-curl
```
Move all of the files inside the docedit folder of the TLP package into any folder of your HTTP Server. In our example we will use the directory /home/tluser/tlp/webtleditor:
```
sudo mkdir -p /home/tluser/tlp
sudo cp -r webtleditor /home/tluser/tlp/
```
You will need to add the following lines in your <VirtualHost> directive(s) of the Apache sites-enabled file (located at /etc/apache2/sites-enabled/000-default).
```
<VirtualHost>
 ...
   Alias /webtleditor /home/tluser/tlp/webtleditor
    <Directory /home/tluser/tlp/webtleditor>
      Options Indexes FollowSymLinks MultiViews
      Require all granted
    </Directory>
 ...
</VirtualHost>
```
Restart your HTTP Server where necessary (in our example, it is):
```
sudo service apache2 restart
```
(Optional) Check whether the Web Translation Editor can be accessed (note that it will show you an error message - this is normal):
```
curl http://localhost/webtleditor/
```

Configuration

The main JavaScript file webtledit/js/main.js must be edited to in order to grant the Web Translation Editor access to the Web Service.

Configuration parameters:

TLP_API_URL: Web Service URL.

On the first line of the file you will find the variable to change.

main.js first line example:

var TLP_API_URL = "http://my.server.com/api/web/";

Ingest Service

Requirements

Python 2.
Psycopg Python library (version >=2.4).
tldextract Python library (version >=1.5).
beautifulsoup4 Python library.
nltk Python library.
FFmpeg with H.264 support.
Zip command-line utility.
(optional) Job scheduling/queue management system.
TLP Database.

Installation Steps

Install the required software dependencies. On Ubuntu 14.04 LTS, this is easy to do using the following command line:
```
sudo apt-get install zip postgresql-server-dev-all
```
The tldextract python library is not available in the official Ubuntu repositories. However, it can be easily installed via pip:
```
sudo apt-get install python-pip
sudo pip install tldextract beautifulsoup4 nltk psycopg2
```
The FFmpeg package stored in the Ubuntu repositories does not include H.264 codec support, so you will have to download all sources and compile them by yourself. You will find a useful guide for said compilation in Debian/Ubuntu distributions here.

You can also set up a job scheduling/queue management system, required in order to execute and manage transcription, translation and media conversion processes. In our case we have tested TLP with the open source version of the Sun Grid Engine.
Copy the ingest-service directory into the desired installation directory. In our example, we use /home/tluser/tlp.
```
sudo mkdir -p /home/tluser/tlp
sudo cp -r ingest-service /home/tluser/tlp/
```
Specify the root directory where Align, ASR, MT and TTS modules will be placed in the systems mount point of the Database. Asuming that we will put all these modules under /home/tluser/tlp/ingest-service/modules/tl:
```
sudo -u tluser psql tldb -c "
 INSERT INTO mount_points (name, machine_id, path) VALUES
     ('systems', 0, '/home/tluser/tlp/ingest-service/modules/tl');"
```

Configuration

The TLP Ingest Service comes with a configuration file in which several parameters and options are defined. The Ingest Service's Core will attempt to load a file named config.ini located in the same directory (i.e. /home/tluser/tlp/ingest-service/config.ini). If the configuration file does not exist or cannot be parsed, the execution will fail. You can manually specify another path to the configuration file using the option --config-file (see Execution).

The specification of the configuration file is as follows:

[general]

In this section, general settings can be configured.

hostname = <string>: Host name of the machine that will run the Ingest Service.
tl_user = <string>: System user that will run the Ingest Service, in order to set ownership of all stored files.
tl_group = <string>: System group to set ownership of all stored files.
rm_finished_up_days = <int>: Automatically delete temporary data from finished uploads after n days.
rm_error_up_days = <int>: Automatically delete temporary data from failed uploads after n days.
local_repository = <boolean>: Store all media files uploaded to TLP instead of accessing them via URL (when provided).

[storage]

In this section, TLP Database connection settings can be customised.

db_name = <string>: Database name to connect with.
db_user = <string>: Database user name.
db_passwd = <string>: Database user password. You can leave this field empty if database SSL auth is enabled.
db_host = <string>: Database hostname or IP address.

[scheduler]

In this section, some settings relating the job management system are defined.

localhost = <string>: Local machine hostname for the job management system. It is used to launch media conversion processes in the local machine, as these tasks are very network-consuming.
status_cmd = <string>: System call to get the status of all processes previously submitted to the job management system.
submit_scr = <string>: Path to the script or binary program to submit tasks to the job management system.
submit_opts = <string>: Options of the submit script (which will be appended to all submit calls).
job_name_prefix = <string>: Job name prefix for all tasks submitted to the job management system.

[sessions]

enabled = <boolean>: Make the Ingest Service to create edit sessions on update operations over the media ID that is being processed. Hence, users won’t be able to edit subtitles with the TLP Player until the update operation finishes.
author_id = <boolean>: Author ID of the Ingest Service.
author_name = <boolean>: Author Name of the Ingest Service.
author_conf = <int>: Confidence of the Ingest Service, from 0 to 100 (100, right?).

[mailing]

In this section, you can customize several parameters of the Mailing module.

enabled = <boolean>: Enables or disables Mailing module.
smtp_server = <string>: SMTP Server hostname or IP address used to send e-mail notifications.
from_address = <string>: E-mail address that will be used as "From" address.
send_client_started_mail = <boolean>: Enables or disables e-mail notifications to the client to inform that an upload has started to be processed.
send_client_error_mail = <boolean>: Enables or disables e-mail notifications to the client to inform that an upload has failed.
send_client_finished_mail = <boolean>: Enables or disables e-mail notifications to the client to inform that an upload has successfully finished.
send_admin_started_mail = <boolean>: Enables or disables e-mail notifications to the system administrator to inform that an upload has started to be processed.
admin_address_started = <string>: E-mail address of the system administrator that will receive notifications about uploads that have started to be processed.
send_admin_error_mail = <boolean>: Enables or disables e-mail notifications to the system administrator to inform that an upload has failed.
admin_address_error = <string>: E-mail address of the system administrator that will receive notifications about uploads that have failed.
send_admin_finished_mail = <boolean>: Enables or disables e-mail notifications to the system administrator to inform that an upload has finished.
admin_address_finished = <string>: E-mail address of the system administrator that will receive notifications about uploads that have finished.

[file_formats]

In this section, the required/allowed file formats for every type of file that can be uploaded to the Ingest Service are defined. Please note that only file formats in the file_formats table of the TLP Database can be used.

max_audio_track_length = <int>: Defines the maximum length allowed, in seconds, of the audio track of the main media file.
generate_pcm_stream = <boolean>: Generate a PCM stream file to be retrieved by the TLP Player under the Advanced Mode.
required_video_formats = <string>: Defines which video formats are needed to maximise compatibility of the TLP Player with all browsers. If the uploaded media files are not in some of the required formats, then the Ingest Service will do the appropriate conversion.
allowed_video_formats = <string>: Comma-separated list of all video formats that will be allowed to be uploaded as main media.
allowed_audio_formats = <string>: Comma-separated list of all audio formats that will be allowed to be uploaded as main media.
allowed_slides_text_formats = <string>: Comma-separated list of all slides text formats that will be allowed to be uploaded.
allowed_slides_video_formats = <string>: Comma-separated list of all slides video formats (video-recorded slides) that will be allowed to be uploaded.
allowed_docs_formats = <string>: Comma-separated list of all document formats that will be allowed to be uploaded.
allowed_caption_formats = <string>: Comma-separated list of all subtitle formats that will be allowed to be uploaded.
allowed_thumbnail_formats = <string>: Comma-separated list of all thumbnail formats that will be allowed to be uploaded.
allowed_packages = <sring>: Comma-separated list of all package formats that will be allowed to be uploaded.

[text_retrieval_module]

In this section, the location of the text retrieval module is defined.

module_path = <string>: Path to the text retrieval module. If it is not available, just leave the right part empty.

[data]

In this section, paths to external data files are defined.

default_background_img = <path>: Default image in case a thumbnail cannot be extracted.
audio_background_img = <path>: Background image that will be used to encode videos for the TLP Player using an uploaded audio file as audio stream. If it is not provided, then the generated videos will show a black background.
test_thumbnail_img = <path>: Background image that will be used to encode a short video for the TLP Player when using the test mode of the Ingest Service.

[url_decoders]

In this section, URL decoders for non-public URLs can be registered to be used by the URL Downloader Module.

[document_format_parsers]

In this section, Document parsers for different document formats can be registered.

Example of a Ingest Service configuration file


[general]
hostname = my-tlp-server.com
tl_user = tluser
tl_group = users
rm_finished_up_days = 1
rm_error_up_days = 5
local_repository = no

[storage]
db_name = tldb
db_user = tluser
db_passwd =
db_host = my-tlp-server.com

[scheduler]
localhost = my-tlp-server
status_cmd = qstat -u tluser
submit_scr = qsubmit
submit_opts = ""
job_name_prefix = TLP-job

[sessions]
enabled = no
author_id = ingest-service
author_name = Ingest Service
author_conf = 100

[mailing]
enabled = yes
smtp_server = smtp.my-tlp.server.com
from_address = no-reply@my-tlp.server.com
send_client_started_mail = yes
send_client_error_mail = yes
send_client_finished_mail = yes
send_admin_started_mail = yes
admin_address_started = admin@my-tlp.server.com
send_admin_error_mail = yes
admin_address_error = admin@my-tlp.server.com
send_admin_finished_mail = yes
admin_address_finished = admin@my-tlp.server.com

[file_formats]
max_audio_track_length = 10800
generate_pcm_stream = yes
required_video_formats = mp4
allowed_video_formats = mp4, m4v, ogv, wmv, avi, mpg, flv, mov, 3gp, webm, mkv
allowed_audio_formats = wav, mp2, mp3, oga, flac, aac, ape, wma, m4a
allowed_slides_text_formats = txt, ppt, pptx, doc, docx, pdf
allowed_slides_video_formats = mp4, m4v, ogv, wmv, avi, mpg, flv
allowed_docs_formats = pdf, doc, docx, ppt, pptx, txt, html, xls, xlsx
allowed_caption_formats = dfxp, trs, srt
allowed_thumbnail_formats = jpg
allowed_packages = zip

[text_retrieval_module]
module_path = /path/to/my/text_retrieval_module.py

[data]
default_background_img = /path/to/default_background.png
audio_background_img = /path/to/audio_background.png
test_thumbnail_img = /path/to/test_thumbnail.png

[url_decoders]
youtube = /path/to/url-decoder.youtube.py

[document_format_parsers]
html = modules/doc_parsers/doc_parser.html.py
txt = modules/doc_parsers/doc_parser.txt.py

Execution

The same instructions can be applied to properly execute the Ingest Service for text documents (ingest-service/is-text.py) and websites (ingest-service/is-web.py).

In order to run the Ingest Service for speech, you have to execute the Ingest Service’s Core (ingest-service/is-speech.py). All options of the Core python script are shown below.

Usage: is-speech.py [options]

Options:
  -h, --help            show this help message and exit
  -v, --verbose         Verbose power on!
  -d, --debug           Debug mode
  -e, --disable-errors  Disable ingest errors
  -C CONFIG_FILE, --config-file=CONFIG_FILE
                        Configuration file. Default: config.ini
  -D DB_NAME, --database=DB_NAME
                        Database with which to work. Default: specified in
                        config file

To run the Ingest Service:

python /home/tluser/tlp/ingest-service/is-speech.py -v

However, you might probably want to schedule its execution periodically. Under UNIX systems you can consider using Crontab. For instance, if you want to execute the Ingest Service every minute, logging all information into a log file, put this line in the tluser's crontab file:

*/1 * * * * /bin/bash -l -c -x '
 source /home/tluser/.bashrc;
 python /home/tluser/tlp/ingest-service/is-speech.py -v >> /home/tluser/tlp/ingest-service/is-speech.log 2>&1;'

To prevent multiple executions of the Ingest Service to be running at the same time, you can simply use a custom lock file.

*/1 * * * * /bin/bash -l -c -x '
  if [ ! -e /home/tluser/.cron-ingest.lock ]; then
    touch /home/tluser/.cron-ingest.lock;
    source /home/tluser/.bashrc;
    python /home/tluser/tlp/ingest-service/is-speech.py -v >> /home/tluser/tlp/ingest-service/is-speech.log 2>&1;
    rm /home/tluser/.cron-ingest.lock;
  fi'

Postprocessor

Requirements

Python 2.
Psycopg Python library (version >=2.4).
TLP Database.

Installation Steps

Install the required software dependencies. On Ubuntu 14.04 LTS, this is easy to do using the following command line:
```
sudo apt-get install postgresql-server-dev-all
```
The tldextract python library is not available in the official Ubuntu repositories. However, it can be easily installed via pip:
```
sudo apt-get install python-pip
sudo pip install psycopg2
```
Copy the misc/postprocessor directory into the desired installation directory. In our example, we use /home/tluser/tlp.
```
sudo mkdir -p /home/tluser/tlp
sudo cp -r misc/postprocessor /home/tluser/tlp/
```

Configuration

The Postprocessor module comes with a configuration file in which several parameters and options are defined. This module will attempt to load a file named config.ini located in the same directory (i.e. /home/tluser/tlp/postprocessor/config.ini). If the configuration file does not exist or cannot be parsed, the execution will fail. You can manually specify another path to the configuration file using the option --config-file (see Execution).

The specification of the configuration file is as follows:

[general]

In this section, general settings can be configured.

end_session_human = <int>: Time in minutes after the last modification received to automatically close the session.
end_session_human_alive = <int>: Time in minutes after the last alive ping received to automatically close the session.
end_session_computer = <string>: Time in minutes after the last modification received by a computer (i.e. Ingest Service) to automatically close the session.
compute_wer = <bool>: Enable/disable computation of the Word Error Rate (WER) measure after the end of each user session.
compute_cer = <bool>: Enable/disable computation of the Character Error Rate (CER) measure after the end of each user session.
compute_bleu = <bool>: Enable/disable computation of the Bilingual Evaluation Understudy (BLEU) measure after the end of each user session.
compute_ter = <bool>: Enable/disable computation of the Translation Error Rate (TER) measure after the end of each user session.
align_transcriptions = <bool>: Enable/disable requesting automatic text-to-speech alignments after the end of each user session.

[database]

In this section, TLP Database connection settings can be customised.

db_name = <string>: Database name to connect with.
db_user = <string>: Database user name.
db_host = <string>: Database hostname or IP address.
db_passwd = <string>: Database user password. You can leave this field empty if database SSL auth is enabled.

[libtlp]

In this section are customised all those parameters needed to query the Web Service using the TLP Python library (libtlp.py).

web_service_url = <string>: Base URL location of the TLP Web Service.
http_auth_enabled = <bool>: Enable or disable HTTP Authentication.
http_username = <string>: HTTP Auth username.
http_password = <string>: HTTP Auth Password.
api_username = <string>: TLP username / API username.
api_secret_key = <string>: API Secret Key Authentication Token.
author_id = <string>: ID to identify the Postprocessor module when dealing with unclosed edit sessions.
author_name = <string>: Name to identify the Postprocessor module when dealing with unclosed edit sessions.

[external_modules]

In this section can be defined external modules to compute evaluation measures over revised subtitles or translations.

wer = <string>: Path to an external python module to compute WER figures.
cer = <string>: Path to an external python module to compute CER figures.
bleu = <string>: Path to an external python module to compute BLEU figures.
ter = <string>: Path to an external python module to compute TER figures.

[websites]

In this section several settings concerning translation of websites can be customised.

wait_time_last_added_min = <int>: Time in minutes after the last added source text to wait before requesting the translation of all pending contents.
delete_texts = <bool>: Enable/disable the automatic deletion of source texts and their corresponding translations when not accessed/requested for a defined period of time.
delete_texts_after_last_access_days = <int>: Time in days after the last access to a specific text content to wait for proceed with the automatic deletion.
delete_texts_supervised_translations = <bool>: Delete or not manually supervised translations.
delete_texts_check_at = <string>: Time in HH:MM format at which check whether there exist contents to be automatically deleted.

Example of a Postprocessor configuration file


[general]
end_session_human = 300
end_session_human_alive = 5
end_session_computer = 2880
compute_wer = True
compute_cer = False
compute_bleu = True
compute_ter = False
align_transcriptions = True

[database]
db_name = tldb
db_user = tluser
db_host = my-tlp-server.com
# You may leave this empty if SSL auth is enabled:
db_passwd =

[libtlp]
web_service_url = http://my-tlp-server.com/api/
http_auth_enabled = no
http_username =
http_password =
api_username = tluser
api_secret_key = akjsfd982323098qwjs209823id09321io3290d
author_id = tlp-postprocessor
author_name = TLP Postprocessor

[external_modules]
wer =
cer =
bleu =
ter =

[websites]
wait_time_last_added_min = 0
delete_texts = yes
delete_texts_after_last_access_days = 30
delete_texts_supervised_translations = no
delete_texts_check_at = 04:00

Execution

In order to run the Postprocessor module, you have to execute the Postprocessor python script (postprocessor/postprocessor.py). All options of this python script are shown below.

Usage: postprocessor.py [options]

Options:
  -h, --help            show this help message and exit
  -q, --quiet           Verbose power on!
  -c CFG_FP, --config-file=CFG_FP
                        Config file. Default: config.ini.
  -H DB_HOST, --database-host=DB_HOST
                        Database hostname. Default: from config file.
  -U DB_USER, --database-user=DB_USER
                        Database username. Default: from config file.
  -P DB_PASS, --database-password=DB_PASS
                        Database password. Default: from config file.
  -n, --do-not-mark-processed
                        Do not mark caption changes as processed.
  -r, --do-not-regenerate
                        Do not launch regeneration processes.
  -S VIDEO_HASH, --regen-stats-video-hash=VIDEO_HASH
                        Just re-compute stats for a Speech object hash and
                        exit.
  -T DOCUMENT_HASH, --regen-stats-text-hash=DOCUMENT_HASH
                        Just re-compute stats for a Speech object hash and
                        exit.

To run the Postprocessor:

python /home/tluser/tlp/postprocessor/postprocessor.py

However, you might probably want to schedule its execution periodically. Under UNIX systems you can consider using Crontab. For instance, if you want to execute the Postprocessor every minute, logging all information into a log file, put this line in the tluser's crontab file:

*/1 * * * * /bin/bash -l -c -x '
 source /home/tluser/.bashrc;
 python /home/tluser/tlp/postprocessor/postprocessor.py >> /home/tluser/tlp/postprocessor/postprocessor.log 2>&1;'

To prevent multiple executions of the Postprocessor module to be running at the same time, you can simply use a custom lock file.

*/1 * * * * /bin/bash -l -c -x '
  if [ ! -e /home/tluser/.cron-postprocessor.lock ]; then
    touch /home/tluser/.cron-postprocessor.lock;
    source /home/tluser/.bashrc;
    python /home/tluser/tlp/postprocessor/postprocessor.py -v >> /home/tluser/tlp/postprocessor/postprocessor.log 2>&1;
    rm /home/tluser/.cron-postprocessor.lock;
  fi'

Appendix B: Calling the TLP Media Player

The TLP Media Player must be called using different input parameters depending on which subtitles are being edited, the language of these subtitles and what kind of user is doing the editing. These parameters are sent as a Base64-encoded JSON string via HTTP GET or POST methods. A full request key is sent for the authentication of the API client on all Web Service calls.

API users should avoid sending their secret key on the Player input parameters, since these parameters are exposed to third-parties (i.e. external Player users) as they travel inside the URL to the Player. Please see Annex Generating a Request Key to learn how to produce valid full request keys in order to protect your private secret key.

The transLectures-UPV Platform includes in its Client Tools Package the player-url-generator.py command-line script that generates valid Player URLs for the given input parameters. The usage of the --debug option might be very useful to check how these URLs are generated. Furthermore, you will find libraries for different platforms that include Player URL generation methods.

Input parameters

JSON Object Specification:

 {
  "id" : <str> ,
  "lang" : <str> ,
  "author_id" : <str> ,
  "author_conf" : <int> ,
  "author_name" : <str> ,
  "expire" : <int> ,
  "api_user" : <str> ,
  "request_key" : <str>
 }

id:`<str>`	Media ID.
lang:`<str>`	Language code of the subtitles being edited (i.e. en, es, ca). If this parameter is not defined, the Player will load the source language transcriptions (optional).
author_id:`<str>`	ID of the user that will edit the subtitles. It is typically the internal user ID that the API client’s organisation assigns to the user.
author_conf:`<int>`	Integer value (range 0-100) that indicates the confidence level that the API client’s organisation provide to the user.
author_name:`<str>`	Full name of the user that will edit the subtitles (optional).
expire:`<int>`	Expiration date of the URL in UNIX timestamp format.
api_user:`<str>`	TLP username / API Client username (Please see Web Service user authentication).
request_key:`<str>`	Request key (see Generating a Request Key).

Example 1. TLP Player base64-JSON-encoded parameters example:

 {
  "id" : "id-001",
  "lang" : "en",
  "author_id" : "bobama",
  "author_conf" : 100,
  "author_name" : "Barack Obama",
  "expire" : 1400173491,
  "api_user" : "tluser",
  "request_key" : "5251982f3d00544e6e9a91962a2eec2f0b3df38c"
 }

Parameters are sent as a Base64-encoded JSON string. The JSON string for the above example would be as follows:

 {"id" : "id-001", "lang" : "en", "author_id" : "bobama", "author_conf" : 100, "author_name" : "Barack Obama", "expire" : 1400173491, "api_user" : "tluser", "request_key" : "5251982f3d00544e6e9a91962a2eec2f0b3df38c"}

Base64 encode of the above JSON string:

 eyJpZCIgOiAiaWQtMDAxIiwgImxhbmciIDogImVuIiwgImF1dGhvcl9pZCIgOiAiYm9iYW1hIiwgImF1dGhvcl9jb25mIiA6IDEwMCwgImF1dGhvcl9uYW1lIiA6ICJCYXJhY2sgT2JhbWEiLCAiZXhwaXJlIiA6IDE0MDAxNzM0OTEsICJhcGlfdXNlciIgOiAidGx1c2VyIiwgInJlcXVlc3Rfa2V5IiA6ICI1MjUxOTgyZjNkMDA1NDRlNmU5YTkxOTYyYTJlZWMyZjBiM2RmMzhjIn0=

HTTP call

The requested Base64 JSON string is received by the Player via HTTP GET or POST methods using the following parameters:

request → Base64 JSON string
t → Media start time in seconds (optional)

Example 2. TLP Player URL:

http://my-tlp-server.com/player?request=eyJpZCIgOiAiaWQtMDAxIiwgImxhbmciIDogImVuIiwgImF1dGhvciIgOiAiYm9iYW1hIiwgImF1dGhvcl9uYW1lIiA6ICJCYXJhY2sgT2JhbWEiLA0KImF1dGhvcl9jb25mIiA6IDEwMCwgImludGVybmFsdXNlciIgOiAwLCAiZXhwaXJlIiA6IDE0MDAxNzM0OTEsICJhcGlfdXNlciIgOiAidGx1c2VyIiwgDQoicmVxdWVzdF9rZXkiIDogIjUyNTE5ODJmM2QwMDU0NGU2ZTlhOTE5NjJhMmVlYzJmMGIzZGYzOGMifQ==

Appendix C: Calling the TLP Transcription Editor

The Transcription Editor and the Media Player are called in an identical manner. Please refer to Calling the TLP Media Player for further information. The same HTTP Base64 JSON string is valid for both editors:

Example 3. TLP Transcription Editor URL:

http://my-tlp-server.com/transedit?request=eyJpZCIgOiAiaWQtMDAxIiwgImxhbmciIDogImVuIiwgImF1dGhvciIgOiAiYm9iYW1hIiwgImF1dGhvcl9uYW1lIiA6ICJCYXJhY2sgT2JhbWEiLA0KImF1dGhvcl9jb25mIiA6IDEwMCwgImludGVybmFsdXNlciIgOiAwLCAiZXhwaXJlIiA6IDE0MDAxNzM0OTEsICJhcGlfdXNlciIgOiAidGx1c2VyIiwgDQoicmVxdWVzdF9rZXkiIDogIjUyNTE5ODJmM2QwMDU0NGU2ZTlhOTE5NjJhMmVlYzJmMGIzZGYzOGMifQ==

Appendix D: Calling the TLP Translation Editor

The TLP Translation Editor must be called using different input parameters depending on which document is being edited, the language of the translation and what kind of user is doing the editing. These parameters are sent as a Base64-encoded JSON string via HTTP GET or POST methods. A full request key is sent for the authentication of the API client on all Web Service calls.

API users should avoid sending their secret key on the input parameters, since these parameters are exposed to third-parties (i.e. external users) as they travel inside the URL to the Editor. Please see Annex Generating a Request Key to learn how to produce valid full request keys in order to protect your private secret key.

Input parameters

JSON Object Specification:

 {
  "id" : <str> ,
  "lang" : <str> ,
  "author_id" : <str> ,
  "author_conf" : <int> ,
  "author_name" : <str> ,
  "expire" : <int> ,
  "api_user" : <str> ,
  "request_key" : <str>
 }

id:`<str>`	Media ID.
lang:`<str>`	Language code of the translation being edited (i.e. en, es, ca).
author_id:`<str>`	ID of the user that will edit the translation. It is typically the internal user ID that the API client’s organisation assigns to the user.
author_conf:`<int>`	Integer value (range 0-100) that indicates the confidence level that the API client’s organisation provide to the user.
author_name:`<str>`	Full name of the user that will edit the translation (optional).
expire:`<int>`	Expiration date of the URL in UNIX timestamp format.
api_user:`<str>`	TLP username / API Client username (Please see Web Service user authentication).
request_key:`<str>`	Request key (see Generating a Request Key).

Example 4. TLP Player base64-JSON-encoded parameters example:

 {
  "id" : "id-001",
  "lang" : "en",
  "author_id" : "bobama",
  "author_conf" : 100,
  "author_name" : "Barack Obama",
  "expire" : 1400173491,
  "api_user" : "tluser",
  "request_key" : "5251982f3d00544e6e9a91962a2eec2f0b3df38c"
 }

Parameters are sent as a Base64-encoded JSON string. The JSON string for the above example would be as follows:

 {"id" : "id-001", "lang" : "en", "author_id" : "bobama", "author_conf" : 100, "author_name" : "Barack Obama", "expire" : 1400173491, "api_user" : "tluser", "request_key" : "5251982f3d00544e6e9a91962a2eec2f0b3df38c"}

Base64 encode of the above JSON string:

 eyJpZCIgOiAiaWQtMDAxIiwgImxhbmciIDogImVuIiwgImF1dGhvcl9pZCIgOiAiYm9iYW1hIiwgImF1dGhvcl9jb25mIiA6IDEwMCwgImF1dGhvcl9uYW1lIiA6ICJCYXJhY2sgT2JhbWEiLCAiZXhwaXJlIiA6IDE0MDAxNzM0OTEsICJhcGlfdXNlciIgOiAidGx1c2VyIiwgInJlcXVlc3Rfa2V5IiA6ICI1MjUxOTgyZjNkMDA1NDRlNmU5YTkxOTYyYTJlZWMyZjBiM2RmMzhjIn0=

HTTP call

The requested Base64 JSON string is received by the Editor via HTTP GET or POST methods using the following parameters:

request → Base64 JSON string

Example 5. TLP Translation Editor URL:

http://my-tlp-server.com/docedit?request=eyJpZCIgOiAiaWQtMDAxIiwgImxhbmciIDogImVuIiwgImF1dGhvcl9pZCIgOiAiYm9iYW1hIiwgImF1dGhvcl9jb25mIiA6IDEwMCwgImF1dGhvcl9uYW1lIiA6ICJCYXJhY2sgT2JhbWEiLCAiZXhwaXJlIiA6IDE0MDAxNzM0OTEsICJhcGlfdXNlciIgOiAidGx1c2VyIiwgInJlcXVlc3Rfa2V5IiA6ICI1MjUxOTgyZjNkMDA1NDRlNmU5YTkxOTYyYTJlZWMyZjBiM2RmMzhjIn0=

Appendix E: Calling the TLP Web Translation Editor

Input parameters

JSON Object Specification:

 {
  "id" : <str> ,
  "expire" : <int> ,
  "api_user" : <str> ,
  "request_key" : <str> ,
  "lang" : <str>
 }

id:`<str>`	Website key string.
expire:`<int>`	Expiration date of the URL in UNIX timestamp format.
api_user:`<str>`	TLP username / API Client username (Please see Web Service user authentication).
request_key:`<str>`	Request key (see Generating a Request Key).
lang:`<str>`	Language code of the translation being edited (i.e. en, es, ca).

Example 6. TLP Player base64-JSON-encoded parameters example:

 {
  "id" : "b16fc9bfc043752047d7b932fedc861e",
  "lang" : "es",
  "expire" : 1400173491,
  "api_user" : "tluser",
  "request_key" : "5251982f3d00544e6e9a91962a2eec2f0b3df38c"
 }

Parameters are sent as a Base64-encoded JSON string. The JSON string for the above example would be as follows:

 {"id" : "b16fc9bfc043752047d7b932fedc861e", "lang" : "es", "expire" : 1400173491, "api_user" : "tluser", "request_key" : "5251982f3d00544e6e9a91962a2eec2f0b3df38c"}

Base64 encode of the above JSON string:

 eyJpZCIgOiAiYjE2ZmM5YmZjMDQzNzUyMDQ3ZDdiOTMyZmVkYzg2MWUiLCAibGFuZyIgOiAiZXMiLCAiZXhwaXJlIiA6IDE0MDAxNzM0OTEsICJhcGlfdXNlciIgOiAidGx1c2VyIiwgInJlcXVlc3Rfa2V5IiA6ICI1MjUxOTgyZjNkMDA1NDRlNmU5YTkxOTYyYTJlZWMyZjBiM2RmMzhjIn0=

HTTP call

The requested Base64 JSON string is received by the Editor via HTTP GET or POST methods using the following parameters:

request → Base64 JSON string

Example 7. TLP Translation Editor URL:

http://my-tlp-server.com/webtledit?request=eyJpZCIgOiAiaWQtMDAxIiwgImxhbmciIDogImVuIiwgImF1dGhvcl9pZCIgOiAiYm9iYW1hIiwgImF1dGhvcl9jb25mIiA6IDEwMCwgImF1dGhvcl9uYW1lIiA6ICJCYXJhY2sgT2JhbWEiLCAiZXhwaXJlIiA6IDE0MDAxNzM0OTEsICJhcGlfdXNlciIgOiAidGx1c2VyIiwgInJlcXVlc3Rfa2V5IiA6ICI1MjUxOTgyZjNkMDA1NDRlNmU5YTkxOTYyYTJlZWMyZjBiM2RmMzhjIn0=

Appendix F: Web Service API: Technical Preface

The transLectures-UPV Platform includes in its Client Tools Package several libraries for different platforms that implement all the interfaces described below. Also, you will find in that package the ws-client.py command-line tool ready to be used for making all these API calls.

Please read carefully the following considerations before interacting with this API. These considerations are valid for almost all API interfaces except for /web/getjs, /web/translate and /web/amend.

Allowed HTTP methods

The interfaces featured by the Web Service can be called using either GET or POST methods.

GET Method:

Using multiple GET parameters. Example:

GET Parameters:

parameter1=value1
parameter2=value2

URL:

http://my-tlp-server.com/api/action?parameter1=value1&parameter2=value2

Using a single Base64-encoded JSON dict data GET parameter. Example:

Parameters encoded into a JSON Dict:

{"parameter1":"value1", "parameter2": "value2"}

Base64-encoded JSON dict:

eyJwYXJhbWV0ZXIxIjoidmFsdWUxIiwgInBhcmFtZXRlcjIiOiAidmFsdWUyIn0

URL:

http://my-tlp-server.com/api/action?data=eyJwYXJhbWV0ZXIxIjoidmFsdWUxIiwgInBhcmFtZXRlcjIiOiAidmFsdWUyIn0

POST Method:
- All interfaces except /ingest/new and ingest/update:
  - Content-Type: application/json
    
    Query parameters: must be sent as a JSON dictionary in the body of the request.
- /ingest/new and ingest/update interfaces. Two possibilities:
  - Content-Type: application/json as above, or
  - Content-Type: multipart/form-data, if a media/document package have to be uploaded too.
    
    Query parameters: must be sent as a JSON dictionary stored in a field called json:
    
    Content-Disposition: form-data; name="json"; Content-Type: application/json
    
    Media Package or Document Package: must be sent as a zip file in a field called pkg:
    
    Content-Disposition: form-data; name="pkg"; Content-Type: application/zip

The Web Service will return an HTTP 400 Bad Request error code whenever required arguments are missing or are provided under an incorrect format.

The ws-client.py command-line tool implements (via the libtlp python library) all possible ways to send query parameters to the API described above (see the options --use-get-query and --use-data-param). If you plan to implement your own API client, the --debug option of this script might be very useful for you to check how parameters and HTTP calls must be generated.

API Client Authentication

API clients have to send in every call to the Web Service the following parameters:
- user → API Client username / TLP username,
- auth_token → The authentication token for user authentication.
And additionally, if auth_token is a request_key:
- expire → Expiration date UNIX timestamp for the request (seconds since 01-01-1970 in UTC to the expiration date).
To learn how to generate a valid request key, please refer to the Generating a Request Key Annex.
The request key authentication method is valid for all interfaces, except /ingest, /uploadslist, /status, /systems and /revisions.
The Web Service will return the following HTTP error codes:
- 401 Unauthorized, if:
  - The API user does not exist,
  - the authentication token is invalid,
  - the API user does not have permissions to get information about the provided object ID.
- 419 Authentication Timeout, if:
  - the authentication token has expired (only for Request Key).

Appendix G: Web Service API: /speech API specifications

Please note that all the following interfaces will share a common prefix: /speech.

Interfaces for media upload and management:

/ingest	Upload media (audio/video) files and any attachments and metadata to the TLP Server for automatic multilingual subtitling and speech synthesis.
/uploadslist	Get a list of all the user’s uploads.
/status	Check the current status of a specific upload ID.
/systems	Get a list of all available Speech Recognition, Machine Translation, and Text-To-Speech Systems that can be applied to transcribe, translate, and synthesize a media file.
/list	Get a list of all existing audio and videos in the Database.

Interfaces for downloading media and subtitle files:

/metadata	Get metadata and media file locations for a given media ID.
/langs	Get a list of all subtitle and audiotrack languages available for a given media ID.
/get	Download the current subtitle file for a given media ID and language.
/audiotrack	Download an audiotrack file for a given media ID and language.

Interfaces for editing subtitles:

/start_session	Starts an edition session to send and commit modifications of a subtitles file.
/session_status	Returns the current status of the given session ID.
/mod	Send and commit subtitle corrections under an edit session.
/align	Request automatic alignments of long manual speech transcripts with the corresponding audio.
/end_session	Ends an open edition session, and depending on the confidence of the user, editions are directly stored in the corresponding subtitles files or left for revision.

Interfaces for managing subtitles' user revisions:

/lock	Allow/disallow regular users to send subtitles modifications for an specific Media ID.
/edit_history	Returns a list of all edit sessions that involved an specific media ID.
/revisions	Returns a list of all edit sessions for all API user’s media files that are pending to be revised.
/mark_revised	Mark/unmark as revised an specific edit session ID, typically from another Session ID on the TLP Player.
/accept	Accept modifications of one or more pending edit sessions without having to revise them. Modifications are commited into the corresponding subtitles files.
/reject	Reject modifications of one or more pending edit sessions without having to revise them.

/speech/ingest

This interface allows the client to upload media files to the platform so they can be automatically transcribed and translated into several languages by the Ingest Service. The uploaded data (media, slides, documents, etc.) are bundled into a non-compressed ZIP file called Media Package (MPF). The Web Service stores the Media Packages in the server and returns an Upload ID, which can be used afterwards to check the upload progress via the /status interface.

If you are developing your own API client, it is recommended to enable the Test Mode of the Ingest Service when performing call tests to this interface. Please refer to the Manifest JSON File Specification.

The /ingest interface is split into four sub-interfaces, every one devoted to implement the four operation types supported by the Ingest Service:

/new → New media operation.
/update → Update media operation.
/delete → Delete media operation.
/cancel → Cancel upload operation.

Every sub-interface has different input specifications, please refer to each sub-section. Particularly, the /new and /update operations require as input a base64-encoded Manifest JSON string (please see Manifest JSON sub-section). In case any media files have to be uploaded to the TLP Server, those files have to be declared in the Manifest JSON string, and attached to the HTTP query as a Media Package file (please see Media Package sub-section). Finally, all sub-interfaces share the same output specifications, please see the Output sub-section.

Manifest JSON

The Manifest JSON is a JSON string that provides several metadata required by the /new and /update operations. For example, on new operations it must describe media attributes such as ID, title or language. Also, it is used to tell the Ingest Service that one or more media files such as a video file or a slides file have been uploaded, bundled into a Media Package file.

Manifest file JSON Specification

{
  "media" : {
              "filename" : <str> ,
              "fileformat" : <str> ,
              "md5" : <str> ,
              "slides" : <bool> ,
              "url" : <str>
            } ,
  "attachments" : [
                    {
                      "filename" : <str> ,
                      "fileformat" : <str> ,
                      "md5" : <str> ,
                      "type_code" : <str> ,
                      "language" : <str> ,
                      "human" : <bool>
                    },
                    ...
                  ] ,
  "metadata" : {
                 "external_id" : <str> ,
                 "language" : <str> ,
                 "title" : <str> ,
                 "topic" : <str> ,
                 "keywords" : <str> ,
                 "date" : <str> ,
                 "speakers" : [
                                {
                                  "speaker_id" : <int> ,
                                  "speaker_name" : <str> ,
                                  "speaker_gender" : <str> ,
                                  "speaker_email" : <str>
                                } ,
                                ...
                              ]

               },
  "requested_langs": <dict> ,
  "generate" : <bool> ,
  "re-generate": [ <str>, ...] ,
  "force": <int> ,
  "test_mode" : <bool>,
  "email": [ <str>, ...]
}

media:<dict> → Main media file to be transcribed and/or translated. <dict> keys:
- filename:<str> → File name of the main media file.
- fileformat:<str> → Format of the main media file (see Allowed attachments).
- md5:<str> → MD5 checksum of the main media file.
- slides:<bool> → Use media file also as a video slides file (optional, default false).
- url:<str> → URL to the main media file. If a url field is given, fields filename, fileformat and md5 are ignored.
metadata:<dict> → Media metadata. <dict> keys:
- external_id:<str> → Media ID (typically an internal ID in the client’s media repository database) used to identify the video in further queries to the Web Service.
- title:<str> → Title of the media.
- language:<str> → Media language code in ISO 639-1 format (e.g. "en", "es").
- speakers:<list:dict> → Information about the speaker(s) of the media. <dict> keys:
  - speaker_id:<int> → Speaker ID (client).
  - speaker_name:<str> → Full name of the speaker.
  - speaker_email:<str> → E-mail of the speaker (optional).
  - speaker_gender:<str> → Gender of the speaker (optional).
    
    M → Male.
    
    F → Female.
- topic:<str> → Topic of the media (optional).
- keywords:<str> → Media keywords (optional).
- date:<str> → Publication date of the media (optional).
attachments:<list:dict> → Additional files that have been attached to the media package, such as slides, related documents or subtitles. <dict> keys:
- filename:<str> → File name of the attachment.
- fileformat:<str> → Format of the attachment (see Allowed attachments).
- md5:<str> → MD5 checksum of the attachment.
- type_code:<int> → Attachment type code (see Allowed attachments).
  - 0 → Media file.
  - 1 → Slides file.
  - 2 → Related text document file.
  - 3 → Video Snapshot/Thumbnail file.
  - 4 → Subtitles file.
  - 5 → Audiotrack file.
- language:<str> → Language of the attachment, in case it is a subtitles file, in ISO 639-1 format (e.g. "en", "es") (optional, but mandatory if type_code=4).
- human:<bool> → If the attachment is a subtitles file, determine if provided subtitles have been generated by humans (optional, default true).
requested_langs:<dict> → Explicit request of subtitles and audiotrack languages, along with some advanced options. Please see Requesting subtitle languages (optional).
generate:<bool> → Enable/disable transcription and translation technologies (optional, default true).
re-generate:<list:str> → On update operation: request a regeneration of translations and/or synthesized audiotracks. Must be a list of keywords (optional). Allowed Keywords:
- tx → Request regeneration of the media transcription.
- tl → Request regeneration of media translations.
- tts → Request regeneration of synthesized audiotracks.
force:<bool> → On update operation: Force regeneration of automatic subtitles even if there exist human-supervised subtitles (optional, default false).
test_mode:<bool> → Enable/Disable the Test Mode of the Ingest Service. When enabled, the uploaded media will be available inmediately with an empty subtitles file. This feature is very useful when executing integration tests with the /ingest interface (optional, default false).
email:<list:str> → List of emails to send notifications about the status of the upload (optional).

Requesting Subtitle Languages

The requested_langs option, as stated before, is used to request additional subtitle languages, specifying advanced transcription, translation or text-to-speech options. requested_langs is a JSON dictonary in which keys are ISO 639-1 language codes (e.g. "en", "es"), and values are dictionaries in which keys are the objects or outputs requested to be generated for that particular language, and values are dictionaries with advanced options. Those keys (objects) might be:

sub:<dict> → Generate subtitles for that specific language.
tts:<dict> → Generate text-to-speech audiotracks for that specific language.

sub and tts values can be empty dictionaries ({}) or null values, if no advanced options are specified.

 "requested_langs": {
    "es": {
       "sub": {}
     },
    "en": {
       "tts": {},
       "sub": {}
     }
 }

The example above means: "Generate Spanish and English subtitles and a English TTS audiotrack, using default options in all cases".

Advanced options:

sub options:
- sid:<int> → Specify which System will be applied to generate the transcription, translation, or audiotrack file. If not specified, the default system is used.
- lma:<bool> → Only for transcriptions (ASR): Enable or disable Language Model Adaptation. By default it is enabled.
- tlpath:<list> → Only for translations (MT): Explicitly declare a Translation Path. This is useful to generate translations to a language which is not featured directly from the spoken language, using intermediate translation languages. It is declared as an ordered list of dictionaries, where each dictionary specifies the target language code l of the step, and optionally the System ID sid to apply.
  
  The example below shows how to request Catalan (ca) subtitles from the spoken language (XX) using English (en) and Spanish (es) as intermediate languages, thus defining the following translation path: XX->En->Es->Ca. The intermediate En->Es translation is generated using System ID 3.
  "requested_langs": { "ca": { "sub": { "tlpath": [ { "l":"en" }, { "l":"es", "sid":3}, { "l":"ca"} ] } } }
  If this option is not specified, the Ingest Service will assume that a direct translation from the spoken language is requested.
tts options:
- sid:<int> → Specify which System ID will be applied to generate the synthesized audiotrack. If not specified, the default system is used.

The following example of the requested_langs option requests Estonian (et) subtitles disabling the Language Model Adaptation feature and making use of the System ID 22, as well as English (en) subtitles with default options and a synthesized English audiotrack using System ID 54.

 "requested_langs": {
    "et": {
       "sub": { "lma":false, "sid":22 }
     }
    "en": {
       "sub": { }
       "tts": { "sid":54 }
     }
 }

Allowed attachments

Below follows a table in which are listed the attachents and file formats that are typically allowed to be uploaded to a standard Ingest Service.

Table 1. Allowed Media Package file attachments by file type
Type Code ID	Type Code Name	Allowed File Format List
0	Media (video)	mp4, m4v, ogv, wmv, avi, mpg, flv.
0	Media (audio)	wav, mp3, oga, flac, aac.
1	Slides (text) [Sorted by descending text extraction quality.]	txt, ppt, pptx, doc, docx, pdf.
1	Slides (video) [For the best quality, a video slides file must be a video in which only the slides appear. Text from the slides is obtained using an OCR system.]	mp4, m4v, ogv, wmv, avi, mpg, flv.
2	Documents [Sorted by descending text extraction quality]	txt, doc, docx, ppt, pptx, pdf.
3	Video Thumbnail	jpg.
4	Subtitles	dfxp, srt, trs.
5	Audiotracks	wav, mp3, oga, flac, aac.

Media Package

A Media Package File is an uncompressed ZIP file that contains several media files and attachments declared in the Manifest JSON data sent to the /ingest/new or /ingest/update interfaces. All files must be placed in the root on the ZIP package (not inside folders or sub-folders).

Output

All four sub-interfaces share a common output specification, described as follows:

JSON Object: Content-Type = application/json

 {
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "id" : <str> ,
  "hash" : <str>
 }

rcode:`<int>`	Return code. `0` → Upload completed.
rcode_description:`<str>`	Description of the return code (rcode).
id:`<str>`	Upload ID, which can be used afterwards to check the progress of the upload via the /status interface.
hash:`<str>`	Internal media hash ID.

Output Examples

Media Package File successfully uploaded.

 {
  "rcode" : 0,
  "rcode_description" : "Ingestion complete",
  "id" : "UPLOAD-ID-1234"
  "hash" : "433e11c295c51b94a074a"
 }

/speech/ingest/new

New media operation.

Input

In this operation only the POST method is allowed (Content-Type: multipart/form-data). Input data is divided in two fields:

json: Query parameters encoded in a JSON string (Content-Type: application/json)

Parameter name	Type	Required	Description	Example
manifest	json	Yes	Manifest JSON. See particular specifications for this operation.	`[See this example]`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

pkg (optional): A Media Package zip file (Content-Type: application/zip).

Manifest JSON data

Below is described how to properly set up a Manifest JSON string for New media operations.

Required inputs:

A media file to be transcribed and/or translated in the media section.
Required metadata keys:
- external_id → Media ID in the remote (client) repository.
- title → Title of the Media. It must be as descriptive as possible, since it might be used to search and download from the Internet related documents in order to adapt the ASR system to the topic of the video and enhance the quality of the automatic subtitles.
- language → Spoken language of the media file, in ISO 639-1 format (e.g. "en", "es").
- speakers → Speaker(s) info.

Behavior changes with optional inputs

requested_langs option:
- Not provided: By default the media only will be transcribed, if possible and if needed (depending on the provided attachments). No translation nor TTS processes are launched.
- Provided: Actions that can be inferred from this option will be executed, unless the expected outputs are already provided in the attachments section.
attachments section:
- Language Model Adaptation on transcription depending on provided textual attachments: If Language Model Adaptation is enabled for the ASR System that generates the transcription file, this ASR system will be adapted to the topic of the media using different textual resources:
  - No text attachments provided: The adaptation will be carried out using external resources automatically downloaded from the Internet based on a web search using the title of the media.
  - A slides file: The adaptation will be carried out using the text extracted from the slides file.
  - Related documents: The adaptation will be carried out using the text extracted from the attached documents.
  - A slides file + Related documents: The adaptation will be carried out using both the text extracted from the slides file and the text extracted from the attached documents.
- Providing expected outputs:
  - Subtitles file (spoken language): Media won’t be transcribed. Subtitles will be translated and afterwards synthesized if explicitly requested in the requested_langs option.
  - Subtitles files (other languages): Media will be firstly transcribed, and then translated into other destination languages, except for the provided subtitles language, if explicitly requested in the requested_langs option. Synthesized audiotracks will be also generated if explicitly requested.
  - Subtitles files (spoken language + other languages): The media will be translated into the remaining available destination languages, if any and if explicitly requested in the requested_langs option. Synthesized audiotracks will be generated if explicitly requested.
  - Audiotrack files: Synthesized audiotracks for the languages of the attached audiotracks won’t be generated even if they were explicitly requested.

Call examples

Upload a new media using a public URL to a video file. English subtitles to be generated by default.

JSON string:

 {
    "user": "tluser",
    "auth_token": "edbab44c3a3f1ca8db4de8277a3b",
    "manifest": {

     "media": {
         "url": "http://my-media-server.com/path/to/my/video.mp4"
     },

     "metadata": {
         "external_id": "9abc7230fe36a18b885c",
         "language": "en",
         "title": "To know something or not to know nothing: A brief essay about knowledge.",
         "speakers": [
             {
                 "speaker_name": "Kit Harington",
                 "speaker_id": "kit1234"
             }
         ]
     }
   }
 }

HTTP Call to /ingest/new (note that a Media Package file is not required in this case).

http://my-tlp-server.com/api/speech/ingest/new

 + POST data (JSON string goes in the "json" field)

Upload a new media, requesting English and Spanish subtitles, and a Spanish synthesized audiotrack. A video-slides file is attached to enhance transcription subtitles quality.

JSON string:

 {
    "user": "tluser",
    "auth_token": "edbab44c3a3f1ca8db4de8277a3b",
    "manifest": {

      "media": {
          "fileformat": "mp4",
          "md5": "05b59346bc3fe5d3eac7a0dcd0022fb6",
          "filename": "main_media.mp4"
      },

      "attachments": [
          { "filename":"awesome_slides_in_video_format.wmv",
             "fileformat":"wmv",
             "type_code":1,
             "md5":"c8722d0e8e27d4b5caaa7122a14676e3"
          }
      ],

      "requested_langs": {
          "es": {
              "sub": {},
              "tts": {}
          },
          "en": {
              "sub": {}
          }
      },

      "metadata": {
          "external_id": "9abc7230fe36a18b885c",
          "language": "en",
          "title": "To know something or not to know nothing: A brief essay about knowledge.",
          "speakers": [
              {
                  "speaker_email": "kit@got.com",
                  "speaker_name": "Kit Harington",
                  "speaker_gender": "M",
                  "speaker_id": "kit1234"
              }
          ]
      }
   }
 }

HTTP Call to /ingest/new:

http://my-tlp-server.com/api/speech/ingest/new

 + POST data (JSON string goes in the "json" field, Media Package file in the "pkg" field)

/speech/ingest/update

Update media operation.

Input

In this operation only the POST method is allowed (Content-Type: multipart/form-data). Input data is divided in two fields:

json: Query parameters encoded in a JSON string (Content-Type: application/json)

Parameter name	Type	Required	Description	Example
manifest	json	Yes	Manifest JSON. See particular specifications for this operation.	`[See this example]`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

pkg (optional): A Media Package zip file (Content-Type: application/zip).

Manifest JSON data

Below is described how to properly set up a Manifest JSON string for Update media operations.

Required inputs:

Required metadata keys:
- external_id → Media ID in the remote (client) repository.

Behavior changes with optional inputs

media section:
- Not Provided: the Ingest Service will re-generate transcriptions, translations and/or audiotracks depending on the provided attachments and other options (see below).
- Provided: the uploaded media is assumed to be a re-recording of the existing media; therefore a new transcription file needs to be generated. The Ingest Service will behave as described in the New Media operation. The only difference between both cases is that the video ID is kept. Old media file and subtitles are backed up.
requested_langs option:
- Not provided: By default the media only will be transcribed, if possible and if needed (depending on the provided attachments). No translation nor TTS processes are launched.
- Provided: Actions that can be inferred from this option will be executed, unless the expected outputs are already provided in the attachments section. Actions that involve the re-generation of subtitles that have already been edited or supervised by users won’t be executed, unless the force option is set to true.
re-generate option:
- tx: Transcription file will be automatically regenerated if not supervised before by a user, unless the force option is set to true.
- tl: Translation files will be automatically regenerated if not supervised before by a user, unless the force option is set to true.
- tts: Synthesized audiotracks will be automatically regenerated.
attachments section:
- Language Model Adaptation on transcription regeneration depending on provided textual attachments: If Language Model Adaptation is enabled for the ASR System that generates the transcription file, this ASR system will be adapted to the topic of the media using different textual resources:
  - No text attachments provided: The adaptation will be carried out using external resources automatically downloaded from the Internet based on a web search using the title of the media.
  - A slides file: The adaptation will be carried out using the text extracted from the slides file.
  - Related documents: The adaptation will be carried out using the text extracted from the attached documents.
  - A slides file + Related documents: The adaptation will be carried out using both the text extracted from the slides file and the text extracted from the attached documents.
- Providing outputs:
  - Subtitles file (spoken language): Current transcription file will be overwritten by the supplied one. Subtitles will be translated and afterwards synthesized if explicitly requested in the requested_langs option, or the re-generate option contains the token tl for translation and tts for synthesis.
  - Subtitles files (other languages): Current translation files (if existing) will be overwritten by the supplied ones. Other subtitles languages will be generated if explicitly requested in the requested_langs option, or the re-generate option contains the token tl. Synthesized audiotracks will be also generated if explicitly requested either with the requested_langs option or the re-generate option.
  - Audiotrack files: Synthesized audiotracks for the languages of the attached audiotracks won’t be generated even if they were explicitly requested.

Call examples

Update media by requesting a re-generation of translations and synthesized audiotracks.

JSON string:

 {
    "user": "tluser",
    "key": "edbab44c3a3f1ca8db4de8277a3b",
    "manifest": {
      "metadata": { "external_id":"9abc7230fe36a18b885c" },
      "re-generate": [ "tl", "tts" ]
    }
 }

HTTP Call to /ingest/update (note that no Media Package file is required).

http://my-tlp-server.com/api/speech/ingest/update

 + POST data (JSON string goes in the "json" field)

Update media by uploading a re-recording of the same media ID (attaching a new media file). English subtitles to be generated as default.

JSON string:

 {
    "user": "tluser",
    "auth_token": "edbab44c3a3f1ca8db4de8277a3b",
    "manifest": {

      "media": { "filename": "main_media_NEW_VERSION.mp4" ,
                  "fileformat": "mp4" ,
                  "md5": "a86fae8e7af6fd6786efa876fa0e4212"
                },

      "metadata":
         {
           "external_id": "9abc7230fe36a18b885c",
           "language": "en",
           "title": "To know something or not to know nothing: A brief essay about knowledge. (NEW VERSION)",
            "speakers": [
                {
                    "speaker_email": "kit@got.com",
                    "speaker_name": "Kit Harington",
                    "speaker_gender": "M",
                    "speaker_id": "kit1234"
                }
            ]
         }
    }
 }

HTTP Call to /ingest/update:

http://my-tlp-server.com/api/speech/ingest/update

 + POST data (JSON string goes in the "json" field, Media Package file in the "pkg" field)

/speech/ingest/delete

Delete media operation.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
mode	str	No	Delete mode. * `soft` → media will be marked as "deleted" in the database and its files will be backed up. * `hard` → all media and subtitle files stored will be deleted, as well as all related entries in the Database.	`hard`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 8. [/speech/ingest/delete] delete media ID (GET with multiple parameters).

http://my-tlp-server.com/api/speech/ingest/delete?id=MEDIA-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

/speech/ingest/cancel

Cancel upload operation.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Upload ID, returned by the /ingest interface.	`UPLOAD-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 9. [/speech/ingest/cancel] Cancel upload (GET with multiple parameters).

http://my-tlp-server.com/api/speech/ingest/cancel?id=UPLOAD-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

/speech/uploadslist

Returns a list of all user’s uploads.

Input

Parameter name	Type	Required	Description	Example
object_id	str	No	Get list of uploads involving the provided object ID (could be an Upload ID or a Media ID).	`MEDIA-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 10. [/speech/uploadslist] Get list of uploads.

http://my-tlp-server.com/api/speech/uploadslist?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
    {
        "id": <str>,
        "object_id": <str>,
        "status_code": <int>,
        "uploaded_on": <str>,
        "last_update": <str>
    },
    ...
]

id:`<str>`	Upload ID.
object_id:`<str>`	Object ID involved (could be an Upload ID or a Media ID).
status_code:`<int>`	Status code of the Upload. `0` → Video ingested, not processed yet. `1` → Processing Media Package file. `2` → Transcription in progress. `3` → Translation(s) in progress. `4` → Text-To-Speech audio track(s) generation in progress. `5` → Preparing data for the TLP Player. `6` → New media successfully processed. `20` → Media successfully updated. `30` → Media successfully deleted. `40` → Cancel upload operation successfully processed. `90` → Upload cancelled. `100` → An unknown error occurred. `101` → An error occurred while processing the Media Package. `102` → An error occurred while transcribing the media. `103` → An error occurred while translating the media. `104` → An error occurred while generating Text-To-Speech audio tracks. `105` → An error occurred while preparing data for the TLP Player. `106` → An error occurred while updating internal database.
uploaded_on:`<str>`	Upload timestamp.
last_update:`<str>`	Last upload check timestamp.

Output Examples

Uploads List returned.

[
    {
        "id": "up-ac83be70-a01c-4c18-8cc4-dc0b2676cbb0",
        "object_id": "MEDIA-ID-1234",
        "status_code": 2,
        "uploaded_on": "2015-06-10 17:19:37.239458",
        "last_update": "2015-06-10 17:20:02.557135"
    },
    {
        "id": "up-60a70bbd-e111-4d0c-b41f-6e235c434330",
        "object_id": "MEDIA-ID-1234",
        "status_code": 101,
        "uploaded_on": "2015-06-09 11:21:07.735656",
        "last_update": "2015-06-09 11:22:02.549826"
    },
    {
        "id": "up-776ae4ec-6904-4da1-afa8-1b68017a524a",
        "object_id": "MEDIA-ID-5678",
        "status_code": 6,
        "uploaded_on": "2015-06-10 17:25:04.673541",
        "last_update": "2015-06-10 17:26:02.542902"
    }
]

/speech/status

Returns information about the progress of an uploaded media given an Upload ID. It enables the remote repository to keep track of the automatic uploads and to notice possible processing errors.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Upload ID, returned by the /ingest interface.	`UPLOAD-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 11. [/speech/status] check status of an upload (GET with multiple parameters).

http://my-tlp-server.com/api/speech/status?id=UPLOAD-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "status_code" : <int> ,
  "status_info" : <str> ,
  "error_code" : <int> ,
  "uploaded_on" : <str> ,
  "last_update" : <str> ,
  "estimated_time_complete" : <str>
}

rcode:`<int>`	Return code. `0` → Upload ID exists. `1` → Upload ID does not exist.
rcode_description:`<str>`	Description of the return code (rcode).
status_code:`<int>`	Status code of the upload. `0` → Video ingested, not processed yet. `1` → Processing Media Package file. `2` → Transcription in progress. `3` → Translation(s) in progress. `4` → Text-To-Speech audio track(s) generation in progress. `5` → Preparing data for the TLP Player. `6` → New media successfully processed. `20` → Media successfully updated. `30` → Media successfully deleted. `40` → Cancel upload operation successfully processed. `90` → Upload cancelled. `100` → An unknown error occurred. `101` → An error occurred while processing the Media Package. `102` → An error occurred while transcribing the media. `103` → An error occurred while translating the media. `104` → An error occurred while generating Text-To-Speech audio tracks. `105` → An error occurred while preparing data for the TLP Player. `106` → An error occurred while updating internal database.
status_info:`<str>`	Detailed information about the status code.
error_code:`<int>`	Generic error code that identifies the operation that failed within the process, if any. Otherwise null.
uploaded_on:`<str>`	Upload timestamp.
last_update:`<str>`	Last status check timestamp.
estimated_time_complete:`<str>`	Estimated time to complete the whole process.

Output Examples

The Upload ID provided does not exist.

 {
   "rcode": 1,
   "rcode_description" : "Upload ID [ up-1234 ] does not exist."
 }

The provided upload ID exists and it’s being processed.

 {
   "rcode": 0,
   "rcode_description" : "Upload ID exists.",
   "status_code": 2,
   "info": "Transcription in progress. It may take several hours for it to finish.",
   "uploaded_on": "2014-03-26 19:02:16.174944",
   "last_update": "2014-03-26 19:03:05.298861"
 }

/speech/systems

Get a list of all available ASR/MT/TTS Systems that can be applied to transcribe/translate/synthesize a un uploaded media file using the /ingest interface.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 12. [/speech/systems] Get list of available systems.

http://my-tlp-server.com/api/speech/systems?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
    "asr": [
        {
            "lang": <str>,
            "id": <int>,
            "name": <str>,
            "description": <str>
        },
        ...
    ],
    "mt": [
        {
            "source_lang": <str>,
            "target_lang": <str>,
            "id": <int>,
            "name": <str>,
            "description": <str>
        },
        ...
    ],
    "tts": [
        {
            "lang": <str>,
            "id": <int>,
            "name": <str>,
            "description": <str>,
            "voice_gender": <str>
        },
        ...
    ]
]

asr:`<list:dict>`	List of all available Automatic Speech Recognition Systems. `<dict>` keys: lang:`<str>` → ASR System Language code (ISO-639-1). id:`<int>` → System ID. name:`<str>` → Name of the system. description:`<str>` → Description of the system.
mt:`<list:dict>`	List of all available Machine Translation Systems. `<dict>` keys: source_lang:`<str>` → MT System source language code (ISO-639-1). target_lang:`<str>` → MT System target language code (ISO-639-1). id:`<int>` → System ID. name:`<str>` → Name of the system. description:`<str>` → Description of the system.
tts:`<list:dict>`	List of all available Text-To-Speech Systems. `<dict>` keys: lang:`<str>` → TTS System Language code (ISO-639-1). id:`<int>` → System ID. name:`<str>` → Name of the system. description:`<str>` → Description of the system. voice_gender:`<str>` → Gender of the voice that produces this system.

The system ID might be used to explicitly request to the Ingest Service the application of a particular ASR/MT/TTS system. For further information please see Requesting Subtitle Languages).

Output Examples

Systems List returned.

{
    "asr": [
        {
            "lang": "en",
            "id": 43,
            "name": "English ASR System",
            "description": ""
        },
        {
            "lang": "es",
            "id": 22,
            "name": "Spanish ASR System",
            "description": ""
        },
        {
            "lang": "ca",
            "id": 64,
            "name": "Catalan ASR System",
            "description": ""
        }
    ],
    "mt": [
        {
            "source_lang": "ca",
            "target_lang": "es",
            "id": 14,
            "name": "Catalan-Spanish MT System",
            "description": ""
        },
        {
            "source_lang": "es"
            "target_lang": "ca",
            "id": 11,
            "name": "Spanish-Catalan MT System",
            "description": ""
        },
        {
            "source_lang": "en",
            "target_lang": "es",
            "id": 73,
            "name": "English-Spanish MT System",
            "description": ""
        },
        {
            "source_lang": "es"
            "target_lang": "en",
            "id": 24,
            "name": "Spanish-English MT System",
            "description": ""
        }
    ],
    "tts": [
        {
            "lang": "en",
            "id": 71,
            "name": "English TTS System (Female)",
            "description": "",
            "voice_gender": "f"
        }
    ]
}

/speech/list

Get a list of all existing audios and videos in the TLP Database.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 13. [/speech/list] Get list of all existing audios and videos.

http://my-tlp-server.com/api/speech/list?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
    {
        "id": <str>,
        "title": <str>,
        "language": <str>,
        "duration": <int>
    },
    ...
]

id:<int> → Media ID.
title:<str> → Media’s title.
language:<str> → Media language code (ISO-639-1).
duration:<int> → Media length in seconds.

Output Examples

Media list returned.

{
    {
        "id": "MEDIA-ID-1234",
        "title": "I do know nothing",
        "language": "en",
        "duration": 86
    },
    {
        "id": "MEDIA-ID-6789",
        "title": "Tyrion Lannister and Daenerys Targaryen: the Royal Wedding.",
        "language": "en",
        "duration": 271
    }
}

/speech/metadata

Returns metadata and media file locations of a given media ID. For example, this operation is called by the TLP Player to get the main media file location.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 14. [/speech/metadata] Get media metadata (GET with multiple parameters).

http://my-tlp-server.com/api/speech/metadata?id=MEDIA-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
  "rcode": <int> ,
  "rcode_description": <str> ,
  "mediainfo": {
                   "language" : <str> ,
                   "title" : <str> ,
                   "category" : <str> ,
                   "duration" : <str> ,
                   "speakers" : [
                                 {
                                   "name" : <str>
                                 } ,
                                 ...
                                ]
                 } ,
  "media": [
             {
               "is_url" : <bool> ,
               "type_code" : <int> ,
               "media_format" : <str> ,
               "location" : <str>
             } ,
             ...
           ],
  "audiotracks": [
        {
            "lang": <str>,
            "voice_type": <str>,
            "id": <int>,
            "location": <str>,
            "media_format": <str>,
            "audio_type": <str>,
            "is_url": <bool>,
            "sub_type": <int>,
            "description": <str>
        } ,
        ...
    ],
  "attachments": [
             {
               "is_url" : <bool> ,
               "type_code" : <int> ,
               "media_format" : <str> ,
               "location" : <str>
             } ,
             ...
           ]
 }

rcode:`<int>`	Return code of the WS call. `0` → Media ID exists. `1` → Media ID does not exist.
rcode_description:`<str>`	Description of the return code (rcode).
mediainfo:`<dict>`	Media Metadata. `<dict>` keys: language:`<str>` → Language of the media. title:`<str>` → Title of the media. duration:`<int>` → Media duration (in seconds). speakers:`<list:dict>` → Info about the media’s speaker(s). `<dict>` keys: name:`<str>` → Name of the speaker.
media:`<list:dict>`	List of media files available. `<dict>` keys: is_url:`<bool>` → Defines whether location is an URL or not. If not, it is a relative physical path. type_code:`<int>` → File type code. `0` → Main media file (video or audio). `3` → Small video thumbnail `6` → PCM stream used to draw waveform media_format:`<str>` → Format of the attachment (i.e. mp4, ogv, …) location:`<str>` → Relative path or URL in which the file is located.
audiotracks:`<list:dict>`	List of available audiotracks. `<dict>` keys: id:`<int>` → Audiotrack ID. lang:`<str>` → Language of the audiotrack. location:`<str>` → Relative path or URL in which the file is located. is_url:`<bool>` → Defines whether location is an URL or not. If not, it is a relative physical path. media_format:`<str>` → Audio format (i.e. mp3, wav, …) voice_type:`<str>` → Voice type. Possible values: `tts` → Text-To-Speech, synthesized audiotrack `rec` → Human-recorded audiotrack. voice_gender:`<str>` → Voice gender. Possible values: `m` → Male voice. `f` → Female voice. sub_type:`<int>` → If voice_type is `tts` and the audiotrack was automatically generated form a subtitle file, it shows the corresponding subtitle type code. It defines the level of human supervision of the subtitles that generated the synthesized audiotrack. `0` → Fully Automatic: The whole subtitles are automatic. `1` → Partially Human: The subtitles have been supervised, but not for the whole video. `2` → Fully Human: The subtitles have been fully supervised by humans. description:`<str>` → A brief text description (if any) of the audiotrack.
attachments:`<list:dict>`	List of attachments available. `<dict>` keys: is_url:`<bool>` → Defines whether location is an URL or not. If not, it is a relative physical path. type_code:`<int>` → File type code. `1` → Slides file (video, ppt, pdf, txt) `2` → Related documents (ppt, pdf, txt) media_format:`<str>` → Format of the attachment (i.e. mp4, ogv, …) location:`<str>` → Relative path or URL in which the file is located.

Output Examples

Media ID does not exist.

 {
  "rcode" : 1,
  "rcode_description" : "Media ID [ 1234-abcd ] does not exist or has no media"
 }

Media ID exists.

  {
    "rcode": 0,
    "rcode_description": "Media list and info available.",
    "attachments": [
        {
            "is_url": true,
            "type_code": 1,
            "media_format": "txt",
            "location": "http://my-tlp-server.com/data/9/9bc70b33e49c2/b3f0bee253651191cdd1f1ee6c865074.txt"
        },
        {
            "is_url": true,
            "type_code": 2,
            "media_format": "txt",
            "location": "http://my-tlp-server.com/data/9/9bc70b33e49c2/d41d8cd98f00b204e9800998ecf8427e.txt"
        }
    ],
    "media": [
        {
            "is_url": true,
            "type_code": 0,
            "media_format": "mp4",
            "location": "http://my-tlp-server.com/data/9/9bc70b33e49c2/1a4edf93069b3.mp4"
        },
        {
            "is_url": true,
            "type_code": 3,
            "media_format": "jpg",
            "location": "http://my-tlp-server.com/data/9/9bc70b33e49c2/d7bbbe4210bd3.jpg"
        },
        {
            "is_url": true,
            "type_code": 6,
            "media_format": "pcm",
            "location": "http://my-tlp-server.com/data/9/9bc70b33e49c2/75bae29f33670.pcm"
        }
    ],
    "lectureinfo": {
        "duration": 86,
        "speakers": [
            {
                "name": "John Snow"
            }
        ],
        "language": "en",
        "title": "I do know nothing"
    },
    "audiotracks": [
        {
            "lang": "es",
            "voice_type": "tts",
            "id": 68,
            "location": "http://my-tlp-server.com/data/9/9bc70b33e49c2/audiotrack.es.mp3",
            "media_format": "mp3",
            "audio_type": null,
            "is_url": true,
            "sub_type": 2,
            "description": null
        }
    ]
  }

/speech/langs

Returns list of subtitle languages available for a specific media ID.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 15. [/speech/langs] Get list of subtitle languages available (GET with multiple parameters).

http://my-tlp-server.com/api/speech/langs?id=MEDIA-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "media_lang" : <str> ,
  "locked" : <bool> ,
  "langs" : [
              {
                "lang_code" : <str> ,
                "lang_name" : <str> ,
                "sup_status" : <int> ,
                "sup_ratio" : <float> ,
                "rev_time" : <int> ,
                "rev_rtf" : <float> ,
                "system_time" : <int> ,
                "system_rtf" : <float> ,
                "audiotracks": [
                                 {
                                   "aid": <int>,
                                   "voice_gender": <str>,
                                   "voice_type": <str>
                                 },
                                 ...
                               ]

              },
              ...
            ]
 }

rcode:`<int>`	Return code of the WS call. `0` → Media ID exists. `1` → Media ID does not exist.
rcode_description:`<str>`	Description of the return code (rcode).
media_lang:`<str>`	Language code (ISO-639-1) of the media’s original audio language.
locked:`<bool>`	Lock status of subtitles. If `true`, only authors are allowed to send subtitle modifications.
langs:`<list:dict>`	List of languages available. `<dict>` keys: lang_code:`<str>` → Subtitle language code (ISO-639-1). lang_name:`<str>` → Local language name. sup_status:`<int>` → Subtitle supervision status code. Defines the level of human supervision of the subtitles. `0` → Fully Automatic: The whole subtitles are automatic. `1` → Partially Human: The subtitles have been supervised, but not for the whole video. `2` → Fully Human: The subtitles have been fully supervised by humans. sup_ratio:`<float>` → Percentage of supervised segments. rev_time:`<int>`: Time spent by the user while editing the subtitles. rev_rtf:`<int>`: RTF spent by the user while editing the subtitles. system_time:`<int>`: Time spent by the ASR/MT system to generate the subtitles. system_rtf:`<int>`: RTF spent by the ASR/MT system to generate the subtitles. audiotracks:`<list:dict>` → List of audiotracks available in the corresponding language. `<dict>` keys: voice_gender:`<str>` → Voice gender. `m` → Male voice. `f` → Female voice. aid:`<int>` → Audiotrack ID. voice_type:`<str>` → Voice type. `tts` → Text-To-Speech, synthesized audiotrack `rec` → Human-recorded audiotrack.

Output Examples

Media ID does not exist or has no subtitles.

 {
  "rcode" : 1 ,
  "rcode_description" : "ID 1234-abcd does not exist or has no subtitles" ,
 }

Media ID exists and has subtitles available.

{
    "rcode": 0,
    "rcode_description": "Language list available.",
    "media_lang": "es",
    "locked": false,
    "langs": [
        {
            "lang_code": "es",
            "sup_status": 1,
            "sup_ratio": 61.1650485436893,
            "rev_time" : 682 ,
            "rev_rtf" : 1.2 ,
            "system_time" : null ,
            "system_rtf" : null ,
            "lang_name": "Español",
            "audiotracks": []
        },
        {
            "lang_code": "ca",
            "sup_status": 0,
            "sup_ratio": 0,
            "rev_time" : 0 ,
            "rev_rtf" : 0 ,
            "system_time" : null ,
            "system_rtf" : null ,
            "lang_name": "Català",
            "audiotracks": []
        },
        {
            "lang_code": "en",
            "sup_status": 2,
            "sup_ratio": 100,
            "rev_time" : 923 ,
            "rev_rtf" : 1.6 ,
            "system_time" : null ,
            "system_rtf" : null ,
            "lang_name": "English",
            "audiotracks": [
                {
                    "aid": 12,
                    "voice_gender": "f",
                    "voice_type": "tts"
                }
            ]
        }
    ]
}

/speech/get

Returns subtitles for a specific media ID and language. By default, subtitles are sent in DFXP format, although they can be retrieved in many other formats.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
lang	str	Yes	Language code (ISO 639-1).	`en`
format	int	No	Subtitles format. `0` → transLectures-extended DFXP format (default). `1` → Non-extended DFXP format. `2` → SRT format. `3` → WebVTT format. `4` → Plain Text.	`2`
session_id	int	No	Load subtitles modifications from the given session ID (if any). If format=0, modified segments will include the highlight attribute (`h="1"`).	`327`
seg_filt_policy	int	No	Segment text filtering policy. `-1` → Empty all segments (removes text from all segments). `0` → Filtering disabled. `1` → Filters out special annotations (default).	`1`
sel_data_policy	int	No	Subtitle contents to be returned. `0` → Return both former and current contents (only for DFXP format). `1` → Return former contents (automatic transcription or translation). `2` → Return current contents (current status of supervision of the subtitles) (default).	`2`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 16. [/speech/get] Get English subtitles in srt format for media ID "MEDIA-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/speech/get?id=MEDIA-ID-1234&lang=en&format=2&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

Subtitles file:

DFXP file: Content-Type = application/ttml+xml
SRT file: Content-Type = application/x-subrip
VTT file: Content-Type = text/vtt
TXT file: Content-Type = text/plain

JSON Object (if media ID or subtitles file does not exist):

Content-Type = application/json

 {
  "rcode" : <int> ,
  "rcode_description" : <int>
 }

rcode:`<int>`	Return code. `1` → Media ID does not exist or subtitles in the specified language do not exist. `2` → The provided Session ID does not exist. `3` → The provided Session ID does not involve the given Media ID.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Media ID does not exist.

 {
  "rcode" : 1 ,
  "rcode_description" : "Media ID [MEDIA-ID-1234] does not exist"
 }

Returning English subtitles in srt format.

1
00:08:44,090 --> 00:08:48,680
Good morning, my name is Kit Harington, but people is used to call me John Snow.

2
00:08:48,680 --> 00:08:51,850
Winter is coming, isn't it?

/speech/audiotrack

Sends in binary format an audiotrack file.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
lang	str	Yes	Language code (ISO 639-1).	`en`
aid	int	Yes	Audiotrack ID (from /metadata).	`428`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 17. [/speech/audiotrack] Download audiotrack ID 428 in English language from media ID MEDIA-ID-1234 (GET with multiple parameters).

http://my-tlp-server.com/api/speech/audiotrack?id=MEDIA-ID-1234&lang=en&aid=428&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

Audiotrack file:

A Content-Type different than application/json, typically audio/mpeg (mp3) or audio/wav (wav)

JSON Object (if media ID or audiotrack ID does not exist):

Content-Type = application/json

 {
  "rcode" : <int> ,
  "rcode_description" : <str>
 }

rcode:`<int>`	Return code. `1` → Media ID does not exist or audiotrack ID does not exist.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Media ID does not exist.

 {
  "rcode" : 1 ,
  "rcode_description" : "Media ID [MEDIA-ID-1234] does not exist"
 }

/speech/start_session

Starts an edition session to send and commit modifications of a subtitles file. Edition sessions are a mechanism devoted to avoid race conditions between different users when editing a subtitles file.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
type	str	No	Session type. Allowed values are: `player` (default), `transcriber`.	`player`
author_id	str	Yes	Author ID, authorised by the API client, that starts the edition session.	`jsnow21`
author_name	str	No	Author Name	`John Snow`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 18. [/speech/start_session] Start an edition session for media ID "MEDIA-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/speech/start_session?id=MEDIA-ID-1234&type=player&author_id=jsnow21&author_conf=90&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
    "rcode": <int>,
    "rcode_description": <str>,
    "session_id": <int>,
    "session_type": <str>,
    "author_id": <str>,
    "author_name": <str>,
    "author_conf": <int>,
    "author_type": <str>,
    "started_at": <str>,
    "last_update": <str>
 }

rcode:`<int>`	Return code. `0` → Session successfully started, restarted, or continued. Returns information about the session. `1` → Media ID does not exist. `2` → Media ID has no subtitles. `3` → Subtitles for the Media ID have been locked. Only authors are allowed to modify them. `4` → Could not start session due to the existence of an started session by another author ID for the same Media ID. Returns information about the existing session. `5` → Could not start session due to the existence of an started session of another type.
rcode_description:`<str>`	Description of the return code (rcode).
session_id:`<int>`	Session ID.
session_type:`<int>`	Session type.
author_id:`<str>`	Author ID that started the session and currently editing the Media ID.
author_conf:`<int>`	Confidence level of the author ID, from 0 to 100.
author_name:`<int>`	Author Name.
author_type:`<str>`	Author Type. `human` → Session ID is being carried by a human. `computer` → Session ID has been opened by the Ingest Service, in order to automatically regenerate subtitles and audiotracks.
started_at:`<str>`	Session start timestamp.
last_update:`<str>`	Timestamp of the last update (/mod call) made by the user on the session.

Output Examples

Session started.

 {
    "rcode": 0,
    "rcode_description": "Session started.",
    "session_id": 8,
    "session_type": "player",
    "author_id": "jsnow21",
    "author_name": "John Snow",
    "author_conf": 90,
    "author_type": "human",
    "started_at": "2015-07-04 20:44:55.042786",
    "last_update": "2015-07-04 20:44:55.042786",
 }

Another user is editing the same media ID.

 {
    "rcode": 4,
    "rcode_description": "Cannot start mod session: there exist an open mod session for the given media ID.",
    "session_id": 7,
    "session_type": "player",
    "author_id": "olly666",
    "author_name": "Olly",
    "author_conf": 100,
    "author_type": "human",
    "started_at": "2015-07-04 12:23:15.040186",
    "last_update": "2015-07-04 20:32:11.892843"
 }

/speech/session_status

Returns the current status of the given session ID. If it is alive, it updates the last alive timestamp (last_update output key). This interface is commonly used to avoid the automatic end of session due to user inactivity.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, owner of the session_id.	`jsnow21`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
alive	int	No	Alive message type. `0` → Alive and user update (default). `1` → Only alive.	1
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 19. [/speech/session_status] Check status of session ID 8 (GET with multiple parameters).

http://my-tlp-server.com/api/speech/session_status?id=MEDIA-ID-1234&session_id=8&author_id=jsnow21&author_conf=90&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
    "rcode": <int>,
    "rcode_description": <str>,
    "started_at": <str>,
    "last_update": <str>,
    "last_alive": <str>,
    "ended_at": <str>,
    "ended_by_id": <str>,
    "ended_by_name": <str>,
    "ended_by_type": <str>
}

rcode:`<int>`	Return code. `0` → Session alive. `1` → Session ID does not exist. `2` → The provided Session ID does not involve the given Media ID. `3` → Not allowed to check the status of the given session ID. `4` → Session ID is not open (another user forced its closure, or was automatically closed after several hours of inactivity).
rcode_description:`<str>`	Description of the return code (rcode).
started_at:`<str>`	Session start timestamp.
last_update:`<str>`	Session last update timestamp.
last_alive:`<str>`	Session last alive timestamp.
ended_at:`<str>`	Session end timestamp. Will be `null` unless rcode`=4`.
ended_by_id:`<str>`	Author ID of the user that ended this session. Will be `null` unless rcode`=4`.
ended_by_name:`<str>`	Author name of the user that ended this session. Will be `null` unless rcode`=4`.
ended_by_type:`<str>`	Author type of the user that ended this session. Will be `null` unless rcode`=4`. `human` → Session ID is was closed by a human. `computer` → Session ID was automatically closed due to a large period of inactivity.

Output Examples

Session alive.

{
    "rcode": 0,
    "rcode_description": "Session alive.",
    "started_at": "2015-07-04 20:44:55.042786",
    "last_update": "2015-07-04 21:03:23.017192",
    "last_alive": "2015-07-04 21:03:51.932841",
    "ended_at": null,
    "ended_by_id": null,
    "ended_by_name": null
    "ended_by_type": null
}

Session closed.

{
    "rcode": 4,
    "rcode_description": "Session ID 8 is closed.",
    "started_at": "2015-07-04 20:44:55.042786",
    "last_update": "2015-07-04 20:44:55.042786",
    "last_alive": "2015-07-04 20:45:03.016531",
    "ended_at": "2015-07-04 22:28:51.075672",
    "ended_by_id": "lord_stark1",
    "ended_by_name": "Eddard Stark",
    "ended_by_type": "human"
}

/speech/mod

Send and commit modifications of subtitles files made by a user under a session ID returned by /start_session interface.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, owner of the session_id.	`jsnow21`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
mods	json	Yes	JSON Dictionary containing as many key-values as subtitle languages has been modified, being keys a ISO 639-1 language code of the subtitle languages edited, and values a dictionary containing the following keys: del:`<list:int>` → List of segment IDs that have been deleted. txt:`<list:dict>` → List of segment IDs that have been modified. `<dict>` keys: sI:`<int>` → Segment ID modified. b:`<float>` → Segment begin/start time. e:`<float>` → Segment end time. t:`<str>` → Text of the specified segment ID. aI:`<str>` → Align ID (optional), returned by the /speech/align/new interface. rev_time:`<int>` → Revision time, spent by the user to review these subtitles. Note: When using the GET method with multiple GET parameters, this JSON object must be encoded in Base64.	`{ "en": {"txt":[ {"sI":16, "b":87.96, "e":91.37, "t":"She told me: You know nothing, John Snow"} ], "del":[3,7], "rev_time":27}, "es": {"del":[9], "txt":[], "rev_time":4} }`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 20. [/speech/mod] Send modifications of the English and Spanish subtitles (see mods parameters example above) of media ID "MEDIA-ID-1234" under session ID 8 (GET with multiple parameters).

http://my-tlp-server.com/api/speech/mod?id=MEDIA-ID-1234&session_id=8&language=en&author_id=jsnow21&author_conf=90&mod=eyAiZW4iOiB7InR4dCI6WyB7InNJIjoxNiwgImIiOjg3Ljk2LCAiZSI6OTEuMzcsICJ0IjoiU2hlIHRvbGQgbWU6IFlvdSBrbm93IG5vdGhpbmcsIEpvaG4gU25vdyJ9IF0sICJkZWwiOlszLDddfSwgImVzIjogeyJkZWwiOls5XSwgInR4dCI6W119IH0=&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "details": [
        {
            "language": <str>,
            "rcode_description": <str>,
            "rcode": <int>
        },
        ...
  ]
}

rcode:`<int>`	Return code. `0` → Changes succesfully stored in the session. `1` → Session ID does not exist. `2` → The provided Session ID does not involve the given Media ID. `3` → Not allowed to send modifications under the given session ID. `4` → mods parameter is an empty dictionary. `5` → Format errors where found in all subtitle language modifications (see details key). `6` → Session ID is not open (another user forced its closure, or was automatically closed after several hours of inactivity). In addition, format errors where found in some subtitle language modifications (see details key). changes without errors are not commited to the edition session, but they are backuped. `7` → Session ID is not open (another user forced its closure, or was automatically closed after several hours of inactivity). Changes are not commited to the edition session, but they are backuped. `8` → Some subtitle language modifications were successfully commited, but some other failed (see details key).
rcode_description:`<str>`	Description of the return code (rcode).
details:`<array:dict>`	List of dicts for every subtitle language modifications if format errors were found (rcode`=5,6,8`).

Output Examples

Changes successfully saved.

 {
  "rcode" : 0 ,
  "rcode_description" : "Changes successfully saved."
 }

Session ID is not open.

 {
  "rcode" : 4 ,
  "rcode_description" : "Session ID [ 8 ] is closed, changes have been backuped. Please contact system administrator."
 }

Not all subtitle modifications were saved.

 {
    "rcode_description": "Modifications for some subtitle languages were successfully saved, but other failed. Please see 'details' key.",
    "rcode": 8,
    "details": [
        {
            "rcode_description": "The given media ID does not have 'en' subtitles.",
            "rcode": 10,
            "language": "en"
        },
        {
            "rcode_description": "Changes successfully saved.",
            "rcode": 0,
            "language": "es"
        }
    ]
 }

/speech/end_session

Ends an open edition session. Depending on the confidence of the user, editions are directly stored in the corresponding DFXP files or are left for revision.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, that closes the session.	`jsnow21`
author_name	str	No	Author Name	`John Snow`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
force	int	No	Force end session when user and/or author_id are not the owners. `0` → Do not force end session (default). `1` → Force end session.	1
regenerate	int	No	Request regeneration of subtitles and/or synthesized audiotracks immediately after closing the session. `0` → Do not request regeneration of susbtitles and/or synthesized audiotracks. `1` → Request regeneration of susbtitles and/or synthesized audiotracks (default).	0
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 21. [/speech/end_session] End the edition session ID 8 for media ID "MEDIA-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/speech/end_session?id=MEDIA-ID-1234&session_id=8&author_id=jsnow21&author_conf=90&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str>
}

rcode:<int>

Return code.

0 → Session successfully closed or already closed before.
1 → Session ID does not exist.
2 → The provided Session ID does not involve the given Media ID.
3 → Session ID is owned by the Ingest Service (re-generation of subtitles and audiotracks), and it cannot be ended manually.
4 → Unallowed end session request (different user, author_id and/or author_conf parameters; force parameter, if provided, was even insufficent).

rcode_description:<str>

Description of the return code (rcode).

Output Examples

Session closed.

 {
    "rcode": 0,
    "rcode_description": "Session succesfully closed."
 }

/speech/lock

Allow/disallow regular users to send subtitles modifications for an specific Media ID.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
lock	int	Yes	Lock action: `1` → Lock. `0` → Unlock.	`1`
author_id	str	Yes	Author ID, authorised by the API client, owner of the session_id.	`jsnow21`
author_conf	int	Yes	Confidence level of the author, from 0 to 100. Must be 100.	`100`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 22. [/speech/lock] Disallow modifications for subtitles of media ID "MEDIA-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/speech/lock?id=MEDIA-ID-1234&lock=1&author_id=lord_stark1&author_conf=100&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str>
}

rcode:`<int>`	Return code. `0` → Subtitles successfully locked/unlocked.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Subtitles successfully locked.

 {
  "rcode" : 0 ,
  "rcode_description" : "Subtitles successfully locked."
 }

/speech/edit_history

Returns a list of all edit sessions carried out over an specific Media ID. Session edits can be applied to a subtitles file calling the /get interface and passing to it the proper Session ID to the session_id parameter.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 23. [/speech/edit_history] Get list of edit sessions of a Media ID.

http://my-tlp-server.com/api/speech/edit_history?id=MEDIA-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
    "rcode_description": <str>,
    "rcode": <int>,
    "edit_history": [
       {
           "session_id": <int>,
           "author_id": <str>,
           "author_conf": <int>,
           "author_name": <str>,
           "timestamp": <str>,
           "requires_revision": <bool>,
           "revised": <bool>,
           "revised_at": <str>,
           "revised_by_id": <int>,
           "revised_by_name": <str>,
           "revised_via": <str>,
           "revised_in_session_id": <int>,
           "edit_stats": <dict>
       },
       ...
    ]
 }

rcode:<int>

Return code.

0 → Edit history available. Returns the history in edit_history.
1 → Media ID does not exist.

edit_history:<array:dict>

List of dictionaries containing information about each edit session. <dict> keys:

session_id:<str>: Session ID.
author_id:<str>: Author ID owner of the session.
author_conf:<int>: Confidence level of the author, from 0 to 100.
author_name:<str>: Author name. Can be null.
timestamp:<str>: Session end timestamp.
requires_revision:<bool>: Indicates if session edits require revision to be approved.
revised:<bool>: Declares whether the session has been revised or not (see /mark_revised). Will be null if revised = False.
revised_by_id:<str>: Author ID that revised this session edits. Will be null if revised = False.
revised_by_name:<str>: Name of the author that revised this session edits. Will be null if revised = False.
revised_via:<str>: Specifies from which Web Service interface the revision was set as revised. Could be mark_revised, accept or reject. Will be null if revised = False.
revised_in_session_id:<int>: Session ID under the author ID revised this session edits. Can be null.
edit_stats:<dict>: Dictionary containing statistics of the session’s edits. Each dictionary key is a language code (ISO-639-1) whose value is another dictionary that contains edit statistics of the corresponding subtitles language. These dictionaries feature the following keys:
- del_segs:<int>: Number of segments deleted.
- edit_segs:<int>: Number of segments edited.
- rev_time:<int>: Time spent by the user while editing.
- rev_rtf:<int>: RTF spent by the user while editing.
- edit_time:<float>: Total amount of media time in seconds that has been covered in the edit.
- edit_time_percent:<float>: Percentage of media time that has been covered in the edit with respect to the media length.

Output Examples

Edit history for a particular Media ID is returned.

{
    "rcode_description": "Edit history available.",
    "rcode": 0,
    "edit_history": [
        {
            "session_id": 9,
            "author_id": "lord_stark1",
            "author_conf": 100,
            "author_name": "Eddard Stark",
            "timestamp": "2015-07-05 12:14:15.824233",
            "requires_revision": false,
            "revised": null,
            "revised_at": null,
            "revised_by_id": null,
            "revised_by_name": null,
            "revised_via": null,
            "revised_in_session_id": null,
            "edit_stats": {
                "en": {
                    "del_segs": 2,
                    "edit_segs": 67,
                    "rev_time": 134,
                    "rev_rtf": 2.0,
                    "edit_time": 631.97,
                    "edit_time_percent": 100.00
                },
                "es": {
                    "del_segs": 3,
                    "edit_segs": 5,
                    "rev_time": 65,
                    "rev_rtf": 2.0,
                    "edit_time": 32.52,
                    "edit_time_percent": 5.12
                }
            }
        },
        {
            "session_id": 8,
            "author_id": "jsnow21",
            "author_conf": 90,
            "author_name": "John Snow",
            "timestamp": "2015-07-04 21:23:07.000031",
            "requires_revision": true,
            "revised": true,
            "revised_at": "2015-07-05 12:14:15.824233",
            "revised_by_id": "lord_stark1",
            "revised_by_name": "Eddard Stark",
            "revised_via": "mark_revised",
            "revised_in_session_id": 9,
            "edit_stats": {
                "en": {
                    "del_segs": 0,
                    "edit_segs": 21,
                    "edit_time": 197.23,
                    "edit_time_percent": 31.74
                }
            }
        },
        {
            "session_id": 7,
            "author_id": "olly666",
            "author_conf": 20,
            "author_name": "Olly",
            "timestamp": "2015-07-02 18:53:59.565720",
            "requires_revision": true,
            "revised": false,
            "revised_at": null,
            "revised_by_id": null,
            "revised_by_name": null,
            "revised_via": null,
            "revised_in_session_id": null,
            "edit_stats": {
                "en": {
                    "del_segs": 4,
                    "edit_segs": 3,
                    "edit_time": 13.22,
                    "edit_time_percent": 2.58
                }
            }
        }
    ]
}

/speech/revisions

Returns a list of all edit sessions for all API user’s media files that are pending to be revised.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 24. [/speech/revisions] Get list of all pending revisions.

http://my-tlp-server.com/api/speech/revisions?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 [
    {
        "media_id": <str>,
        "session_id": <int>,
        "author_id": <str>,
        "author_conf": <int>,
        "author_name": <str>,
        "timestamp": <str>,
        "edit_stats": <dict>
    },
    ...
 ]

media_id:`<str>`	Media ID.
session_id:`<str>`	Session ID.
author_id:`<str>`	Author ID owner of the session.
author_conf:`<int>`	Confidence level of the author, from 0 to 100.
author_name:`<str>`	Author name. Can be `null`.
timestamp:`<str>`	Session end timestamp.
edit_stats:`<dict>`	Dictionary containing statistics of the session’s edits. Each dictionary key is a language code (ISO-639-1) whose value is another dictionary that contains edit statistics of the corresponding subtitles language. These dictionaries feature the following keys: del_segs:`<int>`: Number of segments deleted. edit_segs:`<int>`: Number of segments edited. edit_time:`<float>`: Total amount of media time in seconds that has been covered in the edit. edit_time_percent:`<float>`: Percentage of media time that has been covered in the edit with respect to the media length.

Output Examples

Pending revisions list returned.

 [
    {
        "media_id": "MEDIA-ID-1234",
        "session_id": 8,
        "author_id": "jsnow21",
        "author_conf": 90,
        "author_name": "John Snow",
        "timestamp": "2015-07-04 21:23:07.000031",
        "edit_stats": {
            "en": {
                "del_segs": 2,
                "edit_segs": 1,
                "edit_time": 3.41,
                "edit_time_percent": 1.97
            }
        }
    }
 ]

/speech/mark_revised

Mark/unmark an edition session as revised.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Media ID.	`MEDIA-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, who revised the changes made in the Session ID session_id.	`lord_stark1`
author_name	str	No	Author Name	`Eddard Stark`
author_conf	int	Yes	Confidence level of the author. Must be `100`.	`100`
revision_session_id	int	No	Session ID under which the given Author ID revised the changes made in the Session ID session_id.	12
unmark	int	No	Do the inverse process: delete revised mark. `0` → Mark session as revised (default). `1` → Remove revised mark from the session.	1
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 25. [/speech/mark_revised] End the edition session ID 8 for media ID "MEDIA-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/speech/mark_revised?id=MEDIA-ID-1234&session_id=8&author_id=lord_stark1&author_conf=100&revision_session_id=12&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str>
}

rcode:<int>

Return code.

0 → Session ID successfully marked/unmarked as revised.
1 → Media ID does not exit.
2 → Session ID does not exist.
3 → Session ID has not been closed yet.
4 → The provided Session ID does not involve the given Media ID.
5 → Session ID did not require revison.
6 → Session ID was already revised before.
7 → Revision Session ID does not exist.
8 → Revision Session ID does not involve the given Media ID.
9 → Revision Session ID is not owned by the author ID that is requesting this session ID to be marked as revised.

rcode_description:<str>

Description of the return code (rcode).

Output Examples

Edition session marked as revised.

 {
    "rcode": 0,
    "rcode_description": "Session ID marked as revised."
 }

/speech/accept

Accept modifications of one or more pending edit sessions without having to revise them. Modifications are commited into the corresponding subtitles files.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Comma-separated list of Session IDs whose edits are meant to be accepted by the given Author ID.	`27,43,82,121`
author_id	str	Yes	Author ID, authorised by the API client, who revised the changes made in the Session ID session_id.	`lord_stark1`
author_name	str	No	Author Name	`Eddard Stark`
author_conf	int	Yes	Confidence level of the author. Must be `100`.	`100`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 26. [/speech/accept] Accept edits of session IDs 27, 43, 82, 121.

http://my-tlp-server.com/api/speech/accept?id=27,43,82,121&&author_id=lord_stark1&author_conf=100&revision_session_id=12&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

Returns a list of dictionaries, one for each Session ID.

[
  {
    "session_id": <int>,
    "rcode" : <int> ,
    "rcode_description" : <str>
  },
  ...
]

session_id:`<int>`	Session ID.
rcode:`<int>`	Return code. `0` → Changes from Session ID successfully accepted. `1` → Session ID does not exist. `2` → Unallowed access to the provided Session ID. `3` → Session ID has not been closed yet. `4` → Session ID did not require revison. `5` → Session ID was already revised before. `6` → Could not save Session ID changes to subtitles file.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Trying to accept edits of session IDs 27, 43, 82, 121.

[
    {
        "rcode_description": "Session ID does not exist.",
        "rcode": 1,
        "session_id": 27
    },
    {
        "rcode_description": "Session ID did not require revision.",
        "rcode": 4,
        "session_id": 43
    },
    {
        "rcode_description": "Session ID already revised.",
        "rcode": 5,
        "session_id": 82
    },
    {
        "rcode_description": "Changes from Session ID successfully accepted.",
        "rcode": 0,
        "session_id": 121
    }
]

/speech/reject

Reject modifications of one or more pending edit sessions without having to revise them.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Comma-separated list of Session IDs whose edits are meant to be accepted by the given Author ID.	`27,43,82,121`
author_id	str	Yes	Author ID, authorised by the API client, who revised the changes made in the Session ID session_id.	`lord_stark1`
author_name	str	No	Author Name	`Eddard Stark`
author_conf	int	Yes	Confidence level of the author. Must be `100`.	`100`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 27. [/speech/reject] Accept edits of session IDs 27, 43, 82, 121.

http://my-tlp-server.com/api/speech/reject?id=27,43,82,121&&author_id=lord_stark1&author_conf=100&revision_session_id=12&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

Returns a list of dictionaries, one for each Session ID.

[
  {
    "session_id": <int>,
    "rcode" : <int> ,
    "rcode_description" : <str>
  },
  ...
]

session_id:`<int>`	Session ID.
rcode:`<int>`	Return code. `0` → Changes from Session ID successfully rejected. `1` → Session ID does not exist. `2` → Unallowed access to the provided Session ID. `3` → Session ID has not been closed yet. `4` → Session ID did not require revison. `5` → Session ID was already revised before.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Trying to reject edits of session IDs 27, 43, 82, 121.

[
    {
        "rcode_description": "Session ID does not exist.",
        "rcode": 1,
        "session_id": 27
    },
    {
        "rcode_description": "Session ID did not require revision.",
        "rcode": 4,
        "session_id": 43
    },
    {
        "rcode_description": "Session ID already revised.",
        "rcode": 5,
        "session_id": 82
    },
    {
        "rcode_description": "Changes from Session ID successfully rejected.",
        "rcode": 0,
        "session_id": 121
    }
]

Appendix H: Web Service API: /text API specifications

Please note that all the following interfaces will share a common prefix: /text.

Interfaces for document upload and management:

/ingest	Upload document (text/document) files and metadata to the TLP Server to be automatically translated.
/uploadslist	Get a list of all the user’s uploads.
/status	Check the current status of a specific upload ID.
/systems	Get a list of all available Machine Translation systems that can be applied to translate a document file.
/list	Get a list of all existing documents in the Database.

Interfaces for downloading translated document files:

/metadata	Get metadata for a given document ID.
/langs	Get a list of all language translations available for a given document ID.
/get	Download the latest version of the translated document file for a given document ID and language.

Interfaces for editing translations:

/start_session	Starts an edition session to send and commit modifications of a translation file.
/session_status	Returns the current status of the given session ID.
/mod	Send and commit translation corrections under an edit session.
/end_session	Ends an open edition session, and depending on the confidence of the user, editions are directly stored in the corresponding translation files or left for revision.

Interfaces for managing translations' user revisions:

/lock	Allow/disallow regular users to send modifications for an specific document ID.
/edit_history	Returns a list of all edit sessions that involved an specific document ID.
/revisions	Returns a list of all edit sessions for all API user’s document files that are pending to be revised.
/mark_revised	Mark/unmark as revised an specific edit session ID, typically from another Session ID on the TLP Translation Editor.
/accept	Accept modifications of one or more pending edit sessions without having to revise them. Modifications are commited into the corresponding translation files.
/reject	Reject modifications of one or more pending edit sessions without having to revise them.

/text/ingest

This interface allows the client to upload document files to the platform so they can be automatically translated into several languages by the Ingest Service. The uploaded documents are bundled into a non-compressed ZIP file called Document Package (MPF). The Web Service stores the Document Packages in the server and returns an Upload ID, which can be used afterwards to check the upload progress via the /status interface.

The /ingest interface is split into four sub-interfaces, every one devoted to implement the four operation types supported by the Ingest Service:

/new → New document operation.
/update → Update document operation.
/delete → Delete document operation.
/cancel → Cancel upload operation.

Every sub-interface has different input specifications, please refer to each sub-section. Particularly, the /new and /update operations require as input a base64-encoded Manifest JSON string (please see Manifest JSON sub-section). In case any document files have to be uploaded to the TLP Server, those files have to be declared in the Manifest JSON string, and attached to the HTTP query as a Document Package file (please see Document Package sub-section). Finally, all sub-interfaces share the same output specifications, please see the Output sub-section.

Manifest JSON

The Manifest JSON is a JSON string that provides several metadata required by the /new and /update operations. For example, on new operations it must describe document attributes such as ID, title or language. Also, it is used to tell the Ingest Service that one or more document files have been uploaded, bundled into a Document Package file.

Manifest file JSON Specification

{
  "language" : <str> ,
  "documents" : [
                  {
                   "external_id" : <str> ,
                   "title" : <str> ,
                   "filename" : <str> ,
                   "fileformat" : <str> ,
                   "md5" : <str> ,
                   "url" : <str>
                  } ,
                  ...
                ],
  "requested_langs": <dict> ,
  "generate" : <bool> ,
  "re-generate": <bool> ,
  "force": <bool> ,
  "test_mode" : <bool>
}

language:<str> → Document(s)' language code in ISO 639-1 format (e.g. "en", "es").
documents:<list:dict> → List of documents to be translated. <dict> keys:
- external_id:<str> → Document ID (typically an internal ID in the client’s document repository database) used to identify the document in further queries to the Web Service.
- title:<str> → Title of the document.
- filename:<str> → File name of the document file.
- fileformat:<str> → Format of the document file (see Allowed document formats).
- md5:<str> → MD5 checksum of the document file.
- url:<str> → URL to the main document file. If a url field is given, then fields filename, fileformat and md5 are ignored.
requested_langs:<dict> → Request translation languages along with some advanced options. Please see Requesting translation languages (optional).
generate:<bool> → Enable/disable translation technologies (optional, default true).
re-generate:<bool> → On update operation: request regeneration of translations for the provided list of Document IDs. In this case, only the external_id key is required for each documents item (optional, default false).
force:<bool> → On update operation: Force regeneration of automatic subtitles even if there exist human-supervised subtitles (optional, default false).
test_mode:<bool> → Enable/Disable the Test Mode of the Ingest Service. When enabled, the uploaded document(s) will be available immediately with an empty translation file. This feature is very useful when executing integration tests with the /ingest interface (optional, default false).

Requesting translation languages

The requested_langs option, as stated before, is used to request translation languages, specifying advanced translation options. requested_langs is a JSON dictonary in which keys are ISO 639-1 language codes (e.g. "en", "es"), and values are dictionaries with advanced options.

 "requested_langs": {
    "es": {},
    "en": {}
 }

The example above means: "Generate Spanish and English translations using default options in all cases".

Advanced options:

sid:<int> → Specify which System will be applied to generate the translation file. If not specified, the default system is used.
tlpath:<list> → Explicitly declare a Translation Path. This is useful to generate translations to a language which is not featured directly from the spoken language, using intermediate translation languages. It is declared as an ordered list of dictionaries, where each dictionary specifies the target language code l of the step, and optionally the System ID sid to apply.

The example below shows how to request a Catalan (ca) translation from the document language (XX) using English (en) and Spanish (es) as intermediate languages, thus defining the following translation path: XX->En->Es->Ca. The intermediate En->Es translation is generated using System ID 3.
```
 "requested_langs": {
    "ca": {
             "tlpath": [
               { "l":"en" },
               { "l":"es", "sid":3},
               { "l":"ca"}
            ]
      }
 }
```
If this option is not specified, the Ingest Service will assume that a direct translation from the spoken language is requested.

Allowed document formats

Below follows a table in which are listed the document file formats that are typically allowed to be uploaded to a standard Ingest Service.

Table 2. Allowed document formats
File Format	Description
txt	Plain text
html	HTML file
doc	MS Word File
docx	MS Word XML File

Document Package

A Document Package File is an uncompressed ZIP file that contains several document files declared in the Manifest JSON data sent to the /ingest/new or /ingest/update interfaces. All files must be placed in the root on the ZIP package (not inside folders or sub-folders).

Output

All four sub-interfaces share a common output specification, described as follows:

JSON Object: Content-Type = application/json

 {
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "id" : <str> ,
  "hash" : <str>
 }

rcode:`<int>`	Return code. `0` → Upload completed.
rcode_description:`<str>`	Description of the return code (rcode).
id:`<str>`	Upload ID, which can be used afterwards to check the progress of the upload via the /status interface.
hash:`<str>`	Internal document hash ID.

Output Examples

Document Package File successfully uploaded.

 {
  "rcode" : 0,
  "rcode_description" : "Ingestion complete",
  "id" : "UPLOAD-ID-1234"
  "hash" : "433e11c295c51b94a074a"
 }

/text/ingest/new

New document operation.

Input

In this operation only the POST method is allowed (Content-Type: multipart/form-data). Input data is divided in two fields:

json: Query parameters encoded in a JSON string (Content-Type: application/json)

Parameter name	Type	Required	Description	Example
manifest	json	Yes	Manifest JSON. See particular specifications for this operation.	`[See this example]`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

pkg (optional): A Document Package zip file (Content-Type: application/zip).

Manifest JSON data

Below is described how to properly set up a Manifest JSON string for New document operations.

Required inputs:

At least one document file to be translated in the documents section.

Behavior changes with optional inputs

requested_langs option:
- Not provided: The document(s) won’t be translated.
- Provided: Actions that can be inferred from this option will be executed.

Call examples

Upload a new document using a public URL to a document file, requesting Catalan and Spanish translations.

JSON string:

 {
    "user": "tluser",
    "key": "edbab44c3a3f1ca8db4de8277a3b",
    "manifest": {
      "language": "en",
      "documents": [
         {
           "external_id": "9abc7230fe36a18b885c",
           "title": "To know something or not to know nothing: A brief essay about knowledge.",
           "url": "http://my-document-server.com/path/to/my/document.html"
         }
      ],
      "requested_langs": {
         "ca":{},
         "es":{}
      }
   }
 }

HTTP Call to /ingest/new (note that a Document Package file is not required in this case).

http://my-tlp-server.com/api/text/ingest/new

 + POST data (JSON string goes in the "json" field)

/text/ingest/update

Update document operation.

Input

In this operation only the POST method is allowed (Content-Type: multipart/form-data). Input data is divided in two fields:

json: Query parameters encoded in a JSON string (Content-Type: application/json)

Parameter name	Type	Required	Description	Example
manifest	json	Yes	Manifest JSON. See particular specifications for this operation.	`[See this example]`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

pkg (optional): A Media Package zip file (Content-Type: application/zip).

Manifest JSON data

Below is described how to properly set up a Manifest JSON string for Update document operations.

Required inputs:

One or more document file(s) in documents section when updating original document(s), plus the language attribute.

or
One or more document IDs in the re-generate option.

Please note that the documents and re-generate keys cannot be used together.

To generate new translation languages, use the re-generate option combined with a proper setting of the requested_langs option.

In all cases, supplied external_id values (Document IDs) must exist.

Behavior changes with optional inputs

requested_langs option:
- Not provided: The ingest service will attempt to re-generate all existing translations.
- Provided: Actions that can be inferred from this option will be executed. Actions that involve the re-generation of translations that have already been edited or supervised by users won’t be executed, unless the force option is set to true.

Call examples

Update document by requesting a re-generation of translations.

JSON string:

 {
    "user": "tluser",
    "key": "edbab44c3a3f1ca8db4de8277a3b",
    "manifest": {
      "documents": [ { "external_id": "DOCUMENT-ID-1234"},
                     { "external_id": "DOCUMENT-ID-5678"} ] ,
      "re-generate": true
    }
 }

HTTP Call to /ingest/update (note that no Document Package file is required).

http://my-tlp-server.com/api/text/ingest/update

 + POST data (JSON string goes in the "json" field)

Update an existing document and its translation(s) by uploading a new version (attaching a new document file).

JSON string:

 {
    "user": "tluser",
    "key": "edbab44c3a3f1ca8db4de8277a3b",
    "manifest": {
      "documents": [
         {
           "external_id": "9abc7230fe36a18b885c",
           "title": "To know something or not to know nothing: A brief essay about knowledge (NEW VERSION).",
           "url": "http://my-document-server.com/path/to/my/document.new.html"
         }
      ],
      "requested_langs": {
         "ca":{},
         "es":{}
      }
   }
 }

HTTP Call to /ingest/update, including a Document Package file via POST.

http://my-tlp-server.com/api/text/ingest/update

 + POST data (JSON string goes in the "json" field, Media Package file in the "pkg" field)

/text/ingest/delete

Delete document operation.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
mode	str	No	Delete mode. * `soft` → document will be marked as "deleted" in the database and its files will be backed up. * `hard` → all document and subtitle files stored will be deleted, as well as all related entries in the Database.	`hard`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 28. [/text/ingest/delete] delete document ID (GET with multiple parameters).

http://my-tlp-server.com/api/text/ingest/delete?id=DOCUMENT-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

/text/ingest/cancel

Cancel upload operation.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Upload ID, returned by the /ingest interface.	`UPLOAD-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 29. [/text/ingest/cancel] Cancel upload (GET with multiple parameters).

http://my-tlp-server.com/api/text/ingest/cancel?id=UPLOAD-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

/text/uploadslist

Returns a list of all user’s uploads.

Input

Parameter name	Type	Required	Description	Example
object_id	str	No	Get list of uploads involving the provided object ID (could be an Upload ID or a Document ID).	`DOCUMENT-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 30. [/text/uploadslist] Get list of uploads.

http://my-tlp-server.com/api/text/uploadslist?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
    {
        "id": <str>,
        "object_id": <str>,
        "status_code": <int>,
        "uploaded_on": <str>,
        "last_update": <str>
    },
    ...
]

id:`<str>`	Upload ID.
object_id:`<str>`	Object ID involved (could be an Upload ID or a Document ID).
status_code:`<int>`	Status code of the Upload. `0` → Ingested, not processed yet. `1` → Processing Document Package file. `2` → Translation(s) in progress. `3` → Document(s) successfully processed. `20` → Document(s) successfully updated. `30` → Document(s) successfully deleted. `40` → Cancel upload operation successfully processed. `90` → Upload cancelled. `100` → An unknown error occurred. `101` → An error occurred while processing the Document Package. `102` → An error occurred while translating the document. `103` → An error occurred while updating internal database.
uploaded_on:`<str>`	Upload timestamp.
last_update:`<str>`	Last upload check timestamp.

Output Examples

Uploads List returned.

[
    {
        "id": "up-ac83be70-a01c-4c18-8cc4-dc0b2676cbb0",
        "object_id": "DOCUMENT-ID-1234",
        "status_code": 2,
        "uploaded_on": "2015-06-10 17:19:37.239458",
        "last_update": "2015-06-10 17:20:02.557135"
    },
    {
        "id": "up-60a70bbd-e111-4d0c-b41f-6e235c434330",
        "object_id": "DOCUMENT-ID-1234",
        "status_code": 101,
        "uploaded_on": "2015-06-09 11:21:07.735656",
        "last_update": "2015-06-09 11:22:02.549826"
    },
    {
        "id": "up-776ae4ec-6904-4da1-afa8-1b68017a524a",
        "object_id": "DOCUMENT-ID-5678",
        "status_code": 3,
        "uploaded_on": "2015-06-10 17:25:04.673541",
        "last_update": "2015-06-10 17:26:02.542902"
    }
]

/text/status

Returns information about the progress of an upload ID. It enables the remote repository to keep track of the automatic uploads and to notice possible processing errors.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Upload ID, returned by the /ingest interface.	`UPLOAD-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 31. [/text/status] check status of an upload (GET with multiple parameters).

http://my-tlp-server.com/api/text/status?id=UPLOAD-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "status_code" : <int> ,
  "info" : <str> ,
  "error_code" : <int> ,
  "uploaded_on" : <str> ,
  "last_update" : <str>
}

rcode:`<int>`	Return code. `0` → Upload ID exists. `1` → Upload ID does not exist.
rcode_description:`<str>`	Description of the return code (rcode).
status_code:`<int>`	Status code of the Upload. `0` → Ingested, not processed yet. `1` → Processing Document Package file. `2` → Translation(s) in progress. `3` → Document(s) successfully processed. `20` → Document(s) successfully updated. `30` → Document(s) successfully deleted. `40` → Cancel upload operation successfully processed. `90` → Upload cancelled. `100` → An unknown error occurred. `101` → An error occurred while processing the Document Package. `102` → An error occurred while translating the document. `103` → An error occurred while updating internal database.
info:`<str>`	Detailed information about the status code.
error_code:`<int>`	Generic error code that identifies the operation that failed within the process, if any. Otherwise null.
uploaded_on:`<str>`	Upload timestamp.
last_update:`<str>`	Last status check timestamp.

Output Examples

The Upload ID provided does not exist.

 {
   "rcode": 1,
   "rcode_description" : "Upload ID [ up-1234 ] does not exist."
 }

The provided upload ID exists and it’s being processed.

 {
   "rcode": 0,
   "rcode_description" : "Upload ID exists.",
   "status_code": 2,
   "info": "Translation(s) in progress. It may take several hours for it to finish.",
   "uploaded_on": "2014-03-26 19:02:16.174944",
   "last_update": "2014-03-26 19:03:05.298861"
 }

/text/systems

Get a list of all available MT Systems that can be applied to translate the uploaded document file(s) using the /ingest interface.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 32. [/text/systems] Get list of available systems.

http://my-tlp-server.com/api/text/systems?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
    "mt": [
        {
            "source_lang": <str>,
            "target_lang": <str>,
            "id": <int>,
            "name": <str>,
            "description": <str>
        },
        ...
    ]
]

mt:<list:dict>

List of all available Machine Translation Systems. <dict> keys:

source_lang:<str> → MT System source language code (ISO-639-1).
target_lang:<str> → MT System target language code (ISO-639-1).
id:<int> → System ID.
name:<str> → Name of the system.
description:<str> → Description of the system.

The system ID might be used to explicitly request to the Ingest Service the application of a particular MT system. For further information please see Requesting Subtitle Languages).

Output Examples

Systems List returned.

{
    "mt": [
        {
            "source_lang": "ca",
            "target_lang": "es",
            "id": 14,
            "name": "Catalan-Spanish MT System",
            "description": ""
        },
        {
            "source_lang": "es"
            "target_lang": "ca",
            "id": 11,
            "name": "Spanish-Catalan MT System",
            "description": ""
        },
        {
            "source_lang": "en",
            "target_lang": "es",
            "id": 73,
            "name": "English-Spanish MT System",
            "description": ""
        },
        {
            "source_lang": "es"
            "target_lang": "en",
            "id": 24,
            "name": "Spanish-English MT System",
            "description": ""
        }
    ]
}

/text/list

Get a list of all existing documents in the TLP Database.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 33. [/text/list] Get list of all existing audios and documents.

http://my-tlp-server.com/api/text/list?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
    {
        "id": <str>,
        "title": <str>,
        "language": <str>,
        "format": <str>,
        "sentences": <int>,
        "words": <int>,
        "characters": <int>
    },
    ...
]

id:<int> → Document ID.
title:<str> → Document’s title.
language:<str> → Document language code (ISO-639-1).
format:` → Document file format.
sentences:<int> → Document length in sentences.
words:<int> → Document length in words.
characters:<int> → Document length in characters.

Output Examples

Document list returned.

{
    {
        "id": "DOCUMENT-ID-1234",
        "title": "I do know nothing",
        "language": "en",
        "format": "html",
        "sentences": 86,
        "words": 916,
        "characters": 5701
    },
    {
        "id": "DOCUMENT-ID-6789",
        "title": "Tyrion Lannister and Daenerys Targaryen: the Royal Wedding.",
        "language": "en",
        "format": "docx",
        "sentences": 271,
        "words": 2953,
        "characters": 14031
    }
}

/text/metadata

Returns metadata and document file locations of a given document ID. For example, this operation is called by the TLP Translation Editor to get the main document file location.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 34. [/text/metadata] Get document metadata (GET with multiple parameters).

http://my-tlp-server.com/api/text/metadata?id=DOCUMENT-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
  "rcode": <int> ,
  "rcode_description": <str> ,
  "info": {
             "title": <str> ,
             "language": <str > ,
             "format": <str> ,
             "sentences": <int> ,
             "words": <int> ,
             "characters": <int>
          }
 }

rcode:`<int>`	Return code of the WS call. `0` → Document ID exists. `1` → Document ID does not exist.
rcode_description:`<str>`	Description of the return code (rcode).
info:`<dict>`	Document metadata. `<dict>` keys: title:`<str>` → Document’s title. language:`<str>` → Document language code (ISO-639-1). format:` → Document file format. sentences:`<int>` → Document length in sentences. words:`<int>` → Document length in words. characters:`<int>` → Document length in characters.

Output Examples

Document ID does not exist.

 {
  "rcode" : 1,
  "rcode_description" : "Document ID [ 1234-abcd ] does not exist or has no document"
 }

Document ID exists.

  {
    "rcode": 0,
    "rcode_description": "Document info available.",
    "info": {
              "title": "DOCUMENT-ID-1234",
              "language": "en",
              "format": "html",
              "sentences": 86,
              "words": 916,
              "characters": 5701
           }
  }

/text/langs

Returns list of translations (languages) available for a specific document ID.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 35. [/text/langs] Get list of subtitle languages available (GET with multiple parameters).

http://my-tlp-server.com/api/text/langs?id=DOCUMENT-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "document_lang" : <str> ,
  "locked" : <bool> ,
  "langs" : [
              {
                "lang_code" : <str> ,
                "lang_name" : <str> ,
                "sup_status" : <int>
                "sup_ratio" : <float>
              },
              ...
            ]
 }

rcode:`<int>`	Return code of the WS call. `0` → Document ID exists. `1` → Document ID does not exist.
rcode_description:`<str>`	Description of the return code (rcode).
document_lang:`<str>`	Language code (ISO-639-1) of the document language.
locked:`<bool>`	Lock status of translations. If `true`, only authors are allowed to send translation modifications.
langs:`<list:dict>`	List of languages available. `<dict>` keys: lang_code:`<str>` → Subtitle language code (ISO-639-1). lang_name:`<str>` → Local language name. sup_status:`<int>` → Translation supervision status. Defines the level of human supervision of the translation. `0` → Not Translated: Empty translation. `1` → Fully Automatic: All sentences are automatic translations. `2` → Partially Human: Some translated sentences have been supervised, but not for the whole document. `3` → Fully Human: All translated sentences have been fully supervised by humans. sup_ratio:`<float>` → Percentage of words that have been supervised.

Output Examples

Document ID does not exist or has no translations.

 {
  "rcode" : 1 ,
  "rcode_description" : "ID 1234-abcd does not exist or has no translations" ,
 }

Document ID exists and has translations available.

{
    "rcode": 0,
    "rcode_description": "Language list available.",
    "document_lang": "es",
    "locked": false,
    "langs": [
        {
            "lang_code": "es",
            "sup_status": 2,
            "sup_ratio": 61.1650485436893,
            "lang_name": "Español"
        },
        {
            "lang_code": "ca",
            "sup_status": 1,
            "sup_ratio": 0,
            "lang_name": "Català"
        },
        {
            "lang_code": "en",
            "sup_status": 3,
            "sup_ratio": 100,
            "lang_name": "English"
        }
    ]
}

/text/get

Returns translations for a specific document ID and language.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
lang	str	Yes	Language code (ISO 639-1).	`en`
format	int	No	Format. `0` → Original format (default). `1` → Intermediate HTML format. `2` → Internal DTLX format. `3` → Plain text format.	`0`
session_id	int	No	Load translation modifications from the given session ID (if any). If format=2, modified sentences will include the highlight attribute (`h="1"`).	`327`
sel_data_policy	int	No	Subtitle contents to be returned. `0` → Return both former and current contents (only for DTLX format). `1` → Return former contents (automatic translation). `2` → Return current contents (current status of supervision of the translation) (default).	`2`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 36. [/text/get] Get Catalan translation for document ID "DOCUMENT-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/text/get?id=DOCUMENT-ID-1234&lang=ca&format=2&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

Document file:

On its original format
TXT file: Content-Type = text/plain
DTLX file: Content-Type = application/xml

JSON Object (if document ID or translation file does not exist):

Content-Type = application/json

 {
  "rcode" : <int> ,
  "rcode_description" : <int>
 }

rcode:`<int>`	Return code. `1` → Document ID does not exist or translation in the specified language do not exist. `2` → The provided Session ID does not exist. `3` → The provided Session ID does not involve the given Document ID. `4` → Document is not available to be downloaded in the requested format.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Document ID does not exist.

 {
  "rcode" : 1 ,
  "rcode_description" : "Document ID [DOCUMENT-ID-1234] does not exist"
 }

/text/start_session

Starts an edition session to send and commit modifications of a translated document file. Edition sessions are a mechanism devoted to avoid race conditions between different users when editing a translated file.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
author_id	str	Yes	Author ID, authorised by the API client, that starts the edition session.	`jsnow21`
author_name	str	No	Author Name	`John Snow`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 37. [/text/start_session] Start an edition session for document ID "DOCUMENT-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/text/start_session?id=DOCUMENT-ID-1234&author_id=jsnow21&author_conf=90&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
    "rcode": <int>,
    "rcode_description": <str>,
    "session_id": <int>,
    "author_id": <str>,
    "author_name": <str>,
    "author_conf": <int>,
    "author_type": <str>,
    "started_at": <str>,
    "last_update": <str>
 }

rcode:`<int>`	Return code. `0` → Session successfully started, restarted, or continued. Returns information about the session. `1` → Document ID does not exist. `2` → Document ID has no translations. `3` → Translations for the Document ID have been locked. Only authors are allowed to modify them. `4` → Could not start session due to the existence of an started session by another author ID for the same Document ID. Returns information about the existing session.
rcode_description:`<str>`	Description of the return code (rcode).
session_id:`<int>`	Session ID.
author_id:`<str>`	Author ID that started the session and currently editing the Document ID.
author_conf:`<int>`	Confidence level of the author ID, from 0 to 100.
author_name:`<int>`	Author Name.
author_type:`<str>`	Author Type. `human` → Session ID is being carried by a human. `computer` → Session ID has been opened by the Ingest Service, in order to automatically regenerate translations.
started_at:`<str>`	Session start timestamp.
last_update:`<str>`	Timestamp of the last update (/mod call) made by the user on the session.

Output Examples

Session started.

 {
    "rcode": 0,
    "rcode_description": "Session started.",
    "session_id": 8,
    "author_id": "jsnow21",
    "author_name": "John Snow",
    "author_conf": 90,
    "author_type": "human",
    "started_at": "2015-07-04 20:44:55.042786",
    "last_update": "2015-07-04 20:44:55.042786",
 }

Another user is editing the same document ID.

 {
    "rcode": 4,
    "rcode_description": "Cannot start mod session: there exist an open mod session for the given document ID.",
    "session_id": 7,
    "author_id": "olly666",
    "author_name": "Olly",
    "author_conf": 100,
    "author_type": "human",
    "started_at": "2015-07-04 12:23:15.040186",
    "last_update": "2015-07-04 20:32:11.892843"
 }

/text/session_status

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, owner of the session_id.	`jsnow21`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
alive	int	No	Alive message type. `0` → Alive and user update (default). `1` → Only alive.	1
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 38. [/text/session_status] Check status of session ID 8 (GET with multiple parameters).

http://my-tlp-server.com/api/text/session_status?id=DOCUMENT-ID-1234&session_id=8&author_id=jsnow21&author_conf=90&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
    "rcode": <int>,
    "rcode_description": <str>,
    "started_at": <str>,
    "last_update": <str>,
    "last_alive": <str>,
    "ended_at": <str>,
    "ended_by_id": <str>,
    "ended_by_name": <str>,
    "ended_by_type": <str>
}

rcode:`<int>`	Return code. `0` → Session alive. `1` → Session ID does not exist. `2` → The provided Session ID does not involve the given Document ID. `3` → Not allowed to check the status of the given session ID. `4` → Session ID is not open (another user forced its closure, or was automatically closed after several hours of inactivity).
rcode_description:`<str>`	Description of the return code (rcode).
started_at:`<str>`	Session start timestamp.
last_update:`<str>`	Session last update timestamp.
last_alive:`<str>`	Session last alive timestamp.
ended_at:`<str>`	Session end timestamp. Will be `null` unless rcode`=4`.
ended_by_id:`<str>`	Author ID of the user that ended this session. Will be `null` unless rcode`=4`.
ended_by_name:`<str>`	Author name of the user that ended this session. Will be `null` unless rcode`=4`.
ended_by_type:`<str>`	Author type of the user that ended this session. Will be `null` unless rcode`=4`. `human` → Session ID is was closed by a human. `computer` → Session ID was automatically closed due to a large period of inactivity.

Output Examples

Session alive.

{
    "rcode": 0,
    "rcode_description": "Session alive.",
    "started_at": "2015-07-04 20:44:55.042786",
    "last_update": "2015-07-04 21:03:23.017192",
    "last_alive": "2015-07-04 21:03:51.932841",
    "ended_at": null,
    "ended_by_id": null,
    "ended_by_name": null
    "ended_by_type": null
}

Session closed.

{
    "rcode": 4,
    "rcode_description": "Session ID 8 is closed.",
    "started_at": "2015-07-04 20:44:55.042786",
    "last_update": "2015-07-04 20:44:55.042786",
    "last_alive": "2015-07-04 20:45:03.016531",
    "ended_at": "2015-07-04 22:28:51.075672",
    "ended_by_id": "lord_stark1",
    "ended_by_name": "Eddard Stark",
    "ended_by_type": "human"
}

/text/mod

Send and commit modifications of a translation file made by a user under a session ID returned by /start_session interface.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, owner of the session_id.	`jsnow21`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
mods	json	Yes	JSON Dictionary containing as many key-values as subtitle languages has been modified, being keys a ISO 639-1 language code of the subtitle languages edited, and values a dictionary containing the following keys: txt:`<list:dict>` → List of sentence IDs that have been modified. `<dict>` keys: sI:`<int>` → Sentence ID modified. t<`<str>` → Text of the specified sentence ID. Note: When using the GET method with multiple GET parameters, this JSON object must be encoded in Base64.	`{ "ca": {"txt":[ {"sI":16, "t":"She told me: You know nothing, John Snow"} ]} }`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 39. [/text/mod] Send modifications of the English and Spanish subtitles (see mods parameters example above) of document ID "DOCUMENT-ID-1234" under session ID 8 (GET with multiple parameters).

http://my-tlp-server.com/api/text/mod?id=DOCUMENT-ID-1234&session_id=8&language=en&author_id=jsnow21&author_conf=90&mod=eyAiY2EiOiB7InR4dCI6WyB7InNJIjoxNiwgInQiOiJTaGUgdG9sZCBtZTogWW91IGtub3cgbm90aGluZywgSm9obiBTbm93In0gXX0gfQ==

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str> ,
  "details": [
        {
            "language": <str>,
            "rcode_description": <str>,
            "rcode": <int>
        },
        ...
  ]
}

rcode:`<int>`	Return code. `0` → Changes succesfully stored in the session. `1` → Session ID does not exist. `2` → The provided Session ID does not involve the given Document ID. `3` → Not allowed to send modifications under the given session ID. `4` → mods parameter is an empty dictionary. `5` → Format errors where found in all translations modifications (see details key). `6` → Session ID is not open (another user forced its closure, or was automatically closed after several hours of inactivity). In addition, format errors where found in some translation modifications (see details key). Changes without errors are not commited to the edition session, but they are backuped. `7` → Session ID is not open (another user forced its closure, or was automatically closed after several hours of inactivity). Changes are not commited to the edition session, but they are backuped. `8` → Some translation modifications were successfully commited, but some other failed (see details key).
rcode_description:`<str>`	Description of the return code (rcode).
details:`<array:dict>`	List of dicts for every subtitle language modifications if format errors were found (rcode`={5,6,8}`).

Output Examples

Changes successfully saved.

 {
  "rcode" : 0 ,
  "rcode_description" : "Changes successfully saved."
 }

Session ID is not open.

 {
  "rcode" : 4 ,
  "rcode_description" : "Session ID [ 8 ] is closed, changes have been backuped. Please contact system administrator."
 }

Not all modifications were saved.

 {
    "rcode_description": "Modifications for some translations were successfully saved, but other failed. Please see 'details' key.",
    "rcode": 8,
    "details": [
        {
            "rcode_description": "The given document ID does not have 'en' translations.",
            "rcode": 10,
            "language": "en"
        },
        {
            "rcode_description": "Changes successfully saved.",
            "rcode": 0,
            "language": "es"
        }
    ]
 }

/text/end_session

Ends an open edition session. Depending on the confidence of the user, editions are directly stored into the corresponding DTLX files or are left for revision.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, that closes the session.	`jsnow21`
author_name	str	No	Author Name	`John Snow`
author_conf	int	Yes	Confidence level of the author, from 0 to 100.	`90`
force	int	No	Force end session when user and/or author_id are not the owners. `0` → Do not force end session (default). `1` → Force end session.	1
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 40. [/text/end_session] End the edition session ID 8 for document ID "DOCUMENT-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/text/end_session?id=DOCUMENT-ID-1234&session_id=8&author_id=jsnow21&author_conf=90&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str>
}

rcode:<int>

Return code.

0 → Session successfully closed or already closed before.
1 → Session ID does not exist.
2 → The provided Session ID does not involve the given Document ID.
3 → Session ID is owned by the Ingest Service, and it cannot be ended manually.
4 → Unallowed end session request (different user, author_id and/or author_conf parameters; force parameter, if provided, was even insufficent).

rcode_description:<str>

Description of the return code (rcode).

Output Examples

Session closed.

 {
    "rcode": 0,
    "rcode_description": "Session succesfully closed."
 }

/text/lock

Allow/disallow regular users to send translation modifications for an specific Document ID.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
lock	int	Yes	Lock action: `1` → Lock. `0` → Unlock.	`1`
author_id	str	Yes	Author ID, authorised by the API client, owner of the session_id.	`jsnow21`
author_conf	int	Yes	Confidence level of the author, from 0 to 100. Must be 100.	`100`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 41. [/text/lock] Disallow modifications for translations of document ID "DOCUMENT-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/text/lock?id=DOCUMENT-ID-1234&lock=1&author_id=lord_stark1&author_conf=100&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str>
}

rcode:`<int>`	Return code. `0` → Subtitles successfully locked/unlocked.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Subtitles successfully locked.

 {
  "rcode" : 0 ,
  "rcode_description" : "Subtitles successfully locked."
 }

/text/edit_history

Returns a list of all edit sessions carried out over an specific Document ID. Session edits can be applied to a translation file calling the /get interface and passing to it the proper Session ID to the session_id parameter.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 42. [/text/edit_history] Get list of edit sessions of a Document ID.

http://my-tlp-server.com/api/text/edit_history?id=DOCUMENT-ID-1234&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 {
    "rcode_description": <str>,
    "rcode": <int>,
    "edit_history": [
       {
           "session_id": <int>,
           "author_id": <str>,
           "author_conf": <int>,
           "author_name": <str>,
           "timestamp": <str>,
           "requires_revision": <bool>,
           "revised": <bool>,
           "revised_at": <str>,
           "revised_by_id": <int>,
           "revised_by_name": <str>,
           "revised_via": <str>,
           "revised_in_session_id": <int>,
           "edit_stats": <dict>
       },
       ...
    ]
 }

rcode:<int>

Return code.

0 → Edit history available. Returns the history in edit_history.
1 → Document ID does not exist.

edit_history:<array:dict>

List of dictionaries containing information about each edit session. <dict> keys:

session_id:<str>: Session ID.
author_id:<str>: Author ID owner of the session.
author_conf:<int>: Confidence level of the author, from 0 to 100.
author_name:<str>: Author name. Can be null.
timestamp:<str>: Session end timestamp.
requires_revision:<bool>: Indicates if session edits require revision to be approved.
revised:<bool>: Declares whether the session has been revised or not (see /mark_revised). Will be null if revised = False.
revised_by_id:<str>: Author ID that revised this session edits. Will be null if revised = False.
revised_by_name:<str>: Name of the author that revised this session edits. Will be null if revised = False.
revised_via:<str>: Specifies from which Web Service interface the revision was set as revised. Could be mark_revised, accept or reject. Will be null if revised = False.
revised_in_session_id:<int>: Session ID under the author ID revised this session edits. Can be null.
edit_stats:<dict>: Dictionary containing statistics of the session’s edits. Each dictionary key is a language code (ISO-639-1) whose value is another dictionary that contains edit statistics of the corresponding subtitles language. These dictionaries feature the following keys:
- edit_sents:<int>: Number of sentences edited.
- edit_sents_percent:<float>: Percentage of document sentences that have been covered in the edit.

Output Examples

Edit history for a particular Document ID is returned.

{
    "rcode_description": "Edit history available.",
    "rcode": 0,
    "edit_history": [
        {
            "session_id": 9,
            "author_id": "lord_stark1",
            "author_conf": 100,
            "author_name": "Eddard Stark",
            "timestamp": "2015-07-05 12:14:15.824233",
            "requires_revision": false,
            "revised": null,
            "revised_at": null,
            "revised_by_id": null,
            "revised_by_name": null,
            "revised_via": null,
            "revised_in_session_id": null,
            "edit_stats": {
                "en": {
                    "edit_sents": 67,
                    "edit_sents_percent": 100.00
                },
                "es": {
                    "edit_sents": 5,
                    "edit_sents_percent": 5.12
                }
            }
        },
        {
            "session_id": 8,
            "author_id": "jsnow21",
            "author_conf": 90,
            "author_name": "John Snow",
            "timestamp": "2015-07-04 21:23:07.000031",
            "requires_revision": true,
            "revised": true,
            "revised_at": "2015-07-05 12:14:15.824233",
            "revised_by_id": "lord_stark1",
            "revised_by_name": "Eddard Stark",
            "revised_via": "mark_revised",
            "revised_in_session_id": 9,
            "edit_stats": {
                "en": {
                    "edit_sents": 21,
                    "edit_sents_percent": 31.74
                }
            }
        },
        {
            "session_id": 7,
            "author_id": "olly666",
            "author_conf": 20,
            "author_name": "Olly",
            "timestamp": "2015-07-02 18:53:59.565720",
            "requires_revision": true,
            "revised": false,
            "revised_at": null,
            "revised_by_id": null,
            "revised_by_name": null,
            "revised_via": null,
            "revised_in_session_id": null,
            "edit_stats": {
                "en": {
                    "edit_sents": 3,
                    "edit_sents_percent": 2.58
                }
            }
        }
    ]
}

/text/revisions

Returns a list of all edit sessions for all API user’s document files that are pending to be revised.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 43. [/text/revisions] Get list of all pending revisions.

http://my-tlp-server.com/api/text/revisions?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

 [
    {
        "document_id": <str>,
        "session_id": <int>,
        "author_id": <str>,
        "author_conf": <int>,
        "author_name": <str>,
        "timestamp": <str>,
        "edit_stats": <dict>
    },
    ...
 ]

document_id:`<str>`	Document ID.
session_id:`<str>`	Session ID.
author_id:`<str>`	Author ID owner of the session.
author_conf:`<int>`	Confidence level of the author, from 0 to 100.
author_name:`<str>`	Author name. Can be `null`.
timestamp:`<str>`	Session end timestamp.
edit_stats:`<dict>`	Dictionary containing statistics of the session’s edits. Each dictionary key is a language code (ISO-639-1) whose value is another dictionary that contains edit statistics of the corresponding translation. These dictionaries feature the following keys: edit_sents:`<int>`: Number of segments edited. edit_sents_percent:`<float>`: Percentage of document sentences that have been covered in the edit.

Output Examples

Pending revisions list returned.

 [
    {
        "document_id": "DOCUMENT-ID-1234",
        "session_id": 8,
        "author_id": "jsnow21",
        "author_conf": 90,
        "author_name": "John Snow",
        "timestamp": "2015-07-04 21:23:07.000031",
        "edit_stats": {
            "en": {
                "edit_sents": 1,
                "edit_sents_percent": 1.97
            }
        }
    }
 ]

/text/mark_revised

Mark/unmark an edition session as revised.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Document ID.	`DOCUMENT-ID-1234`
session_id	int	Yes	Session ID	8
author_id	str	Yes	Author ID, authorised by the API client, who revised the changes made in the Session ID session_id.	`lord_stark1`
author_name	str	No	Author Name	`Eddard Stark`
author_conf	int	Yes	Confidence level of the author. Must be `100`.	`100`
revision_session_id	int	No	Session ID under which the given Author ID revised the changes made in the Session ID session_id.	12
unmark	int	No	Do the inverse process: delete revised mark. `0` → Mark session as revised (default). `1` → Remove revised mark from the session.	1
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 44. [/text/mark_revised] End the edition session ID 8 for document ID "DOCUMENT-ID-1234" (GET with multiple parameters).

http://my-tlp-server.com/api/text/mark_revised?id=DOCUMENT-ID-1234&session_id=8&author_id=lord_stark1&author_conf=100&revision_session_id=12&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode" : <int> ,
  "rcode_description" : <str>
}

rcode:<int>

Return code.

0 → Session ID successfully marked/unmarked as revised.
1 → Document ID does not exit.
2 → Session ID does not exist.
3 → Session ID has not been closed yet.
4 → The provided Session ID does not involve the given Document ID.
5 → Session ID did not require revison.
6 → Session ID was already revised before.
7 → Revision Session ID does not exist.
8 → Revision Session ID does not involve the given Document ID.
9 → Revision Session ID is not owned by the author ID that is requesting this session ID to be marked as revised.

rcode_description:<str>

Description of the return code (rcode).

Output Examples

Edition session marked as revised.

 {
    "rcode": 0,
    "rcode_description": "Session ID marked as revised."
 }

/text/accept

Accept modifications of one or more pending edit sessions without having to revise them. Modifications are commited into the corresponding translation files.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Comma-separated list of Session IDs whose edits are meant to be accepted by the given Author ID.	`27,43,82,121`
author_id	str	Yes	Author ID, authorised by the API client, who revised the changes made in the Session ID session_id.	`lord_stark1`
author_name	str	No	Author Name	`Eddard Stark`
author_conf	int	Yes	Confidence level of the author. Must be `100`.	`100`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 45. [/text/accept] Accept edits of session IDs 27, 43, 82, 121.

http://my-tlp-server.com/api/text/accept?id=27,43,82,121&&author_id=lord_stark1&author_conf=100&revision_session_id=12&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

Returns a list of dictionaries, one for each Session ID.

[
  {
    "session_id": <int>,
    "rcode" : <int> ,
    "rcode_description" : <str>
  },
  ...
]

session_id:`<int>`	Session ID.
rcode:`<int>`	Return code. `0` → Changes from Session ID successfully accepted. `1` → Session ID does not exist. `2` → Unallowed access to the provided Session ID. `3` → Session ID has not been closed yet. `4` → Session ID did not require revison. `5` → Session ID was already revised before. `6` → Could not save Session ID changes to translation file.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Trying to accept edits of session IDs 27, 43, 82, 121.

[
    {
        "rcode_description": "Session ID does not exist.",
        "rcode": 1,
        "session_id": 27
    },
    {
        "rcode_description": "Session ID did not require revision.",
        "rcode": 4,
        "session_id": 43
    },
    {
        "rcode_description": "Session ID already revised.",
        "rcode": 5,
        "session_id": 82
    },
    {
        "rcode_description": "Changes from Session ID successfully accepted.",
        "rcode": 0,
        "session_id": 121
    }
]

/text/reject

Reject modifications of one or more pending edit sessions without having to revise them.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Comma-separated list of Session IDs whose edits are meant to be accepted by the given Author ID.	`27,43,82,121`
author_id	str	Yes	Author ID, authorised by the API client, who revised the changes made in the Session ID session_id.	`lord_stark1`
author_name	str	No	Author Name	`Eddard Stark`
author_conf	int	Yes	Confidence level of the author. Must be `100`.	`100`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`
expire	int	No	Expiration date UNIX timestamp of the request (seconds since 01-01-1970 in UTC to the expiration date). Required only if `auth_token` is a request key.	`1434132876`

Example 46. [/text/reject] Accept edits of session IDs 27, 43, 82, 121.

http://my-tlp-server.com/api/text/reject?id=27,43,82,121&&author_id=lord_stark1&author_conf=100&revision_session_id=12&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

Returns a list of dictionaries, one for each Session ID.

[
  {
    "session_id": <int>,
    "rcode" : <int> ,
    "rcode_description" : <str>
  },
  ...
]

session_id:`<int>`	Session ID.
rcode:`<int>`	Return code. `0` → Changes from Session ID successfully rejected. `1` → Session ID does not exist. `2` → Unallowed access to the provided Session ID. `3` → Session ID has not been closed yet. `4` → Session ID did not require revison. `5` → Session ID was already revised before.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Trying to reject edits of session IDs 27, 43, 82, 121.

[
    {
        "rcode_description": "Session ID does not exist.",
        "rcode": 1,
        "session_id": 27
    },
    {
        "rcode_description": "Session ID did not require revision.",
        "rcode": 4,
        "session_id": 43
    },
    {
        "rcode_description": "Session ID already revised.",
        "rcode": 5,
        "session_id": 82
    },
    {
        "rcode_description": "Changes from Session ID successfully rejected.",
        "rcode": 0,
        "session_id": 121
    }
]

Appendix I: Web Service API: /web API specifications

Please note that all the following interfaces will share a common prefix: /web.

Interfaces to be used by the API client:

/manage	Features four sub-interfaces to list, add, edit and delete websites.
/systems	Get a list of all available Machine Translation systems that can be applied to translate a document file.

Interfaces to be used by the client web browser:

/getjs	Returns a JavaScript plugin that enables web translation on web browsers.
/translate	Requests translations of a given set of texts, sending part or all of them to translate if no translations are available yet.
/amend	Send user corrections of translated texts.

Interfaces to be used by the TLP Web Translation Editor:

/editor

Features three sub-interfaces: /langs, /get and /mod.

/web/manage

This interface allows the client to manage all web sites registered on TLP.

The /manage interface is split into the following four sub-interfaces:

/list → List all user’s websites.
/add → Add/register a new website.
/edit → Edit an existing website.
/delete → Delete an existing website.

Every sub-interface has different input specifications, please refer to each sub-section.

/web/manage/list

List all user’s websites.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 47. [/web/manage/list] Get list of websites (GET with multiple parameters).

http://my-tlp-server.com/api/web/manage/list?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
   {
     "name": <str>,
     "language": <str>,
     "key": <str>,
     "active": <bool>,
     "url_pattern": <str>,
     "options": <dict>
   },
   ...
]

name:`<str>`	Website name
language:`<str>`	Website’s source language code (ISO-639-1).
key:`<str>`	Website key.
active:`<bool>`	Tells whether a website is active (new text contents are sent to be translated) or not.
url_pattern:`<str>`	URL pattern: only website’s pages that match the pattern are translated.
options:`<dict>`	JSON dictionary with website options, such as languages to which the website has to be translated.

Output Examples

Returning list of websites.


[
    {
        "name": "www.mllp.upv.es",
        "language": "en",
        "key": "bf6fc9bfc04f752047f7b932fedc861e",
        "active": true,
        "url_pattern": "www.mllp.upv.es",
        "options": {
            "requested_langs": {
                "fr": {
                    "sid": 42
                },
                "sl": {
                    "sid": 50
                },
                "ca": {
                    "sid": 10
                },
                "es": {
                    "sid": 27
                },
                "it": {
                    "sid": 37
                }
            }
        }
    }
]

/web/manage/add

Add/register a new website.

Input

Parameter name	Type	Required	Description	Example
name	str	Yes	Website name.	`MLLP`
language	str	Yes	Website source language code (ISO 639-1).	`en`
url_pattern	str	Yes	URL Pattern. Only those URLs that match the given pattern are sent to be translated. Supported wildcard patterns: *.	`www.mllp.upv.es`
requested_langs	dict	Yes	JSON dictionary defining languages to which the website will be translated. Please refer to Requesting translation languages, "requested_langs" option.	`{"es":{},"ca":{"sid":27}}`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 48. [/web/manage/add] Add a new website (GET with multiple parameters).

http://my-tlp-server.com/api/web/manage/add?name=MLLP&language=en&url_pattern=www.mllp.upv.es&requested_langs=%7B%22es%22%3A%7B%7D%2C%22ca%22%3A%7B%22sid%22%3A27%7D%7D&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>,
  "id":<str>
}

rcode:`<int>`	Return code. `0` → New website successfully added/registered. `1` → Unknown `language` code. `2` → Provided `url_pattern` overlaps the URL pattern of an existing website.
rcode_description:`<str>`	Description of the return code (rcode).
id:`<str>`	Website key, that will be required by other interfaces to identify the website.

Output Examples

Website successfully added.

{
  "rcode":0,
  "rcode_description":"New website successfully added",
  "id":"bf6fc9bfc04f752047f7b932fedc861e"
}

/web/manage/edit

Edit an existing website.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`
enabled	bool	No	Enable/Disable translation of new contents for this website.	`false`
url_pattern	str	No	URL Pattern. Only those URLs that match the given pattern are sent to be translated. Supported wildcard patterns: *.	`www.mllp.upv.es`
requested_langs	dict	No	JSON dictionary defining languages to which the website will be translated. Please refer to Requesting translation languages, "requested_langs" option.	`{"es":{},"ca":{"sid":27}}`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 49. [/web/manage/edit] Disable new translations for an existing website (GET with multiple parameters).

http://my-tlp-server.com/api/web/manage/edit?id=bf6fc9bfc04f752047f7b932fedc861e&enabled=false&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>
}

rcode:`<int>`	Return code. `0` → Website successfully edited. `1` → Unknown website `key`. `2` → Provided `url_pattern` overlaps the URL pattern of an existing website.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Website successfully edited.

{
  "rcode":0,
  "rcode_description":"Website successfully edited"
}

/web/manage/delete

Delete an existing website. This action can only be performed by admin API users.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`
purge	bool	No	Delete and purge all website contents. By default, manually supervised translations are preserved (`false`).	`true`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 50. [/web/manage/delete] Delete and purge all contents of an existing website (GET with multiple parameters).

http://my-tlp-server.com/api/web/manage/delete?id=bf6fc9bfc04f752047f7b932fedc861e&purge=true&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>
}

rcode:`<int>`	Return code. `0` → Website successfully deleted. `1` → Unknown website `key`.
rcode_description:`<str>`	Description of the return code (rcode).

Output Examples

Website successfully deleted.

{
  "rcode":0,
  "rcode_description":"Website successfully deleted"
}

/web/editor

This interface is called by the TLP Web Translation Editor and it is split into the following three sub-interfaces:

/langs → Returns a list of translation languages available for a specific website.
/get → Returns all existing source texts or translations of a specific website.
/mod → Updates text translations of a specific website and language.

Every sub-interface has different input specifications, please refer to each sub-section.

/web/editor/langs

Returns a list of translation languages available for a specific website.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 51. [/web/editor/langs] Get list of translation languages available (GET with multiple parameters).

http://my-tlp-server.com/api/web/editor/langs?id=bf6fc9bfc04f752047f7b932fedc861e&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>,
  "web_lang_code":<str>,
  "web_lang_name":<str>,
  "langs": [
     {
       "lang_code":<str>,
       "lang_name":<str>
     },
     ...
   ]
}

rcode:`<int>`	Return code. `0` → Returning language list. `1` → Unknown website `id`.
web_lang_code:`<str>`	Language code (ISO-639-1) of the document language.
web_lang_name:`<str>`	Language name of the document language.
langs:`<list:dict>`	List of languages available. `<dict>` keys: lang_code:`<str>` → Subtitle language code (ISO-639-1). lang_name:`<str>` → Local language name.

Output Examples

Returning available translation languages.

{
    "rcode_description": "Returning language list",
    "rcode": 0,
    "web_lang_code": "en",
    "web_lang_name": "English",
    "langs": [
        {
            "lang_code": "fr",
            "lang_name": "Français"
        },
        {
            "lang_code": "ca",
            "lang_name": "Català"
        },
        {
            "lang_code": "it",
            "lang_name": "Italiano"
        },
        {
            "lang_code": "sl",
            "lang_name": "Slovene"
        },
        {
            "lang_code": "es",
            "lang_name": "Español"
        }
    ]
}

/web/editor/get

Returns all existing source texts or translations of a specific website.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`
language	str	Yes	Language code (ISO 639-1).	`en`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 52. [/web/editor/get] Get all source (en) texts (GET with multiple parameters).

http://my-tlp-server.com/api/web/editor/get?language=en&id=bf6fc9bfc04f752047f7b932fedc861e&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Example 53. [/web/editor/get] Get all Spanish (es) translations (GET with multiple parameters).

http://my-tlp-server.com/api/web/editor/get?language=es&id=bf6fc9bfc04f752047f7b932fedc861e&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>,
  "texts":<dict>
}

rcode:<int>

Return code.

0 → Returning texts.
1 → Unknown website id.

texts:<dict>

JSON dictionary of text hashes and text contents. <dict> keys are text hashes, whose values are another <dict> with the following keys:

text:<str> → Text contents.
status:<str> → Supervision status of the text. It can be automatic or revised. NULL when returning source texts.
added:<str> → Timestamp in which the source text was added to the database. NULL when returning translations.
last:<str> → When returning source texts, it is the last access time performed from any web browsers when visiting the website. In case of translations, it is the timestamp of the last text update.

Output Examples

Returning all source (English) texts.

{
    "rcode_description": "Returning texts",
    "rcode": 0,
    "texts": {
        "dcbe226b3510c277e9fe42f89f179f6e": {
            "status": null,
            "text": "See larger map",
            "added": "2016-05-31 14:31:49.340621",
            "last": "2016-06-21 12:33:58.247419"
        },
        "8f9bfe9d1345237cb3b2b205864da075": {
            "status": null,
            "text": "User",
            "added": "2016-06-02 11:33:45.780144",
            "last": "2016-06-29 20:23:38.410331"
        }
    }
}

Returning all Spanish translations.

{
    "rcode_description": "Returning texts",
    "rcode": 0,
    "texts": {
        "dcbe226b3510c277e9fe42f89f179f6e": {
            "status": "revised",
            "text": "Ver mapa más grande",
            "added": null,
            "last": "2016-07-08 11:01:30.205199"
        },
        "8f9bfe9d1345237cb3b2b205864da075": {
            "status": "automatic",
            "text": "Usuario",
            "added": null,
            "last": "2016-06-02 11:37:02.346747"
        }
    }
}

/web/editor/mod

Updates text translations of a specific website and language.

Input

Parameter name	Type	Required	Description	Example
id	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`
language	str	Yes	Language code (ISO 639-1).	`en`
text	dict	Yes	Dictionary of text hashes and supervised translations.	`{"0321c2c507c5b21377a6bad5bbcd6ec2":"Esto es un texto traducido y corregido"}`
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 54. [/web/editor/mod] Update a Spanish (es) translation (GET with multiple parameters).

http://my-tlp-server.com/api/web/editor/mod?text=%7B%220321c2c507c5b21377a6bad5bbcd6ec2%22%3A%22Esto%20es%20un%20texto%20traducido%20y%20corregido%22%7D&language=es&id=bf6fc9bfc04f752047f7b932fedc861e&user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>,
  "translations":<int>
}

rcode:`<int>`	Return code. `0` → Returning texts. `1` → Unknown website `id`.
rcode_description:`<str>`	Description of the return code (rcode).
translations:`<int>`	Number of translations modified.

Output Examples

Translations successfully updated.

{
  "rcode":0,
  "rcode_description":"Updated 1 translations.",
  "translations": 1
}

/web/systems

Get a list of all available MT Systems that can be applied to translate webpages.

Input

Parameter name	Type	Required	Description	Example
user	str	Yes	TLP Username / API Username.	`tluser`
auth_token	str	Yes	Authentication token for the provided `user`.	`edbab44c3a3f1ca8db4de8277a3b`

Example 55. [/web/systems] Get list of available systems.

http://my-tlp-server.com/api/web/systems?user=tluser&auth_token=edbab44c3a3f1ca8db4de8277a3b

Output

JSON Object: Content-Type = application/json

[
    "mt": [
        {
            "source_lang": <str>,
            "target_lang": <str>,
            "id": <int>,
            "name": <str>,
            "description": <str>
        },
        ...
    ]
]

mt:<list:dict>

List of all available Machine Translation Systems. <dict> keys:

source_lang:<str> → MT System source language code (ISO-639-1).
target_lang:<str> → MT System target language code (ISO-639-1).
id:<int> → System ID.
name:<str> → Name of the system.
description:<str> → Description of the system.

The system ID might be used to explicitly request to the Ingest Service the application of a particular MT system. For further information please see Requesting Subtitle Languages).

Output Examples

Systems List returned.

{
    "mt": [
        {
            "source_lang": "ca",
            "target_lang": "es",
            "id": 14,
            "name": "Catalan-Spanish MT System",
            "description": ""
        },
        {
            "source_lang": "es"
            "target_lang": "ca",
            "id": 11,
            "name": "Spanish-Catalan MT System",
            "description": ""
        },
        {
            "source_lang": "en",
            "target_lang": "es",
            "id": 73,
            "name": "English-Spanish MT System",
            "description": ""
        },
        {
            "source_lang": "es"
            "target_lang": "en",
            "id": 24,
            "name": "Spanish-English MT System",
            "description": ""
        }
    ]
}

/web/getjs

Generates a JavaScript plugin that enables the translation of a given website on the client-side (on web browsers). This interface should be called by the client web browser when importing the JS plugin.

Input

This interface only supports GET requests.

Parameter name	Type	Required	Description	Example
key	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`

Example 56. [/web/getjs] Get Javascript plugin for a given website key.

http://my-tlp-server.com/api/web/getjs?key=bf6fc9bfc04f752047f7b932fedc861e

In order to translate all website’s pages, these pages must include, inside the <header> tag, a <script> tag with the src parameter pointing to this interface. Example:

 <header>
   <script src="http://my-tlp-server.com/api/web/getjs?key=bf6fc9bfc04f752047f7b932fedc861e"></script>
 </header>

/web/translate

Requests translations of a given set of texts, sending part or all of them to translate if no translations are available yet. This interface is called by the JavaScript plugin generated by the /web/getjs interface.

Input

This interface only supports POST requests.

Parameter name	Type	Required	Description	Example
key	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`
language	str	Yes	Translation language code (ISO 639-1).	`es`
text	dict	Yes	Dictionary of text hashes and text contents.	`{"0321c2c507c5b21377a6bad5bbcd6ec2":"This is a text to translate"}`

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>,
  "translations":<dict>
}

rcode:`<int>`	Return code. `0` → New website successfully added/registered. `1` → Could not get HTTP referer. `2` → Unknown website `key`. `3` → Unallowed HTTP referer URL.
rcode_description:`<str>`	Description of the return code (rcode).
translations:`<dict>`	Dictionary of text hashes and translated text contents.

Output Examples

Returning available translations.

{
  "rcode":0,
  "rcode_description":"Returning available translations.",
  "translations": {
      "0321c2c507c5b21377a6bad5bbcd6ec2":"Esto es un texto traducido"
  }
}

/web/amend

Sends user corrections of translated texts. This interface is called by the JavaScript plugin generated by the /web/getjs interface.

Input

This interface only supports POST requests.

Parameter name	Type	Required	Description	Example
key	str	Yes	Website key.	`bf6fc9bfc04f752047f7b932fedc861e`
language	str	Yes	Translation language code (ISO 639-1).	`es`
text	dict	Yes	Dictionary of text hashes and supervised translations.	`{"0321c2c507c5b21377a6bad5bbcd6ec2":"Esto es un texto traducido y corregido"}`
pass	str	Yes	MD5 hash of API user’s password.	`9bbb51190e5586c1bf3dd33a6f310d5a`

Output

JSON Object: Content-Type = application/json

{
  "rcode":<int> ,
  "rcode_description":<str>,
  "translations":<int>
}

rcode:`<int>`	Return code. `0` → New website successfully added/registered. `1` → Could not get HTTP referer. `2` → Unknown website `key`. `3` → Unallowed HTTP referer URL. `4` → Invalid user password.
rcode_description:`<str>`	Description of the return code (rcode).
translations:`<int>`	Number of translations modified.

Output Examples

Translations successfully updated.

{
  "rcode":0,
  "rcode_description":"Updated 1 translations.",
  "translations": 1
}

Appendix J: Generating a Request Key

The request key is an alternative authentication method for the TLP Web Service that avoids revealing the API secret key of the client in the requests to the Web Service. This method can be used only with a reduced subset of Web Service interfaces (see Web Service API Specification Preface).

A request key depends on the values of some call parameters, and therefore it has to be explicitly generated for each API call.

The request key token is divided in two parts. The first part - the basic request key - is mandatory for all interfaces, while the second part is needed only when calling the /mod interface. Both parts are SHA-1 sums of a string composed by the concatenation of different call parameters plus the user’s API secret key. The concatenation of both key parts becomes the full request key.

On the one hand, the first part (basic request key) consists of the SHA-1 sum (40 bytes length) of:

id API call parameter
expire API call parameter
user API call parameter
User’s secret key.

 SHA-1 (id + expire + user + secret_key)

On the other hand, the second part is the SHA-1 sum (40 bytes length) of:

author_id API call parameter
author_conf API call parameter
expire API call parameter
user API call parameter
User’s secret key.

 SHA-1 (author_id + author_conf + expire + user + secret_key)

The full request key (80 bytes length) is the concatenation of both parts:

 SHA-1 (id + expire + user + secret_key ) + SHA-1 (author_id + author_conf + expire + user + secret_key)

It is important to note that all Web Service interfaces will read only the first part (the first 40 bytes, ignoring the remaining bytes) of the request key, except the /mod one that requires the full request key.

Example 57. Generation of a request key.

Input parameters:

id = media-1234
expire = 1433875891
user = tluser
author_id = player_user_1234
author_conf = 100
secret_key = mhes28gfj7vfdg7ylnpapom26ksjfyvjmsoe

First part (Basic request key):

SHA-1 (id + expire + user + secret_key)

SHA-1 (media-12341433875891tlusermhes28gfj7vfdg7ylnpapom26ksjfyvjmsoe)

0cce9cd1c2a0f8d4cd1486bc317c24cb137fbf58

Second part:

SHA-1 (author_id + author_conf + expire + user + secret_key)

SHA-1 (player_user_12341001433875891tlusermhes28gfj7vfdg7ylnpapom26ksjfyvjmsoe)

5ff4dec6d50b3aac13f3d0a7de4e3bb111bc107d

Full request key:

0cce9cd1c2a0f8d4cd1486bc317c24cb137fbf585ff4dec6d50b3aac13f3d0a7de4e3bb111bc107d

Appendix K: Python Commandline Utilities

TLP offers two useful Python command-line scripts to interact with the TLP Server. These scripts are ws-client.py, tx-editor-url-generator.py, player-url-generator.py, tl-editor-url-generator.py and web-tl-editor-url-generator.py. All them require the libtlp python library, as well as a configuration file properly set up.

On the client tools package, all this files are located at:

 client-tools/python/scr/ws-client.py
 client-tools/python/scr/tx-editor-url-generator.py
 client-tools/python/scr/player-url-generator.py
 client-tools/python/scr/tl-editor-url-generator.py
 client-tools/python/scr/web-tl-editor-url-generator.py
 client-tools/python/scr/config.ini

Configuration File

Both scripts requires a configuration file (config.ini) properly set up in order to work. Each parameter of the configuration file is detailed below.

[general]

web_service_url = <str>: Base URL location of the TLP Web Service.
transcription_editor_url = <str>: Base URL location of the TLP Transcription Editor.
player_url = <str>: Base URL location of the TLP Media Player.
translation_editor_url = <str>: Base URL location of the TLP Translation Editor.
web_translation_editor_url = <str>: Base URL location of the TLP Web Translation Editor.

[http_auth]

enabled = <bool>: Enable or disable HTTP Authentication.
username = <str>: HTTP Auth username.
password = <str>: HTTP Auth Password.

[api_client_auth]

username = <str>: TLP username / API username.
secret_key = <str>: API Secret Key Authentication Token.
url_lifetime = <int>: Time slot, starting from the generation of the URL, in which the user will be allowed to access the Player/Editor (expire input parameter).

[player_user_info]

user_id = <user_id>: Client-side user ID of the user who will edit the subtitles/translations (author_id input parameter).
user_full_name = <user_full_name>: Client-side user name of the user who will edit the subtitles/translations (author_name input parameter). (optional)
user_confidence = <user_confidence>: Confidence level of the above user (author_conf input parameter).

Example of WSClient Config file

 [general]
 web_service_url = http://my-tlp-server.com/api
 player_url = http://my-tlp-server.com/player
 translation_editor_url = http://my-tlp-server.com/editor

 [http_auth]
 enabled = no
 username =
 password =

 [api_client_auth]
 username = tluser
 secret_key = akjsfd982323098qwjs209823id09321io3290d
 request_key_expire_lifetime = 1440

 [player_user_info]
 user_id = jsnow21
 user_full_name = John Snow
 user_confidence = 100

By default, both scripts will attempt to load a configuration file named config.ini from the same script directory. You can provide an alternative configuration file location with the --config-file option. The option --print-sample-config-file prints a sample config file to the standard output.

ws-client.py

ws-client.py is a Python command-line utility that interacts with the TLP Web Service API. It can perform queries to all Web Service interfaces. The configuration file has to be properly set up in order to work.

Software requirements

Python 2
libtlp Python Library (included in the TLP Client Tools Package).

ws-client.py help:

usage: ws-client.py [-h] [-d] [-g] [-D] [-c <file>] [-C] [-I <user_tuple>]
                    [-k] [-K REQUEST_KEY_LIFETIME] [-B <username>] [-f <dest>]
                    {speech,text,web} ...

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Print debug information
  -g, --use-get-query   Use GET HTTP queries instead of POST when possible
  -D, --use-data-param  Use a single base64-encoded JSON 'data' GET parameter
                        instead of multiple GET parameters
  -c <file>, --config-file <file>
                        Config file. Default: config.ini
  -C, --print-sample-config-file
                        Print sample config file and exit
  -I <user_tuple>, --api-client-auth <user_tuple>
                        API client user name and authentication token, in the
                        following format: USERNAME:AUTH_TOKEN. Default: from
                        config file.
  -k, --use-request-key
                        Use request-key as auth_token instead of secret_key.
  -K REQUEST_KEY_LIFETIME, --request-key-lifetime REQUEST_KEY_LIFETIME
                        Set request key lifetime in minutes. Default: from
                        config file.
  -B <username>, --su <username>
                        su (substitute user) option: specify username. Only
                        for admin users.
  -f <dest>, --store-output-file <dest>
                        Store the Web Service response in a file.

Available Commands (Web Service's sub-APIs):
  The Web Service implements three sub-APIs for the three main classes of objects than can be processed by TLP.
  These sub-APIs are implemented in this script and accessible by their respective commands.

  {speech,text,web}
    speech              /speech API: is aimed to deal with video and audio
                        files.
    text                /text API: is intended to deal with text documents and
                        contents.
    web                 /web API: is intended to specifically deal with text
                        contents extracted from html documents from the client
                        (web browser) side.

This script makes use of the TLP Python Library and features almost all of TLP Web Service’s interfaces, accessible using commands and sub-commands that are named as the paths of the corresponding interfaces.

More precisely, commands correspond to the sub-APIs featured by the Web Service, while sub-commands are the paths defined by the interfaces implemented by each sub-API.

For instance, to call the /speech/ingest/new interface, you just have to call this script setting the corresponding sequence of commands and sub-commands:

python ws-client.py speech ingest new (...)

Every command and sub-command have specific arguments and options. You can check them easily adding the -h (--help) option after the command or sub-command. For instance:

python ws-client.py speech -h
python ws-client.py speech ingest -h
python ws-client.py speech ingest new -h

Example 58. To upload a new English video file (with ID "1234") and request English and Spanish subtitles [/speech/ingest/new API call]:

python ws-client.py speech ingest new -l "en" -t "My first video" -M "/path/to/my/video.mp4" -p en -p es "1234"

Example 59. To check the status of an uploaded video, using an upload ID returned by the /speech/ingest/new interface (i.e. "up-1234") [/speech/status API call]:

python ws-client speech status "up-1234"

Example 60. To get the subtitle languages available for the video ID "1234" [/speech/langs API call]:

python ws-client.py speech langs "1234"

Example 61. To get English subtitles in SRT format for the video ID "1234" [/speech/get API call]:

python ws-client.py speech get -f "srt" "1234" "en"

tx-editor-url-generator.py

tx-editor-url-generator.py is a Python command-line utility that generates valid URL links to the Transcription Editor according to these specifications. The configuration file has to be properly set up in order to work.

Software requirements

Python 2
libtlp Python Library (included in the TLP Client Tools Package).

tx-editor-url-generator.py help

usage: tx-editor-url-generator.py [-h] [-d] [-C] [-c <file>] [-s START_TIME]
                               [-l LANGUAGE] [-S SESSION_ID] [-t TIME_SLOT]
                               [-a AUTHOR_ID] [-n AUTHOR_NAME]
                               [-k AUTHOR_CONF] [-I <user_tuple>]
                               media_id

tx-editor-url-generator.py: Generate valid URLs for calling the TLP Transcription Editor.

positional arguments:
  media_id              Media ID

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Print debug information
  -C, --print-sample-config-file
                        Print sample config file and exit
  -c <file>, --config-file <file>
                        Config file. Default: config.ini
  -s START_TIME, --start-time START_TIME
                        Start time in seconds.
  -l LANGUAGE, --language LANGUAGE
                        Transcription language.
  -S SESSION_ID, --session-id SESSION_ID
                        Review subtitle modifications made under the given
                        Session ID.
  -t TIME_SLOT, --time-slot TIME_SLOT
                        Time slot for editing in minutes. Default: from config
                        file.
  -a AUTHOR_ID, --author-id AUTHOR_ID
                        Author ID ('author_id'). Default: from config file.
  -n AUTHOR_NAME, --author-name AUTHOR_NAME
                        Author Name. Default: from config file.
  -k AUTHOR_CONF, --author-conf AUTHOR_CONF
                        Author confidence level [0-100]. Default: from config
                        file.
  -I <user_tuple>, --api_user <user_tuple>
                        User name and authentication token, in the following
                        format: USERNAME:AUTH_TOKEN. Default: from config
                        file.

Example 62. tx-editor-url-generator.py call: Get URL for editing the transcription of a given media ID:

--$ tx-editor-url-generator.py MEDIA-ID-1234

player-url-generator.py

player-url-generator.py is a Python command-line utility that generates valid URL links to the TLP Player according to these specifications. The configuration file has to be properly set up in order to work.

Software requirements

Python 2
libtlp Python Library (included in the TLP Client Tools Package).

player-url-generator.py help

usage: player-url-generator.py [-h] [-d] [-C] [-c <file>] [-s START_TIME]
                               [-l LANGUAGE] [-S SESSION_ID] [-t TIME_SLOT]
                               [-a AUTHOR_ID] [-n AUTHOR_NAME]
                               [-k AUTHOR_CONF] [-I <user_tuple>]
                               media_id

player-url-generator.py: Generate valid URLs for calling the TLP Player.

positional arguments:
  media_id              Media ID

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Print debug information
  -C, --print-sample-config-file
                        Print sample config file and exit
  -c <file>, --config-file <file>
                        Config file. Default: config.ini
  -s START_TIME, --start-time START_TIME
                        Start time in seconds.
  -l LANGUAGE, --language LANGUAGE
                        Subtitles language.
  -S SESSION_ID, --session-id SESSION_ID
                        Review subtitle modifications made under the given
                        Session ID.
  -t TIME_SLOT, --time-slot TIME_SLOT
                        Time slot for editing in minutes. Default: from config
                        file.
  -a AUTHOR_ID, --author-id AUTHOR_ID
                        Author ID ('author_id'). Default: from config file.
  -n AUTHOR_NAME, --author-name AUTHOR_NAME
                        Author Name. Default: from config file.
  -k AUTHOR_CONF, --author-conf AUTHOR_CONF
                        Author confidence level [0-100]. Default: from config
                        file.
  -I <user_tuple>, --api_user <user_tuple>
                        User name and authentication token, in the following
                        format: USERNAME:AUTH_TOKEN. Default: from config
                        file.

Example 63. player-url-generator.py call: Get URL for editing English subtitles of a given media ID:

--$ player-url-generator.py -l en MEDIA-ID-1234

tl-editor-url-generator.py

tl-editor-url-generator.py is a Python command-line utility that generates valid URL links to the TLP Translation Editor according to these specifications. The configuration file has to be properly set up in order to work.

Software requirements

Python 2
libtlp Python Library (included in the TLP Client Tools Package).

tl-editor-url-generator.py help

usage: tl-editor-url-generator.py [-h] [-d] [-C] [-c <file>] [-S SESSION_ID]
                                  [-t TIME_SLOT] [-a AUTHOR_ID]
                                  [-n AUTHOR_NAME] [-k AUTHOR_CONF]
                                  [-I <user_tuple>]
                                  media_id language

player-url-generator.py: Generate valid URLs for calling the TLP Player.

positional arguments:
  media_id              Media ID
  language              Translation language.

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Print debug information
  -C, --print-sample-config-file
                        Print sample config file and exit
  -c <file>, --config-file <file>
                        Config file. Default: config.ini
  -S SESSION_ID, --session-id SESSION_ID
                        Review subtitle modifications made under the given
                        Session ID.
  -t TIME_SLOT, --time-slot TIME_SLOT
                        Time slot for editing in minutes. Default: from config
                        file.
  -a AUTHOR_ID, --author-id AUTHOR_ID
                        Author ID ('author_id'). Default: from config file.
  -n AUTHOR_NAME, --author-name AUTHOR_NAME
                        Author Name. Default: from config file.
  -k AUTHOR_CONF, --author-conf AUTHOR_CONF
                        Author confidence level [0-100]. Default: from config
                        file.
  -I <user_tuple>, --api_user <user_tuple>
                        User name and authentication token, in the following
                        format: USERNAME:AUTH_TOKEN. Default: from config
                        file.

Example 64. tl-editor-url-generator.py call: Get URL for editing English subtitles of a given media ID:

--$ tl-editor-url-generator.py -l en MEDIA-ID-1234

web-tl-editor-url-generator.py

web-tl-editor-url-generator.py is a Python command-line utility that generates valid URL links to the TLP Web Translation Editor according to these specifications. The configuration file has to be properly set up in order to work.

Software requirements

Python 2
libtlp Python Library (included in the TLP Client Tools Package).

web-tl-editor-url-generator.py help

usage: web-tl-editor-url-generator.py [-h] [-d] [-C] [-c <file>]
                                      [-t TIME_SLOT] [-I <user_tuple>]
                                      web_key language

web-tl-editor-url-generator.py: Generate valid URLs for calling the TLP Web
Translation Editor.

positional arguments:
  web_key               Web Key
  language              Translation language.

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Print debug information
  -C, --print-sample-config-file
                        Print sample config file and exit
  -c <file>, --config-file <file>
                        Config file. Default: config.ini
  -t TIME_SLOT, --time-slot TIME_SLOT
                        Time slot for editing in minutes. Default: from config
                        file.
  -I <user_tuple>, --api_user <user_tuple>
                        User name and authentication token, in the following
                        format: USERNAME:AUTH_TOKEN. Default: from config
                        file.

Example 65. web-tl-editor-url-generator.py call: Get URL for editing Spanish translations of a given website key:

--$ wen-tl-editor-url-generator.py MYWEBKEY-1234 es

Appendix L: DFXP Format Specification

This Appendix describes a format extension from the original DFXP format. This extension was made in order to reflect the needs of the transLectures EU project:

Confidence measures for automatic transcription and translations have to be reflected in the DFXP document.
Track needs to be kept of all subtitle edits made by human users, starting from an automatic transcription/translation.

For this purpose, new XML tags have been proposed. These tags belong to a new namespace called tl. Therefore, these new XML tags will be something like <tl:XXX>, where tl is the namespace and XXX is the tag. The root <tt> element has been extended in this way:

 <tt xml:lang="en" xmlns="http://www.w3.org/2006/04/ttaf1"
 xmlns:tts="http://www.w3.org/2006/10/ttaf1#style" xmlns:tl="translectures.eu">

The MLLP research group launched with TLP 1.2 an updated version of the DFXP format, namely DFXP v1.1, which enables the modification of the speech segmentation. The counterpart is that user edition history cannot be tracked inside the DFXP file.

DFXP Tags

Tags are defined at four levels: document, segment, group and word. Document tags are located at the head section, while segment, group and word tags are located at the body section. An additional tag to relate alternative transcriptions/translations is also included. A detailed explanation of tags follows:

<tl:document>: This tag defines the attributes of the transcription/translation at the top level. As the attributes are inherited, the value of the attributes defined here are the default values, unless otherwise redefined. It contains a specific attribute to associate the current file to a unique video ID. Abbreviation: <tl:d>.
<tl:current>: This tag defines the current status of the subtitle file with the last modifications made by users, it contains an ordered sequence of text segments or captions. Abbreviation: <tl:c>.
<tl:origin>: This tag defines the former status of the subtitle file (typically the automatic transcription/translation), it contains an ordered sequence of text segments or captions. Abbreviation: <tl:o>.
<tl:segment>:This tag defines text segments or captions. Abbreviation: <tl:s>
<tl:group>: This tag defines a group of words inside a segment. This tag will usually appear as a result of the interaction with the user. Abbreviation: <tl:g>
<tl:word>: A simple tag used to specify single word properties, mostly used for time alignments and confidence measures. Other attributes are generally inherited. Abbreviation: <tl:w>

Next, we define the set of attributes related to the tags just defined. Most of the attributes are applicable to all levels:

authorType: Type of author. Their values are automatic or human. Human for those transcriptions/translations generated by human experts or completely supervised by human experts. Automatic for those transcriptions/translations fully generated by an ASR/MT system. Abbreviation: aT.
authorId: Author identifier. For example: RWTH, XEROX, UPV, Maria Gialama, etc. Abbreviation: aI.
authorConf: Confidence measure of the author when the authorType is human. This attribute is coupled with an authorId. This tag could be useful for non-native users supervising a foreign language. Abbreviation: aC.
wordSegId: It identifies the system that performs the automatic segmentation at the word level. It could be different from the authorId, since groups of words supervised by the user may be segmented at the word level with a different system from that providing the automatic transcription. Abbreviation: wS.
timeStamp: Instant of creation or modification. The timestamp format is a combination of date and time of day in Chapter 5.4 of ISO 8601. The format is [-]CCYY-MM- DDThh:mm:ss[Z$|$(+$|$-)hh:mm]. Abbreviation: tS.
confMeasure: Confidence measure of the level. These values are generated by ASR and MT systems. Abbreviation: cM.
videoId: Tag only defined at the document level. It links the current transcription or translation DFXP file to a unique video. Abbreviation: vI.
segmentId: It is used to uniquely identify a segment in a transcription or translation file. As mentioned above, alternative segments have the same segmentId. Abbreviation: sI.
begin: Instant of the beginning of an audio portion of the current tag in seconds. Abbreviation: b.
end: Instant of the end of an audio portion of the current tag in seconds. Abbreviation: e.
elapsedTime: Processing time. Abbreviation: eT.
modelID: Model used by decoder. Abbreviation: mI.
processingSteps: Processing steps of decoder. Abbreviation: pS.
audioLength: Complete length of the video. Abbreviation: aL.
status: Supervision status of the subtitles. Abbreviation: st. Possible values:
- fully_automatic → All segments are automatic.
- partially_human → Some segments have been supervised.
- fully_human → All segments have been supervised.

Special characters such as & “ < > ' must be escaped in the DFXP files according to the XML standard (see http://xml.silmaril.ie/specials.html).

Examples of Extended DFXP Tags

Examples at <head>

DFXP Format: Document tags

 <tl:d aT="automatic" aI="UPV-v1.0" tS="2012-10-03T21:32:52" aC="0.6"
  cM="0.75" vI="1234-abcd" b="0.0" e="400.6"/>

 <tl:d aT="human" aI="John Doe" tS="2012-10-03T21:32:52" aC="1.0"
  cM="1.0" videoId="1234-abcd" b="1.0" e="400.6"/>

Examples at <body>

DFXP Format: Segment tags

 <tl:s sI="1" aT="automatic" aI="UPV" wS="UPV" tS="2012-10-03T21:32:52" cM="0.62" aC="0.5" b="0.0" e="15.6">
   i am very hungry with you
 </tl:s>

DFXP Format: Group tags

 <tl:g aT="human" aI="John Doe" aC="0.75" tS="2012-10-03T21:32:52" cM="1.0" b="2.7" e="3.5">
    the way we train in IBM
 </tl:g>

DFXP Format: Word tags

 <tl:w aT="automatic" aI="UPV" aC="0.5" cM="0.61" tS="2012-10-03T21:32:52" cM="1.0" b="1.3" e="2.1">the</tl:w>
 <tl:w cM="1.3" b="1.6" e="2.1">way</tl:w>

Use cases

A transcription/translation is automatically generated by an automatic system creating a DFXP file from scratch.
A transcription/translation is manually generated by a human expert creating a new DFXP file.
A user supervises an automatic/manual transcription/translation.

Use case examples

A transcription/translation is generated by an automatic system creating a DFXP file from scratch.

 <?xml version="1.0" encoding="utf-8"?>
 <tt xml:lang="en" xmlns="http://www.w3.org/2006/04/ttaf1"
 xmlns:tts="http://www.w3.org/2006/10/ttaf1#style" xmlns:tl="translectures.eu">
  <head>
    <tl:d aT="automatic" aI="UPV-v1.0" wS="UPV-v1.0" tS="2012-10-03T21:32:52" aC="0.56" cM="0.75"
     videoId="00505-Profesores_Alcoy.M03.B01" b="0.0" e="12.50"/>
  </head>
  <body>
    <tl:c>
      <tl:s sI="1" cM="0.75" b="0.00" e="3.20">
        <tl:w cM="0.85" b="0.00" e="0.75">most</tl:w>
        <tl:w cM="0.89" b="0.75" e="0.95">of</tl:w>
        <tl:w cM="0.63" b="0.95" e="1.15">you</tl:w>
        <tl:w cM="0.40" b="1.15" e="1.35">are</tl:w>
        <tl:w cM="0.90" b="1.35" e="1.50">probably</tl:w>
        <tl:w cM="0.85" b="1.50" e="1.75">ventured</tl:w>
        <tl:w cM="0.55" b="1.75" e="2.00">the </tl:w>
        <tl:w cM="0.98" b="2.00" e="2.75">problem</tl:w>
        <tl:w cM="0.60" b="2.75" e="3.20">that</tl:w>
      </tl:s>
      <tl:s sI="2" cM="0.19" b="8.50" e="12.50">
        <tl:w cM="0.1" b="8.50" e="9.00">To</tl:w>
        <tl:w cM="0.2" b="9.00" e="10.00">solve</tl:w>
        <tl:w cM="0.1" b="10.00" e="10.70">on</tl:w>
        <tl:w cM="0.1" b="10.70" e="12.50">this</tl:w>
      </tl:s>
    </tl:c>
    <tl:o>
      <tl:s sI="1" cM="0.75" b="0.00" e="3.20">
        <tl:w cM="0.85" b="0.00" e="0.75">most</tl:w>
        <tl:w cM="0.89" b="0.75" e="0.95">of</tl:w>
        <tl:w cM="0.63" b="0.95" e="1.15">you</tl:w>
        <tl:w cM="0.40" b="1.15" e="1.35">are</tl:w>
        <tl:w cM="0.90" b="1.35" e="1.50">probably</tl:w>
        <tl:w cM="0.85" b="1.50" e="1.75">ventured</tl:w>
        <tl:w cM="0.55" b="1.75" e="2.00">the </tl:w>
        <tl:w cM="0.98" b="2.00" e="2.75">problem</tl:w>
        <tl:w cM="0.60" b="2.75" e="3.20">that</tl:w>
      </tl:s>
      <tl:s sI="2" cM="0.19" b="8.50" e="12.50">
        <tl:w cM="0.1" b="8.50" e="9.00">To</tl:w>
        <tl:w cM="0.2" b="9.00" e="10.00">solve</tl:w>
        <tl:w cM="0.1" b="10.00" e="10.70">on</tl:w>
        <tl:w cM="0.1" b="10.70" e="12.50">this</tl:w>
      </tl:s>
    </tl:o>
  </body>
 </tt>

A transcription/translation is manually generated by a human expert creating a new DFXP file

 <?xml version="1.0" encoding="utf-8"?>
 <tt xml:lang="en" xmlns="http://www.w3.org/2006/04/ttaf1"
 xmlns:tts="http://www.w3.org/2006/10/ttaf1#style" xmlns:tl="translectures.eu">
  <head>
    <tl:d aT="manual" aI="Maria" aC="1.0" videoId="00505-Profesores_Alcoy.M03.B01"
     tS="2012-10-03T21:32:52" cM="1.0" b="0.0" e="12.50"/>
  </head>
  <body>
    <tl:c>
      <tl:s sI="1" b="0.00" e="3.20">
        most of you have probably ventured into the problem set.
      </tl:s>
      <tl:s sI="2" b="8.50" e="12.50">
        The solution is:
      </tl:s>
    </tl:c>
    <tl:o>
      <tl:s sI="1" b="0.00" e="3.20">
        most of you have probably ventured into the problem set.
      </tl:s>
      <tl:s sI="2" b="8.50" e="12.50">
        The solution is:
      </tl:s>
    </tl:o>
  </body>
 </tt>

A user supervises an automatic transcription/translation by editing a segment.

 <?xml version="1.0" encoding="utf-8"?>
 <tt xml:lang="en" xmlns="http://www.w3.org/2006/04/ttaf1"
 xmlns:tts="http://www.w3.org/2006/10/ttaf1#style" xmlns:tl="translectures.eu">
  <head>
    <tl:d aT="automatic" aI="UPV-v1.0" wS="UPV-v1.0"
    tS="2012-10-03T21:32:52" aC="0.56" cM="0.75"
    videoId="00505-Profesores_Alcoy.M03.B01" b="0.0" e="12.50"/>
  </head>
  <body>
    <tl:c>
      <tl:s sI="1" aT="human" aC="0.81" cM="1.0" aI="John" b="0.17" e="3.32" tS="2012-10-04T13:31:45">
        most of you probably ventured into the problem set
      </tl:s>
      <tl:s sI="2" cM="0.19" b="8.5" e="12.50">
        <tl:w cM="0.1" b="8.5" e="9">To</tl:w>
        <tl:w cM="0.2" b="9" e="10">solve</tl:w>
        <tl:w cM="0.1" b="10" e="10.7">on</tl:w>
        <tl:w cM="0.1" b="10.7" e="12.5">this</tl:w>
      </tl:s>
    </tl:c>
    <tl:o>
      <tl:s sI="1" cM="0.75" b="0.00" e="3.20">
        <tl:w cM="0.85" b="0.00" e="0.75">most</tl:w>
        <tl:w cM="0.89" b="0.75" e="0.95">of</tl:w>
        <tl:w cM="0.63" b="0.95" e="1.15">you</tl:w>
        <tl:w cM="0.40" b="1.15" e="1.35">are</tl:w>
        <tl:w cM="0.90" b="1.35" e="1.50">probably</tl:w>
        <tl:w cM="0.85" b="1.50" e="1.75">ventured</tl:w>
        <tl:w cM="0.55" b="1.75" e="2.00">the </tl:w>
        <tl:w cM="0.98" b="2.00" e="2.75">problem</tl:w>
        <tl:w cM="0.60" b="2.75" e="3.20">that</tl:w>
      </tl:s>
      <tl:s sI="2" cM="0.19" b="8.50" e="12.50">
        <tl:w cM="0.1" b="8.50" e="9.00">To</tl:w>
        <tl:w cM="0.2" b="9.00" e="10.00">solve</tl:w>
        <tl:w cM="0.1" b="10.00" e="10.70">on</tl:w>
        <tl:w cM="0.1" b="10.70" e="12.50">this</tl:w>
      </tl:s>
    </tl:o>
  </body>
 </tt>

The transLectures-UPV Platform (TLP) 3.4.0: User Guide and Documentation

License

1. Introduction

2. Getting Started

2.1. TLP Overview

2.2. Use Cases

2.2.1. TLP for large media repositories

2.2.2. TLP for MOOC platforms

2.2.3. TLP for multilingual web translation

3. Database

4. Web Service

4.1. /speech: API Interfaces for audio/video transcription and translation

4.2. /text: API Interfaces for text/document translation

4.3. /web: API Interfaces for multilingual web translation

4.4. API User Authentication

5. Transcription Editor

6. Media Player

7. Translation Editor

8. Web Translation Plugin

9. Web Translation Editor

10. Ingest Service

10.1. Uploads Workflow

10.1.1. Video/Audio transcription and translation workflow

10.1.2. Text/Document translation workflow

10.2. User Quota

11. Postprocessor

12. Client tools

Appendices

Appendix A: Installation

Database

Requirements

Installation Steps

Web Service

Requirements

Installation Steps

Configuration

Transcription Editor

Requirements

Installation steps

Configuration

Media Player

Requirements

Installation steps

Configuration

Translation Editor

Requirements

Installation steps

Configuration

Web Translation Editor

Requirements

Installation steps

Configuration

Ingest Service

Requirements

Installation Steps

Configuration

Execution

Postprocessor

Requirements

Installation Steps

Configuration

Execution

Appendix B: Calling the TLP Media Player

Input parameters

HTTP call

Appendix C: Calling the TLP Transcription Editor

Appendix D: Calling the TLP Translation Editor

Input parameters

HTTP call

Appendix E: Calling the TLP Web Translation Editor

Input parameters

HTTP call

Appendix F: Web Service API: Technical Preface

Allowed HTTP methods

API Client Authentication

Appendix G: Web Service API: /speech API specifications

/speech/ingest

Manifest JSON

Manifest file JSON Specification

Requesting Subtitle Languages