TAdviser tested PROMT Neural Translation Server. Product overview.

Introduction

TAdviser We have a special attitude towards the company's products. In PROMT 2019, TAdviser and PROMT launched a joint project to create an English-language www.tadviser.com portal, where the entire content www.tadviser.ru site is translated from to Russian language English using PROMT machine translation technologies. For the first two years, translation was carried out on the basis of RBMT technology (PROMT Rule-based Machine Translation) - it provided translation sufficient to understand the meaning, but retained the "machine accent" in the texts. Since the beginning of 2021, NMT technology (- PROMT Neural Machine Translation machine neural network translation) has been launched on the portal, which provides a translation quality comparable to a translation made by a person.

Automatic (machine) translation within the framework of this project is carried out using the PROMT Neural Translation Server server product running in the cloud. It was this solution that TAdviser took for expert testing, but evaluated it for use in a local network or closed loop. This is due to the fact that the main customers of the company's machine translation programs are commercial and government structures, for which the issue of ensuring confidentiality in the translation of corporate information is fundamental . With this in mind, test scenarios are also selected.

Test scenarios

For testing, we identified two types of use of automatic translation and formulated user expectations for each of them.

The first scenario is the translation of a text or document into Russian to understand its meaning. This task regularly arises for users working with materials in foreign languages, such as business correspondence with foreign partners, financial and legal documents from other countries, scientific articles and publications in foreign languages, technical documentation, etc. At the same time, the user can be anyone (IT specialist, manager, civil servant, researcher, expert in a certain subject area, lawyer, engineer), but the requirements for automatic translation to understand the meaning are the same for all users. First of all, a high basic quality of translation is expected with accurate transmission of the meaning of the original and preservation of entities (names, dates, geographical names, abbreviations, product names). In addition, users would like to receive translation in the environment in which they work with text without wasting time copying and pasting text into the application. Users working with documents in foreign languages also need to translate the entire document while preserving the original formatting.

The second scenario, which we also consider important and in demand among users, is automatic translation for further post-editing. Usually this task occurs when you need to prepare a text or document for a foreign addressee in a short time. In such a situation, you can translate the existing text from the Russian language into the reader's language by means of automatic translation, and then be sure to check the resulting translation and correct fundamental inaccuracies. In such a use case, the automatic translation is expected to be of high quality, so that it takes less time to edit it than to translate the text "from scratch."

Another aspect that we have highlighted for consideration is the user's ability to influence the quality of automatic translation declared by PROMT developers. Training a neural model to improve the quality of translation is a highly specialized task that we deliberately left outside the scope of our testing. However, we decided to test the possibility of point correction of individual terms using the user dictionary in practice.

Selecting the Software Configuration for Testing

For testing, the server version of the PROMT translator (PROMT Neural Translation Server product) was chosen, which provides the possibility of multi-user work through the server within the company without transferring translated texts to the Internet.

Taking into account the requirements for the use of domestic software in Russian organizations, the operating system Astra Linux Common Edition, version 2.12, was chosen as the software environment for the server part of the PROMT solution. In the system requirements of the Linux version of PROMT Neural Translation Server, compatibility with this OS was declared, as well as the ability to operate the server in two modes - CPU (by means of the central processor on the server) and GPU (by means of the video card on the server). For GPU mode, a significantly higher translation speed was declared compared to CPU mode, but within the framework of this testing, we decided to limit ourselves to small texts and only a few users, so the CPU mode was chosen as more economical in terms of equipment cost.

The final configuration of the PROMT server part is as follows:

• Dedicated physical machine without Internet access
• CPU: Intel Core i5-7400 (4 cores) 3 GHz
• RAM: 32GB

• ОС: Astra Linux CE 2.12

The PROMT Neural Translation Server server was installed on the specified configuration. The installation and activation of the test provided was carried licenses software developer out normally. PROMT is used as a client part on browsers user workstations. Taking into account the large number of workstations Windows in Russian organizations, a workstation with a browser was chosen as the client part Windows 10. Firefox In addition, the PROMT Agent client application was installed on the expert workstation to translate the selected text in any editor where selection is supported.

Introduction to the program interface

After installing the server part, the administrator provided the expert with an internal link to connect to the translation server through a browser and credentials data for. authorizations The figure below shows the PROMT user interface after clicking on the link and entering credentials.

PROMT user interface

Before testing began, the TAdviser expert did not study product descriptions with screenshots of the user interface to test the first scenario. Today, any user is faced with machine translation in the form of publicly available online services, so we expected the interface to be intuitive. But we studied the documentation and videos related to translation of the selected fragment using PROMT Agent and adding of its translations to the system, or as the developer calls it, solution settings on glossaries.

The first impression of getting to know the interface is its visually calm design, all the main elements are recognizable and logical. The structure of the menu also raises no questions. The main sections are entitled "Text, Document, Agent." The first two possibilities are clear by the names of what such Agent can be read in the section itself by clicking on it, or in the documentation.

On the Text tab, similar to any online translation services, you can print the text or paste the copied text. During testing, it turned out that the Instant Translation option controls whether the translation appears as it is printed or not. It is not entirely clear why you need to disable the appearance of translation as you print, but the option is enabled by default, so there are no special questions for it.

It is worth noting that the original language may not be indicated. By default, there is an option "Define language," but you can also specify the language by selecting the desired one from the list. In our version, a translation from English into Russian and back was available to us. The developer's website states that a total of 50 + languages ​ ​ are available in a variety of combinations.

If you start printing text in the left window, not only its translation appears automatically, but also sentences appear at the bottom containing fragments of our text with translation (Contexts). In the documentation, we found information that these are examples of the use of words and phrases with translations from open sources. These examples are not machine translation, but are performed by professional translators. According to the developer, this is reference information that is useful for editing machine translation or self-compiling texts in a foreign language.

Translation and Contexts

When a separate word is selected in the text, reference information for it - Dictionary subsection will appear in the interface. In it, the user has access to all translation options and collocations with this word.

Contexts and Dictionary

On the Document tab, we see an interface for downloading a document for translation, the ability to specify languages. An advanced mode is also available, where you can also independently specify the format of the downloaded file.

Document translation

In addition, the administrator installed the PROMT Agent application on the expert workstation and specified a link to the internal server with PROMT in the application settings, and also enabled the application launch mode when the operating system starts. Thus, the expert has the opportunity to select text in any application and translate it by the combination of hot keys. We will discuss the operation of this function in the next section.

Test results

Scenario 1

To verify the translation in order to understand the meaning, we have chosen an English-language publication on the topic of cryptography, available on the website of the National Institute of Standards and Technology (NIST). First, we translated the name of this publication in the list on the site, highlighting the text of interest to us (and having received its translation by the combination of hot keys using the PROMT Agent application). The result of PROMT translation was a correct connected text.

Translation of the publication title in the PROMT Agent application.

Next, we decided to translate the entire publication. To do this, we downloaded it as a PDF file from the NIST website and translated the translation of the document through the page on our local server with the PROMT product installed. The document from 11 A4 pages was translated in 4 minutes, the server returned the result of the PDF translation in editable DOCX format. The structure and formatting of the original as a whole were preserved in the translated file, the Russian text read easily and accurately conveyed the meaning of the original.

PDF Translation

Having received a positive result on a small document, we decided to translate a large document in order to assess the throughput and stability of the system. For this task, we have selected a 700-page document on the same site. The translation of a document of this volume took 3.5 hours, at which time we also translated other texts and documents, while the server was fully operational.

After successfully completing the translation of the document, we received the translated file with the original formatting preserved, which is a great advantage. Working with a document that stores all format characteristics and selections is much more convenient than just with text. As for the quality of the translation, it is pleasantly surprising. Basically, this is a correct and competent text that is easy to read.

To check the possibility of changing the translation of terms, we added the term "guidelines" to the user dictionary with the translation of "recommendations" (by default, PROMT translated it as "guidelines," which is also correct, but for example, we need another translation of this term, which appears 58 times in the document).

Entering the term "guidelines" in the dictionary with the translation of "recommendations"

When re-translating the document, the translation of the term "guidelines" was as we indicated it, while the translated terms were used in the word form that the context required.

The work of the dictionary when translating text using the example of the term guidelines (original and translation)

Scenario 2

To check the translation for post-editing, we selected the PROMT presentation file in Russian (15 slides) received from PROMT. Let's say we need to prepare the English version of this presentation for partners in a short time.

The automatic translation of this presentation into English took 1.5 minutes, while editing took no more than 30 minutes of translation and was mainly associated with layout features.

Перевод презентации PowerPoint (original and translation)

Test Results and Work Impressions

PROMT made a pleasant impression. Many features of the product are intuitive to the average user. The quality of the translation also seemed high and sufficient for solving most problems. For those who need to make a translation of perfect quality, it will still take some time to edit, but it is obvious that the use of machine translation will significantly speed up the time for preparing a document in another language.

It seems to us that the most popular opportunity is the translation of documents. It is precisely in order to translate corporate documents without the risk of posting their content somewhere in the cloud, and it is worth purchasing a PROMT solution.

At the same time, all the main functions of the programs are available through the browser and do not require installing the client application on the workstation. Using the Agent to translate the selected text into a hot key combination is an added benefit. Translation without switching between windows is very convenient.

The disadvantages of the product could include the need to deploy and support a dedicated server for PROMT within the organization, as well as a long time for translating large files on the server without a video card.