Modus ETL Enterprise Storage Management

The main articles are:

• Data management
• Business Intelligence Definition

Modus ETL is a Russian solution designed to control ETL and Data Quality Management processes. It allows you to collect data from many sources, provides processes for verification, standardization, categorization and transformation of data.

Modus ETL allows you to form and ensure the management of enterprise data storage on,, or DBMS MSSQL PostgreSQL. Vertica

Provides fast connection of data sources through ready-made integrations with 1C, SAP, Oracle, various DBMS, Excel files and other web services.

The solution is implemented on the 1C: Enterprise 8.3 platform. Open-source code with the possibility of self-improvement of the 1C product by developers.

It works servers under control OS Windows and. Linux

Modus ETL is included in the unified register of Russian software and is certified "1C: Compatible!."

2025: Modus ETL 1.6.11

On March 10, 2025, Modus announced the release of the Modus ETL 1.6.11 update.

𝓲

Фото: Modus

According to the company, the changes affected the following.

Retrieving data from sources:

• Added fast download of large XLSX and CSV files. Now 1,000,000 XLSX records load in 53 seconds, while 4,000,000 CSV records load in 32 seconds.
• The mechanism for obtaining the structure of storage metadata from 1C databases has been improved: re-obtaining the list of available tables and the composition of their fields (updating metadata) from 1C databases is now ~ 5 times faster.
• The mechanism for checking the correctness of user settings responsible for interaction between the ETL Agent and Modus ETL has been improved. Now, when you check the correctness of user settings, clicking on the "Check availability" button not only checks the availability of the ETL Agent, but also checks the settings of the Modus ETL. So users can identify incorrect settings immediately, without waiting for errors in the process of obtaining data.
• Optimized data retrieval from 1C sources. Now the data is transferred in fixed-size bursts and immediately placed in storage. This helps to circumvent 1C source restrictions on transferring large files and reduces RAM consumption in the source database.
• ETL Agent has been modified: now the filling of the parameters' Address', 'Login', 'Password' is automatically controlled. Also added is the output of errors detected during verification to the work log.
• The test Alpha version adds the ability to use ETL Agent data acquisition modules written in Python. Now, to load data with a complex and non-standard structure, it will be possible to prepare special Python code, embed this code in the data acquisition module in Modus ETL and configure automatic data acquisition.

Data Transformation:

• Now, when copying and transferring data processing scripts, the schema (top) of the step display will be reproduced in the WorkFlow model
• You can use a space as a row separator in the Split Row step template
• In the template of the step "Arbitrary code 1C" added the ability to use "Script parameters" in the code 1C
• In the " Data Transfer" step template, the mechanism for creating a sink table is optimized and adapted for modern versions of ClickHouse
• Added the ability to modify the SQL query to fine-tune the parameters for creating temporary storage tables
• An Advanced Settings button has been added to manage the selected WorkFlow object. You can now access the script step properties and change the relationship types between the steps by selecting the Time Table or Subquery type.
• New templates have been added in data scenarios: Table Cleanup, Table Update, Table Update from Source Table
• ETL Agent has been improved: now the work of script step templates in Machine Learning does not require the installation of a separate special service.

Storage Management:

• The form of creating and editing a vault table has been improved: interactive control of field (column) parameters has been implemented without the need to "fall" into the configuration form of each field. This improves user experience and saves table setup time by ~ 30%.
• The form of editing the storage table has been improved: the ability to create and update a simple dataset on the analytical portal Modus BI based on the storage table has been added without the need to recreate the storage table.
• You have improved the form for creating and editing a vault table by adding the Alias column, which allows you to specify an alias for descriptive documentation for each table field, as well as use it when creating simple datasets for the Modus BI analytical portal.
• The ability to export the description and structure of the storage tables in the form of files with descriptive documentation to PDF, XLSX, DOCX, TXT and more formats has been added. Now the user can use them to prepare project reporting documentation, explanatory notes, instructions, and so on. This optimizes the work required to document the tables and datasets used at least twice.

The Modus team also worked on the information security Modus ETL. Now the password of the ETL connection line when outputting information the log to the file is hidden.

Other modifications:

• "1C: Compatible" Certificate for Modus ETL Received Again
• Updated 1C:Subsystems Library to version 3.1.9.
• The "Basic settings" section has been improved: on the "Other" tab, additional control options have been added to the "Clearing data collection event logs" group (clearing data collection event logs, setting up schedules and cleaning events).
• Updated service form to test interaction between Modus ETL and ETL Agent
• The Modus ETL homepage adds the ability to download ETL Agent and ETL Adapter distributions for 1C. If earlier you had to get distributions on the 1C website or search for them in the directory with 1C configuration templates, now you do not need to do this.

2021

Add Workflow Interface

The updated release of Modus ETL 1.5 introduced the Workflow interface, which is a more functional and convenient alternative to configuring script steps as a table. This was announced on July 21, 2021 by Modus BI.

Modus ETL develops in the low-code concept, which implies the ability to configure ETL operations in the visual interface without the need to write code.

The WorkFlow interface allows you to design data processing scenarios:

• placing the steps on the canvas and associating the steps with each other to transfer data and/or control between the steps,
• setting up data processing rules for the step using ready-made templates (wizards) and rules for collecting statistics and checking data quality.
• when running a script for execution, the interface displays the execution of the script, saves execution logs and data statistics for subsequent analysis.

In addition, the steps of the data processing scenario can be performed in parallel, which provides flexibility and greater performance.

Example of parallel execution of script steps in Workflow:

Modus ETL 1.5 also implements an updated application design: color styles have been changed and the interface has been significantly redesigned for greater ease of use.

Key Data Quality Management Capabilities

• regulatory reference information management
• reporting in reference terms
• transformation of primary data into a target model
• calculation of KPIs

Modus ETL Storage Management

Modus ETL Architecture

2017

Certificate "1C: Compatible!"

In 2017, the product "Modus: Enterprise Data Warehouse Management (ETL)" received the certificate "Compatible! Program system 1C: Enterprise. "

Features and features of the program

Modus: Enterprise Data Warehouse Management (ETL) is designed to build and manage Enterprise Data Warehouse (CDS), automate data collection from a variety of data sources, including data quality management tools.

(Data current as of April 2017)

Data Collection Management:

• Use as a source "1C: Enterprise" (including 1C: Fresh), database data (MS SQL, Oracle), web services, MS Excel files ;
• Possible data receivers: external sources "1C: Enterprise," MS SQL databases;
• The ability to configure and save data collection rules, including settings for the specified data source;
• Ability to load data into multiple threads (multithreading);
• Ability to download data on a schedule, including visualization of the data download schedule;
• The ability to visually design a query in the SQL/1C language;
• The ability to visualize the metadata structure of the 1C: Enterprise database;
• Data Management:
• The ability to configure data transformation rules - scripts in the SQL/1C language;
• The ability to store current data and data change history (data version) in one table;
• Setting up data versioning rules: defining key fields and calculating hash sums;
• Setting up validation and data cleaning rules (using predefined templates);
• Set up data consolidation rules;

Other possibilities:

• The ability to combine data upload rules and data transformation scenarios into packages and set up a schedule for launching packages;
• Data loading and transformation process logging subsystem;
• The ability to notify e-mail about system events;
• The ability to write an arbitrary algorithm (AddOn) when loading and processing data;
• The ability to connect plugins for data processing processes;
• Implementing the data collection analyst (analytics AWS) workplace system.

HandySoft Storage Management (ETL) is not a standalone configuration and requires an installed 1C : Enterprise 8.3 platform (version 8.3.8 at least).

The Handisoft: Enterprise Data Warehouse Management (ETL) configuration was developed using 1C:Subsystems Library functionality.