Bank St. Petersburg got rid of Oracle and almost refused the services of integrators when working with data

For several years, Bank St. Petersburg abandoned Oracle in the corporate data warehouse (QCD) and built a data management platform based on open source solutions and its own development. Gleb Smirnov, deputy CDO, owner of the Bank of St. Petersburg data management platform, spoke about how this happened on September 25 at the TAdviser Banks IT Day conference.

TAdviser

QCD, built on the basis of Oracle by integrators, has been operating since 2015, but back in 2019 the bank decided to replace it with a new data management platform (DMC) based on Hadoop. This was due to the fact, first of all, that satisfying data requests in QCD at Oracle required the involvement of integrators. This caused temporary losses. In addition, there were problems with scalability.

As a result, in 2020-2022 a pre-project was carried out to build a platform and set up data supply streams from sources to the ODS layer (Operational Data Store - online data storage area). In the same period, the construction of the first data marts and reports was implemented, as well as the allocation of the data office into a separate structural unit subordinate to the CFO.

In 2022-2023, the migration of the main reports from CDS to PDA began, as well as the migration of users and their training. At the same time, the bank organized internal development, developed competencies, and scaled the team.

And as of 2024, due to the fact that the bank is dealing with regulatory reporting and the need to comply with the requirements of a business-critical system, a full-fledged platform reserve was built and the PDA was classified by the criticality class of the Business Critical System. Now the development of functionality and consolidation of the data landscape on the platform, the development of Data Governance and the systematization of data management are underway, said Gleb Smirnov.

source = From the presentation of Gleb Smirnov

The new data management platform still has an unimportant BI system, it has yet to be replaced.

The data management platform is based on Arenadata Hadoop. At the start of the project, a contractor was involved in building storage and developing basic frameworks. But now the development of the storage and the development of the platform is being carried out by its own development team.

The introduction of a new solution instead of QCD on Oracle was not without challenges. Among them was the absence at that time in the bank of expertise in servicing a new stack, the habit of working with the "monolith," the lack of understanding of the necessary number of administration teams. There was also no expertise in the development. The data office team became the first internal enterprise development team at Bank St. Petersburg.

When forming an internal development team, there was a difficult task - to introduce all team development processes and develop competencies among existing employees, as well as build production processes. To increase the competencies of the team, the first thing they did was send a team of data offices and administrators for external training. GitFlow and an autotest system were also built. The latter made it possible to reduce the time for debate during development and reduce the number of errors.

source = From the presentation of Gleb Smirnov

As a result of the development of its own development, the bank significantly reduced its dependence on integrators. Now the ratio of internal development on the platform to external development is 80 to 20, Gleb Smirnov cited the data.

Further plans for the development of data management processes include the transition from centralization of data management to federalization. Moving away from centralized ownership of data and development will increase business productivity and increase the number of data products and their monetization, according to Bank St. Petersburg.