Big Data in Sberbank
Content |
2020: Big Data turn into huge. Sberbank increases data scientists army
On March 4, speaking at the TAdviser Big Data and BI Day 2020 conference, the senior managing director of department of data management of SberData Boris Rabinovich told how it is arranged and one of the platforms of data, largest in Russia, which is used in bank develops. He noted that in Sberbank there are more and more tasks requiring data processing in real time and algorithms of artificial intelligence are for this purpose used.
In bank more and more specialized tools for developers – data-engineers, by data scientists are also created. The quantity of data scientists very big, but is necessary for bank them more and more, Rabinovich says.
According to its information, in Sberbank as of March 4 more than 120 PB of data are saved up. During the quiet periods the bank performs about 12 thousand transactions per second, and restless – to 20 thousand transactions, and information on them is loaded into Factory of data which, in turn, is a part of the digital platform of Sberbank.
In Sberbank more than 200 commands which based on Factory of data develop the products and solutions. Tasks in which solution Factory of data is involved, - a set. Boris Rabinovich gave several examples: the management reporting in real time, regulatory and tax statements, transaction scoring of AI in consumer crediting, etc.
 | To work with 120 petabytes of data, we created the platform, and it grows with such speed that it is not Big Data, but Huge Data any more (huge data are a comment of TAdviser). We will popularize this term, - the senior managing director of department of data management of Sberbank told. |  |
In terms of technologies earlier in Factory of data there was "a roll towards Hadoop", Boris Rabinovich says. But after 2018 in the company realized that this approach was not absolutely correct. Now depending on solvable tasks different technologies, including Oracle and Teradata from which transition to Hadoop was initially performed are offered users of factory.
| | We invest both in open source, and in vendor products that we had a margin of safety, - noted during the performance Boris Rabinovich. | |
In total the bank uses about 100 sources of data loading – internal and external. The technology for work with data (Near Real Time, NRT) allowed to process up to 50 thousand messages per second half a year ago. But now in bank a trend on connecting as much as possible sources, including at the expense of the companies of an ecosystem of Sberbank, in NRT, and this indicator already reaches 300 thousand messages, and at peaks – to 1 million messages, Rabinovich provided data. Until the end of the year, calculate in bank, it will have about 160 sources.
The bank representative stopped on the "supermarket of data" existing also where engineers and analysts of Sberbank can study and order data. Delivery is performed automatically on the set schedule.
Plans of Sberbank include an output the Factories of Data component in own cloud platform of bank – SberCloud, and the offer of tools of factory, including, to foreign market. Part of them is already brought in a cloud, Boris Rabinovich says.
2017
Big Data allows bank to reduce rates on the credits
In the annual report of Sberbank for 2016 issued in April it is said that the analysis of Big Data on activity of clients allowed bank to reduce the level of the idle credits and to reduce risks. It, in turn, "led to interest rates reduction on the credits, formation of special offers with more interesting conditions for different segments of borrowers".
Sberbank told TAdviser that use of Big Data technologies helps to define more precisely the current risk profile of the client, his interests and requirements that as a result allows bank to do "the timely and personalized offers" regarding the provided services.
For example, use of information on cash flow on customer accounts, analyzing their structure of expenditure, the bank can estimate ability of the client to dispose of his money. It, in turn, directly influences the probability of a non-return of money in time.
| | Naturally, such probability is put in a loan interest rate, and we can issue a loan at the lowered rate for the people having high financial literacy and correctly planning the expenditure, - explained TAdviser in Sberbank. | |
As an example of the clients able to calculate correctly the expenditure despite small income, pensioners are, speak in Sberbank. Often the bank issues them the credits on the lowered interest rate.
In the report it is also specified that Sberbank scoops data on clients for the subsequent analysis, including, from social networks and from mobile operators, covering categories of the population from youth to pensioners.
Students of MSU will analyze Big Data of Sberbank
In March, 2017 Sberbank and faculty of calculus mathematics and cybernetics (VMK) of MSU announced opening of research laboratory "VMK-Sberbank" which will specialize in the theory of risk and data analysis for bank. The laboratory will be focused on support of the advanced research and development in the field of statistical techniques of the analysis of Big Data and machine learning.
Answering a question of TAdviser at official opening of laboratory, the vice president of Sberbank Alexander Vedyakhin told that in the register of bank there are about 500 tasks connected with analytics and every quarter is added till 30-50 new tasks. Sberbank is going to solve the most difficult and interesting challenges within new laboratory, he told.
Range of the tasks connected with data analysis in bank, very wide: from the analysis of client experience for providing the optimal credit proposal, to risk management, before information security management and optimization of IT processes, added in Sberbank.
The head of a chair of mathematical statistics of VMK MSU Victor Korolyov who directs laboratory, reported to TAdviser that the first task in data slicing pane which VMK already solved for Sberbank is connected with optimization of collection activity of bank. From the mathematical point of view it is a problem of optimum control of resources. She demanded development of new approaches and use of the technologies connected with machine learning, Korolyov told. Results of its accomplishment are already accepted by Sberbank to implementation, he added.
In the current portfolio of tasks of laboratory there are tasks connected with risk analysis, the analysis of texts, with processing of large volumes of information, for example, to make a portrait of the potential client of bank, Korolyov told TAdviser.
Representatives of Sberbank and VMK MSU told TAdviser that will depend on a type of results of works to whom they will belong at the exit. Results can be presented in the form of models and algorithms, services and applications, in the form of scientific articles, etc. Alexander Vedyakhin told TAdviser that applied results will be on the party of Sberbank. At the same time in the course of the solution of applied tasks there can be also new fundamental results, new approaches which will remain for MSU.
In accomplishment of tasks in laboratory it is going to involve students and graduate students of VMK. How much they will be involved in work of laboratory, on VMK found it difficult to tell. In total at faculty on full-time department about 2000 students study. Generally students of department of mathematical statistics will participate in its work. About 10 employees of faculty VMK will supervise work, representatives of faculty specified TAdviser.
In addition to carrying out research and development the laboratory sets as the purpose to promote training. Sberbank says that students of VMK are very demanded both in the market, and in their bank and in SberTech.
The faculty is interested in that through those important and necessary tasks which are delivered by Sberbank, "pump over", pass as much as possible students and graduate students, the representative of VMK MSU told TAdviser.
Does not disclose the amounts of financing of Sberbank laboratory. Alexander Vedyakhin characterizes them as "sufficient to carry away students, department and that it was interesting to all".
In March, 2016 MSU and Sberbank signed the agreement on strategic cooperation. It provides cooperation in education, research and social and economic activity.
Expansion of opportunities of the Informatica Intelligent Data Platform platform for work with Big Data
For expansion of functionality of the Informatica Intelligent Data Platform platform Sberbank at the beginning of 2017 purchased a component for work with the Big Data Informatica Big Data Management. In more detail about the project here
2016
Sberbank began hunting for specialists in a blockchain and Big Data
Sberbank needs the qualified IT specialists, the head of bank German Gref reported at the beginning of December, 2016 during "direct line" with employees, reports the Prime agency.[1] For them in bank there are possibilities of the serious wages rise.
Gref noted that the bank needs, in particular, specialists in the field of Big Data and also a developer blockchain. According to him, pros in these directions will have high value in bank, and, unlike other employees whose salary is regulated and is limited to the market, to these specialists the bank is ready to raise significantly salaries in process of accumulation of their competences. Read more here.
Sberbank opened the Big Data
On November 22, 2016 Sberbank announced a project startup "Open data" within which the credit institution began to share information on financial activity of the clients. The project is constructed on Big Data technologies. In more detail about the project here.
Creation of distributed system of storage and processing of big data based on Hadoop
Sberbank selected the Hadoop platform as the standard and in the middle of 2016 carried out purchase of distributed system of storage and processing of big data based on this platform. In more detail about the project here.
Investments into GridGain developer
For development of Big Data at the beginning of 2016 Sberbank invested in the developer specializing in it is GridGain Systems company (GridGain). Gref characterized it as the company, "who won the tender against Oracle, IBM and others, it appeared 10 times more these largest companies". In more detail about the transaction and the company here.
2015: Sberbank selected "Yandex" for work with Big Data
In 2015 "Yandex" became the consultant of Sberbank for solving of tasks, connected with processing and the analysis of big arrays of information. For cooperation on this direction the contract for the amount of 13.7 million rubles was signed. In more detail about cooperation here.
The IT passport of projects in Sberbank of the Russian Federation
| Проект | Интегратор | Продукт | Технология | Год |
|---|---|---|---|---|
 | Ultimeta, NTK (Ultimeta) | LotExpert SRM | SRM - Vendor relationship management | 2018 |
| Coursera (company) | Coursera for Business | Systems of distance learning | 2017 |
| Sberbank-Technologies (Belarus) | 2015 |
| The site content is translated by machine translation software powered by PROMT. The machine-translated articles are not always perfect and may contain errors in vocabulary, syntax or grammar. Read original article If you find inaccuracies or errors in the results of machine translation, please write to editor@tadviser.ru. We will make every effort to correct them as soon as possible. |