How can the financial industry conduct data-driven operations?

This article is a speech by Liu Yuandong, the big data architect of Zhongyuan Bank, entitled " Data Analysis Construction Practice of Zhongyuan Bank". The content is very inspiring, so I have organized and shared it as follows.

1. The history of Zhongyuan Bank’s big data construction

Zhongyuan Bank has done a lot of exploration in data in recent years:

• In December 2016, Zhongyuan Bank completed the launch of the first phase of the big data project; in May 2017, it completed the reconstruction and migration of the bank's data mart; in July 2017, it launched self-service analysis based on the new data platform; in October 2017, it began to use external data platforms and historical data platforms;
• In December 2017, we officially cooperated with Sensors Data to access the real-time behavior analysis platform;
• In May 2018, Zhongyuan Bank completed two new projects - data mining and analysis and the launch of a one-stop data subscription platform;
• In June 2018, Zhongyuan Bank launched a batch of new T+0 data marts.

Personally, I believe that the banking industry has gone through the stage from reporting to BI, and the next stage of development is likely to be scenario-based, such as real-time behavioral analysis and T+0 OLAP systems. T+0 is an inevitable trend, and perhaps in a few years all warehouses will become T+0, and it will be difficult for anyone to accept the T+1 scenario.

Note: T+0 is a securities (or futures) trading system. Any trading system that completes the securities (or futures) and price settlement and delivery procedures on the day of the securities (or futures) transaction is called T+0 trading.

T+1 is a stock trading system, which means that stocks bought on the same day cannot be sold until the next trading day. “T” refers to the transaction registration date, and “T+1” refers to the day after the registration date.

2. Platform construction goals and ideas

Zhongyuan Bank has elevated big data governance and application construction to the bank's overall strategic level, and has formulated a goal of being technologically innovative, adhering to the concepts of independent control, openness and sharing, building a unified, complete, convenient, efficient, intelligent and secure big data technology system, and providing full-process, one-stop and intelligent data services.

It is worth emphasizing that the goal of Zhongyuan Bank has always been not to build a system, but to provide a service. For example, when business personnel want to know how many customers have been lost, it is currently difficult to provide them with BI. Even if data is provided, it is difficult for them to calculate it. However, through scenario-based analysis, the dimensions and indicators required for the scenario can be developed, and by retrieving relevant data, it is easy to conduct analysis. Therefore, we plan to turn a scenario into a service and provide it to business teams in the future.

3. Progress and planning of data platform construction

This is the general progress of the platform construction of Zhongyuan Bank:

• In 2016, the construction of the basic platform and the design of the data architecture were completed;
• In 2017, a data integration platform was established, which laid the foundation for the data service engine, OLAP self-service analysis engine and mining analysis platform;
• In 2018, we plan to develop data governance, real-time computing services, graph computing services, data exchange platforms, machine learning platforms, etc.
• In 2019, we may work on more integrated, improved and enhanced projects from a technical perspective.

In many cases, although the technology department took the lead in establishing the technology system, there was no real business implementation. Now we are trying to shift towards achieving leapfrog development of the business.

In June 2018, Zhongyuan Bank specially established a first-level department - the Data Bank Department, which was separated from the Information Technology Department.

In the past, we focused more on technology, but now we realize that using technology to drive business is very costly and difficult in terms of marketing and risk control, and technology does not necessarily drive business to generate profits. From this point of view, we decided to try the scenario-based methodology and logic provided by Sensors Data.

Zhongyuan Bank has made some technology-driven improvements in the past few years, such as: optimizing the centralized delivery capabilities for the entire bank, including fixed and mobile reports, self-service query models, etc. In particular, self-service queries in the banking industry have become more popular in the past few years. Banks that have not done so must do so in the future. This is a necessary stage.

4. Understanding of data analysis needs: Five modes support bank-wide applications

I think the banking industry's data analysis needs may have five scenarios of data interaction modes (as shown above). Zhongyuan Bank has currently built a data laboratory, which is actually a data lake system, which migrates previous warehouses, markets and other platforms to a new computing architecture.

Its main function is to do data modeling and exploratory analysis. The platform is completely independent of the original P2P platform. Unlike before, it only does reporting, but can also do some self-service analysis and data modeling so that it can support the use of more modelers in the future.

We hope that after the results of this project are produced, it can be applied to different types of business scenarios. However, some old models, such as ad hoc query, will always exist from the Chinese perspective because it is difficult to completely replace them with other application scenarios.

5. Data Analysis Platform System Construction

Zhongyuan Bank has currently planned a data platform system, but this system is still evolving.

In the past few years, Zhongyuan Bank has produced nearly 1,000 reports, self-service analysis of more than 20 topics, and a data laboratory platform.

These platforms are aimed at different people, and the reports are mainly aimed at people who use data. Self-service analysis is mainly aimed at lines, such as people who prepare reports for leaders and publish data downwards; data laboratories are mainly aimed at people with a technological background, such as modelers.

In addition to the above three platforms, Zhongyuan Bank has also established a community platform, which we collectively refer to as a one-stop analysis platform.

The Data-Driven Innovation Community was established to lead the data-driven development of the industry. Currently, the Ministry of Science and Technology is leading the construction of this community.

We will publish a large amount of data-driven content in this community, such as data analysis reports, articles, etc. Personally, I believe that the promotion of data-driven work must be led by the business in the future, because even if technical personnel have ideas, it is difficult to achieve profitability, so it is necessary to practice from a business perspective. Now, we will send our own technical personnel to the business department to learn how to perform analysis in business scenarios.

The data analysis platforms mentioned above are still more technical in nature, but I always believe that they will develop in the direction of scenario-based development in the future. Because scenario-based applications have lower costs, for example: lower usage costs, scenario-based applications make it more convenient for business personnel to use; lower talent costs, scenario-based applications do not require the recruitment of many technical personnel.

6. Comprehensive analysis platform architecture for big data

This is the comprehensive analysis platform architecture of Zhongyuan Bank for big data. There is not much difference among various banks in this regard.

However, the Zhongyuan Bank's marketplace is very thin, with only one ODS for the source and 4-5 marketplaces, such as: accounting, internal operations , marketing, etc. Compared with other banks, Zhongyuan Bank has a very light architecture, which can go directly from the source to the marketplace. Some marketplaces do not even have indicator processing, but directly integrate the details, and then provide self-service analysis and reporting, and some are also supplied to the laboratory.

Personally, I believe that ideas like self-service analysis, scenario-based analysis, and data lakes will sooner or later replace the large number of reports we have made before. T+0 will replace T+1. It is only a matter of time. Therefore, our entire architecture is now migrating here.

7. Practice of Analysis Platform Construction - Performance Optimization

In the past few years, Zhongyuan Bank has done some performance tuning on the analysis platform. Generally speaking, when you first switch from reporting to BI, you will definitely face performance issues, because BI itself is a way of exchanging space for flexibility.

We use big data technology to support computing, using 30 physical computing nodes, and about 20 to 30 topic models designed in an anti-paradigm manner for big data, that is, wide table models. Because the wide table model sacrifices a lot of space, it generally runs more smoothly on this big data platform, has higher redundancy, and improved performance.

However, a major feature of the big data platform is its low cost and acceptable capacity expansion, so we now prefer this approach.

8. Practice of Analytical Platform Construction—Quality Improvement

Improving data quality is the focus of Zhongyuan Bank this year, and we are preparing to launch a new data governance project.

Here is a brief introduction:

We have built an online channel for caliber management. For example, when business personnel find data problems when looking at reports, there will be an online channel to directly feedback the problem, and then dedicated technical personnel will follow up. In addition, we also have a channel similar to a knowledge base, which is conducive to the sedimentation of caliber. However, we still need to further improve data governance, such as master data and standard management.

IX. Practice of Analytical Platform Construction - Flexibility Improvement

This is the BI platform of Zhongyuan Bank, which is generally quite useful. I think banks must do BI at a certain stage. Without BI, it is difficult to truly implement data-driven ideas into the business, because business personnel cannot directly access the data and it is difficult to understand the role of data in the future.

Only after they have mastered BI can you tell them that they can do scenario-based analysis and predictive analysis in the future. Now Zhongyuan Bank is developing along this trajectory.

10. Practice of building an analytical platform - data security control

The financial industry involves a lot of sensitive data, and special attention must be paid to data security issues. In order to ensure data security, we have done some technical processing.

for example:

• Download management: We have established a comprehensive data usage review process and set minimum data access authorization, and allocate report access rights on demand;
• Real-time desensitization: Smartbi supports different forms of data desensitization display, which can realize the desensitization display of sensitive information in the foreground. Sensitive data such as ID card number will be hidden in real-time display;
• For the management of usage traces, we will collect and publish report access data to promote self-monitoring and mutual supervision during report usage, so as to timely discover data leakage risks caused by job adjustments and redundant authorizations;
• Behavior monitoring: We use big data and machine learning technologies to build a data security intelligent analysis and early warning platform to conduct intelligent monitoring of data operation behaviors and prevent internal data security incidents.

11. Future Construction Plan - Data In-depth Exploration Service System

Our future construction plan is roughly as shown in the figure above. At present, we are still in the stage of data exploration and multidimensional analysis. We have launched several data modeling projects this year, and a few are listed below.

We have launched a project called cash flow forecasting for outlets. To achieve accurate forecasting, we need to consider many dimensions, such as: outlet dimension, personnel dimension, passenger flow dimension, and weather dimension.

Therefore, this project analyzes and studies the historical data of all branches to predict how much deposits each branch needs every day, and the deposits here refer to cash. This is because the cash inventory is a very large cost for the branch, for example: transportation costs, and there is no interest on the cash deposited in the branch, which is equivalent to just leaving the cash there.

We hope that forecasting projects will produce some interim results next year, and we also hope that through our efforts in data-driven transformation and innovation, we can contribute to the development of the digital transformation of the financial industry.

Summarize

Competition in the financial industry has become increasingly fierce. If you don’t change, you will fail. If you change blindly, you will also fail. Only data-driven can guide the direction.

I hope this article can provide inspiration for the operations and products of the financial industry!