In the general trend of technological progress towards industrial Internet, Internet experience and traditional financial industry are touching and deeply blending. Enterprise digital transformation is in full swing, the evolution or reconstruction of various mid-stage strategies and the corresponding Internet architecture is the focus of current IT construction. This article is an arrangement of TVP Wang Ye Return teacher’s live speech. And in return, it will introduce the evolution of the technology Center and explain the problems and conditions we meet in practice. And in return, it will lead you to understand its value and future development.
The introduction
This paper will be divided into two stages. The first half introduces the financial full-product trading model, and the second half is the technology practice.
I’m sure many of you have heard the saying that talking about architecture or technology without the whole business scenario is actually going rogue. The technology center is very hot these days, in a variety of conferences, articles have appeared related topics. Many people will think in their minds: as long as the company is big, my business will go to the middle stage, if I do not do the middle stage as if I can not survive, this topic outside the circulation is also very much.
In my opinion, there is no standard in Zhongtai, because every company will practice based on its own business scenes. For example, what I share here is the financial full-product transaction mode. Some people may base on the company’s e-market scene, some may be the scene of the game platform, or some other scenes.
I never think that the middle stage is a standard, because every company has its own characteristics, which are mixed with people, organizational structure, process, SOP and other matters, all of which are more like a corporate management strategy.
Therefore, I think the middle stage is not a technical implementation, but a technical strategy. Including microservices, and SOA before it, it’s not a technology at all. If you say it’s a technology, what does it use? What language it should be in. What about the service? So it’s really a business model that has nothing to do with your technology.
I personally prefer this sentence: it is not a realization, it is actually a strategy or layout. In the middle of this layout, there are four elements, including data and organizational structure in addition to technology and business. The realization of the center is based on their own business to do, some companies have only a single line of business, no reuse, there is no need to do the center.
So why do we have to do middle stage? Whether in the business state, or in the technical state, the short answer is: we need to reuse! Because when our business lines start to increase and gradually form the business division, then we need to have some such technical means, including adjusting the organizational structure, to support the realization of multiple business lines with relatively low cost, so that it is a cost-effective business.
Under what circumstances will the technology medium be valuable
1. Service system evolution
Before talking about the technology of the middle stage, I will briefly introduce to you the evolution of our business system, so that you can understand the background.
Our evolution is similar to that of many companies. We basically started with a single product, and then evolved to a single product that needed to support multiple products, and then redefined. Our company used to be engaged in offline financial transaction business. After yu ‘ebao came out in 2012, we began to do online business. At the beginning, it was a simple transaction. With the increase of product lines and channel suppliers, we had repeated construction process from 2014 to 2016, and formed a business division from 2017 to 2018. At this time, we needed to reconstruct and adjust the structure of our services, structure and business lines, which was quite normal.
The 1.0 development stage is shown in the figure above, because we conduct financial management based on funds, including public and private funds. For the front end, we have cooperation counters of APP, website and external channels. The realization of the transaction is very simple, is to do the transaction service mode and transaction system, including our payment, as well as the operation of the background are very simple, can be solved by connecting the database.
In the 2.0 era, a single architecture is needed to support multiple products. Our software architecture remains the same, but as the volume of transactions increases, we may need to use some basic components, such as Redis and Memcache, as shown in the lower left corner of the image above, which are also commonly used.
We also started to have our own account system, separate our account center from the payment process, and we also used a lot of MQ. There’s an interesting problem that I’m sure many companies have with the infrastructure behind it. Any technology or selection you use will usually be based on the boss of your team, such as your company’s CTO, or some technical debt. Why use these techniques? Because someone used it before, if you take it over, financial, material and human resources cannot solve it in a short time, so you can only use the previous technology. Or your boss thinks something works and asks you to use it.
Therefore, we can find that as we evolve to 2.0, there are various types of technology selection and technical means, which is due to the pressure from business departments, they can only respond to it quickly. In this case, the following four problems will arise:
The front desk is responsible for part of the business assembly, lack of stratification, the system calls each other, the basic components are multifarious, multiple sets of operation background.
For example, let’s say your front end team is busy working on something right now, and your back end team is working on something, but it’s not that busy. The company needs an urgent project. What should we do? At the architectural level, it should be done by the front end. But the front desk is busy right now, keeping projects waiting in line?
For many small and medium enterprises, this is not possible. Since the backstage team is free, you can start working on it first, so it looks like everyone is full. But one of the problems over time is that as you grow quickly, you get a lot of technical debt.
You end up implementing things in the foreground that should have been implemented in the background logic layer. What should be done in the foreground, these logical judgments are done in the background. This leads to the phenomenon that the front desk is responsible for part of the business assembly, especially under the premise of rapid business development. The lack of hierarchical systems calling each other is also caused by this.
What is the foundation of multifarious, multiple sets of operation background repeated construction of the problem need not say more, like this stage I believe we have experienced.
Entering the 3.0 stage, the figure above shows our multi-business lines, including institutional, retail, high-end, piggy bank, product center, account center, batch, background, payment background, statement background and transaction background…… Obviously, we have started to put different products into different business lines.
2. The premise of the technology platform
The technology middle stage is actually a technology strategy, which includes business, data, technology and organizational structure, and even culture.
To do the technology, I think the first two major premise:
(1) The technical organization structure must be vertical
Just like the organizational structure of our company introduced above, the development process of this system went through 1.0, 2.0, 3.0 and so on. After the 3.0 era, we began to consciously say that we want to be a so-called technology center. In the 1.0 and 2.0 era, we were divided by function, as you can see in the chart below, and I believe many companies are basically divided this way.
The red box in the picture is our R & D team. The R & D team only includes R & D. They only do development. Then there are two teams, quality management, which is actually the test team and QA team, including our systems operations team. The organizational structure is divided by function, test next to test, r&d next to R&D.
So, is there a problem with this structure?
To solve this problem, we need to look at our business characteristics. As a strong business driven company, you usually look at your technical team, cost center and product center. Some people may wonder what is strong business drive? The way to think about it is that the product you sell to make money is the product of your business. Like how do we make money? That is, we sell our financial products, bring profits to our customers, they give us money, we live on that.
Maximize productivity efficiency at the lowest cost to help your business generate more revenue and traffic. This is the value of your cost center.
Not to mention quality, so why did I improve efficiency? For a company that is growing fast, which one is most important? It must be efficiency! I can do it regardless of cost, and quality doesn’t matter, as long as there are no problems in the middle of the main link of the transaction, I can go online under the premise of sacrificing quality.
Why do you say that? Because if I have something missing, but as long as the service is cut well, a certain thing is broken, it can be offline, do service degradation is no problem, but your efficiency must be fast, because you have to compete against time.
In our 1.0 and 2.0 era, the functional structure of the organization was very problematic and could not achieve the result of improving efficiency. Why do you say that? Because we had a pain point: the variety of middleware caused by the ass deciding the head!
Each team chooses the middleware independently, so there is no unified maintenance, no unified knowledge accumulation, and no unified guarantee. In the process of fast delivery, there are bound to be obstacles, and development, test, and operations teams have different goals.
For example, test A, the development requires you to only do functional tests and get online quickly, but the test boss requires you to do non-functional tests to ensure quality and avoid blame… Who is it? And get into an endless debate.
After this savage growth process in phases 1.0 and 2.0, we decided to take technical services down, trial and error fast, and take baby steps to address each team’s goals and improve delivery speed. Finally, the entire test team and operation team were split into the current structure, which means that each test, R&D, operation and maintenance team was divided into these functional goal teams.
(2) Business lines are numerous and complex
Again, take ourselves as an example. As shown in the figure below, the 7 red words at the bottom represent the whole category of products made by our company.
The whole integration of these products, not only to their qualifications, licenses put forward high requirements, but also associated with a lot of technical and business problems, the most terrible is the problem of compliance. For example, the things of the CSRC and the INSURANCE regulatory Commission can not be put together, but for customers, it is a very normal thing to buy 500,000 insurance and stock funds for a list of 1 million, but it is not allowed in compliance.
That’s what I’m talking about. There’s a lot of complexity in building systems, and there’s some compliance issues. Because of the variety of products, the system is messy. However, there are many business innovations, which require customized development of front and background systems and increase the difficulty of logical compatibility. IN addition, the business logic is scattered and there is no unified adaptation layer, so ALL IN is required for each test.
As shown in the figure below, based on the previous content, our current entire technology is divided in this way.
The front desk is mainly to do functions, and is now the largest number of people in the whole R&D team, accounting for about 60~70% of the total.
The main solution is middleware automation operation and maintenance and centralized automation. What is the technical background? It’s the whole infrastructure, network security, storage virtualization, including our servers, cloud vendors.
The function of the technical platform is a bit like the adaptation layer during programming. The adaptation layer separates the technical and business capabilities of the whole company from the top. In the middle, we separated the whole team, including the overall KPI and strategy, separated the technical ability from the business ability, and provided technology empowerment to the front desk in a productized way to form a strong support.
As for the test operation and maintenance to the product line can also run efficiently, it depends on your ability to integrate resources in the technology.
Ii. Problems and challenges in the practice of technology in Taiwan
In the evolution of the technology platform, we still face three challenges:
1. “Ass determines head”
Our technical front desk team, as mentioned earlier, is 60 or 70 percent of the staff. Obviously, if I have a front office team of 100 people, maybe 20 to 30 people in the technical center. In this context, a team dealing with so many people on the front end is bound to encounter the following problems.
There are many conflicts between front desk and center stage due to different philosophies, responsibilities, rhythms and missions, plus the butt determines the position of the head. From the perspective of responsibility, the front desk needs rapid response, while the middle desk needs stable and efficient service. One pursues efficiency while the other pursues quality. Contradiction exists naturally.
2. “Damn technical debt”
So the pursuit of efficiency at the front desk, the pursuit of quality, the goals on both sides are not consistent, so how to achieve your needs at the front desk? Let’s take A look at an example scenario. The following figure shows A product in our company’s technology center where both team A and team B need access to distributed caching.
A team and B team refer to the front desk team. Due to business needs, the front Desk team A also proposed the requirement of access cache to the technical Center (there is A proxy-based resource in the middleware product line of the technical Center). Because the technical debt of team A and Team B is different, and the systems of the two teams are different, additional adapters must be required to complete the access.
What is an adapter? It could simply be a proxy-based SDK or one based on a protocol. What if another team had previously contacted Redis directly instead of using an agent? Can only through the middle of the adapter to help it bridge the current system to go inside.
Why do you do that? Because for a mid-stage team, the technical cost of doing so would be low. Because I only have to maintain a system, with two adapters, two SDK packages, can solve.
3. It’s hard to please all
In our technology to do technology selection, basically are out of the boss’s rich experience. If your boss uses A, use A. If your boss uses B, use B. Of course, there are also some technical debt issues, which leads to the awkward situation of who says it counts, who says it doesn’t count.
Iii. Where is the future of technology In the hybrid trading scenario?
As mentioned above, multi-product trading is based on the mixed scenario trading under the multi-financial system. How should our technology go in the future? Now, as our company needs to transform from online to offline, including some adjustments of ourselves and changes in the industry, our technical team is shrinking and the investment in technology is getting smaller and smaller.
The whole technology shelf has been built to such a large scale in the early stage, but now there are not so many people and costs to invest, so it is a very tragic process. When you have more products than people, for example, your technical center provides 30 products, the existing team is only more than 20, you think about your service will do well?
So this is a big problem we have to solve next, the front desk can not change, after all, the business has to do. What do you do when your people shrink but your business is still there? We have to take this out of the way.
For example, when we switch from online to offline, apps and websites are still in place, and outsourcing will limit your efficiency and industry sensitivity. Self-built r&d teams are also limited by their size and technical input. You are short of people, lack of experience, lack of money, coupled with some objective conditions, many small and medium-sized enterprises may only cross the river by feeling the stones, there is no time for trial and error, also have no trial and error capital. But the demand is still there! The company will continue to demand you, so you have to do little by little, break little by little, fix little by little, in a state of exhaustion. Based on this situation, we are finally ready to go on Tencent cloud.
Due to our business transformation, our dependence on online is decreasing, so now we can choose to upload some of our standard services to Tencent Cloud to complete the gradual introduction of our technical standards, tools and teams to the cloud. The background is the same. We can upload all our servers, including our network, to Tencent cloud to solve some problems we need at present.
I think for the construction of zhongtai, it does not mean winning the lottery, it is actually rely on the gradual evolution of each company to generate income. Because you do anything all need to invest, must remember income, is not doing to play.
The technology platform should be based on its own characteristics, it is like fitness, 3 months of effect, 10 months of results, unfortunately, most enterprises are basically a clap head on. Our technology center is based on our previous characteristics, with data, business, team, process and culture in the middle, which has formed our so-called technology center now.
It really addresses what was needed to get us through the rapid growth of 2015-2016. The reason why we can now go to the cloud, and we can also make this process, is because we now turn the whole technology into the front and middle background, can be very clear distinction between the front and the middle and back. We can put the background gradually to Tencent cloud migration, we also know what things need to go to the cloud, what things are not on the cloud.
Follow the public account of Yunga community and reply to “TVP Live” to obtain the PPT of the teacher’s speech ~
Fourth, the Q&A
Q: The company also wants to make mid-taiwan products. Is this the future path of the company?
A: No matter what you do, first of all, you need to have A certain scale, and you need multiple business lines, and then your organizational structure should be relatively vertical, that is, everyone should have A clear division of labor, and you have A lot of reuse requirements, you need to do it.
Q: How do start-up companies build the middle stage at low cost? How do they persuade leaders to build the middle stage and solve the problems of low technical and business efficiency?
A: Personally, if you have A team of less than 100 people and don’t have that many lines of business, I would suggest you don’t go down that path and stick with it. In addition to the organizational structure mentioned above, there are many new problems even if it is changed to the central-Taiwan vertical model. If your reuse is not much, I suggest that you persuade the leader not to do the middle stage.
Author’s brief introduction
Wang Ye Return, TVP, Tencent Cloud Most Value Expert, And Return Is The Architecture Director of Good Buy wealth. Responsible for the development and operation of Haomai middleware and platform, team management and implementation of major technical decisions. 19 years of IT experience, programmer who experienced the bubble of Internet economy in 2000, joined Daicwisdom in 2011 as the test director, led the team to develop the “Daicwisdom Cloud Test Platform” within two years, and gradually transformed the financial data service business from waterfall to DevOps through platformization. Joined Haomai Fortune in 2013, experienced the company’s Internet-oriented business transformation and technological change in the past four years, passed through different business teams, and gained a certain in-depth understanding of technology and business.