In this paper, I would like to share with you how Song Xiaocai has evolved from a single-point boulder system to a domain-driven service-oriented design in the past three years. This evolution is still going on. We have encountered many difficulties in practice and gained experience and reflection.
1. Early systems
I was the first programmer to join Song Xiaocai when it was in the preparatory stage and witnessed the whole process of song Xiaocai system from 0 to 1. The tech team consisted of one product manager, one programmer (me), and five outsourcing classmates. I still remember very clearly the first day I worked in Song Xiaocai, which was January 18, 2015. I also remember very clearly our first task, which was to produce a trading platform before March. What do you need for transactions, users, accounts, goods, funds, orders… These modules formed Song Xiaocai’s earliest trading system. There was a lack of overall architectural design, a lack of adequate business understanding, a lack of functional abstraction, and a lack of performance analysis for the sake of speed. These actions now seem unthinkable and unacceptable. I often wonder what would happen if we could go back three years ago to develop the earliest Song Xiaocai system based on our current experience and understanding? Would I start with a user center, a merchandise center, etc.? Would I design stateless systems and scale-out capabilities from the start? Would I have developed a gateway platform to collect and standardize all API calls from the beginning? The answer is certainly not, and the results are not always better than they were in the first place. Early stage systems don’t require some sort of nuclear weapons of architecture and middleware, what they really need is the ability to deliver combat to a company’s business as quickly as possible.
1.2 Problems
At that time, there were two main systems, namely the internal ERP system and the API service system called by the client, which completed the closed loop of our transaction. Over the next two years, frantic business code accumulation and uncontrolled duplication of code made the system increasingly cumbersome. At that time, the system structure was very simple. Common modules were extracted in the form of JAR packages. The dependency relationship of ERP system can be seen in the following figure:
At this time, the system transformation becomes very painful, every release will become extremely nervous, afraid of which module JAR package has not updated, leading to the system can not start smoothly. There was a time when I dreaded the release of the system, and I would be relieved if it went well.Copy the code
Scattered in various parts of the system are some extremely harsh comments, such as "this code is too good, I dare not change, so I copied a copy", "this part of the logic is written by XXX, do not easily change" and so on. The following six points are the main problems with our system:Copy the code- Jar packages are poorly managed, often causing systems to fail to build;
- Clumsy, coupled, confused release impact points, changes in the underlying JAR, resulting in every application needing to publish;
- Difficulty in collaboration during development;
- New people are difficult to get started, difficult to understand;
- Not easy to manage, a lot of dirty code;
- Mounting technical debt; When the company is small and the business complexity and system complexity are still in a relatively acceptable stage, the monolith system is often a relatively efficient architecture design. But as the business grows, complexity and traffic increases, and a critical point is reached, the upgrade of the system architecture and the dismantling of services becomes a necessity.
2.1 Lack of planning
When we decided to split the service, the size of the technical team was not very large, and none of the students had much experience in service splitting. In retrospect, that was a hasty start. We didn’t do a lot of planning, we just went straight to the “just do it” part. It can be imagined that the service will appear how messy, for a while a “price center”, for a “authority center”, which is simple to open which, which new business to go up, a service to say. Subsequently, problems such as insufficient server resources, difficult operation and maintenance, and chaotic invocation between services were exposed. It wasn’t the height of illness when people realized it. We immediately carried out the song xiaocai overall service of the design and division, the service is divided into two layers. The top layer is the business layer, the bottom layer is the capability layer, and the calls between services should be as tree-like as possible, not as circular.
Service unbundling is very easy, and the key is granularity. At the time, there was an almost paranoid understanding of microservices that as long as a functional module stood alone, it had to stand alone as a service, and two nodes were needed to ensure availability. We’ve put this theory into practice, and there have been quite a few extreme moves. It became clear that there were problems with this approach, and that to solve this problem we had to agree on the philosophy of service unbundling. We try to use DDD (domain-driven design) to carry out service planning. Based on DDD design method, we start from the current business, find out the core capabilities of Song Xiaocai business by abstracting the business, and split it as a service. But DDD is a methodology, not a dogma and a golden rule. Don’t go overboard when using DDD. After using DDD analysis, we found a more headache problem, that is, the previous services were too detailed, many services that should not be separated into two or more. Therefore, we launched the “Ark Project” internally. The main objectives of the project are as follows:
- Build a common understanding of how services are broken down and promote DDD use in the team;
- Define microservice engineering structures and specifications;
- Consolidation of previously fine-grained services; Through this trip to the pit, we also understand that the separation of services is a very careful thing, separation is cool for a while, if you want to merge and unify, it is not so easy a thing.
2.3.1 experience
Whether the development team has enough experience to navigate the microservices technology stack is probably the first consideration. It is not necessary for the team to have complete experience in initiating a service split; it would be nice to have experts on the team. If not, there may be a need for thorough technical demonstration and rehearsals, or at least not unprepared battles. Also mentioned above, we start a bit hasty, the team is not very professional students in this aspect, so on some of the distributed common problems, on the pits, such as call retry, timeout mechanism, a distributed transaction, etc., these problems appeared, many inexperienced homecoming is very crazy, even do not know how to start.
2.3.2 Stability test
The separation of services, the inevitable problem is to increase the students’ development of self-testing difficulty. In previous systems, all calls were deterministic, whether they were launched locally or published in a test environment. However, once a service is split off, it may face many problems:
- Local calls to remote services may be prohibited, so use mock methods to solve this problem. In the native unit test, the main test is the business logic and process, and the return data of the remote call is not perfectly measured.
- The test environment for the invocation can be very unstable, and the service may be dead by the time you want to invoke it.
- When multiple project teams develop new functions, they invoke the same service, which may lead to resource competition due to limited server resources. Different companies have different testing environments for their students. But one thing is certain, if the test is complicated, it will cause a lot of pressure to the development students. It took him an hour to complete a complete test of a simple function that might take only 5 minutes to change. If the solution cannot be solved well after the service separation, it will lead to more and more fluke mentality and laziness of developers, who submit the code or release it without testing.
2.3.3 log
The unbundling of services inevitably leads to the scattering of logs everywhere, making it much more complex to view logs to locate problems or to implement events based on logs. At the beginning, we were not aware of this problem, so many developers do not know how to locate bugs or faults. Because the invocation of a method may be magnified several or even a dozen times due to servitization invocation, it is a problem where the final error occurs and how to find it.
Third, concluding remarks
About how to set up efficient raw B2B platform, because of the large contents, also is very complex, unable to give you an article, this article is just beginning, the following will be divided into several articles from industry present situation, the situation of business and product overview, technical team building, the server-side technology platform, the front-end development, such as multiple dimensions to tell, We will be more than three years in the B2B field of the precipitation of core products and technology platform to open, hope that more people in the industry can have a deeper understanding, less detdetments, hope to help you, the distribution of this series of articles is as follows (will continue to update) :
1, “How to build an efficient FRESH B2B platform (B2B technology sharing the first article)”
2, “Song Xiaocai how to enter the fresh B2B market (B2B Technology sharing second)”
3, “fresh B2B platform product system iteration (B2B technology sharing third chapter)”
4, “fresh B2B how to build an efficient technical team (B2B technology sharing fourth chapter)”
5, “how to build fresh B2B technology system from 0 to 1 (B2B technology sharing fifth chapter)”
6, song Xiaocai technology how to deal with the rapid change of fresh B2B business (B2B technology sharing sixth chapter)
7, “fresh B2B technology platform front-end team how to build (B2B technology sharing seventh)”
8, Song Xiaocai about “Ability” design and Thinking (B2B Technology Sharing chapter 8)
9. Design and Thinking of Service Separation (B2B Technology Sharing chapter 9)