This article was originally posted on GitChat by Gu Yu with permission. Please contact the author or GitChat for republication.
preface
It has been four years since I joined ThoughtWorks in 2013. In the past four years, I have participated in the microservices consulting and implementation of a total of seven products and projects as a developer, DevOps engineer, DevOps consultant, microservices architect and microservices consultant. There are success, failure, reflection, more learning and summary. The following is a summary of my experience in micro-service consulting over the years, hoping to bring some help to those who are caught in the dilemma of micro-service implementation.
Difficulty 1: “One-step” cognitive illusion
Microservices have been all the rage these days, but there are few practical examples. Most microservice cases only see the “evolution result” of the microservice architecture, but not the “evolution process”. It’s like everyone can see the peak of an architecture, but no way to climb it.
This creates the illusion that the architecture of microservices was designed in one step by highly competent architects.
This is similar to how many teams feel about top-down architectural design. Architects flocked to the site, and analytic methodologies proliferated, and discussions and sharing proliferated. However, the actual implementation is very few, making micro-service on the network slowly become a “metaphysics” : the implementation of micro-service has been in the stage of “theoretical research”.
This violates the most basic law of software architecture: Architecture evolves by addressing current needs and pain points, and cannot be designed for problems and pain points that do not arise. Therefore, a one-step, holistic microservices architecture design is completely unnecessary. Moreover a centralized design, it is difficult to reflect the lightweight advantages of micro services.
I believe that technology must move in the direction of continuous cost reduction. If the new technology raises costs instead of lowering them, either there is something wrong with the new technology, or it is in the wrong posture or in the wrong direction.
Therefore, there must be a long-term mental preparation for the implementation of micro-services. But once that initial threshold is crossed, the rest of the work can be replicated, and faster and faster.
Difficulty 2: “Architect elitism”
Many products rely heavily on architects, known as “architect elitism” : the belief that only the “technical elite” of the organization, the architects, can complete the architecture of the product, while the rest of the team only needs to implement the architect’s design. This is a common problem with large enterprises and large systems, and stems from a long-standing habit of heavyweight enterprise architectures.
Microservices, on the other hand, are akin to the “agile marginal revolution” : lightweight architectures that can be built by a small team of no more than 2 to 8 people. And for a team of this size, removing the entire microservices team from the product team would have no impact on the overall product development schedule. Therefore, even if the failure does not bring too much loss. However, when the first microservice revolution is successful, the multiplier effect of replicating successful experiences can bring big benefits.
In terms of the risk-return ratio of the investment in architecture transformation, this is very cost-effective.
As a result, there is no need for a microservices team to be big, just two or three people. However, no one has the practical experience of micro-service ah, what if it fails?
This brings us to the next difficulty.
Difficulty 3: Lack of an environment of trust and encouragement of innovation
In the face of unknown territory, failure is inevitable. And in a world of constant uncertainty, success and failure no longer matter: failure today may be success tomorrow, and vice versa.
Success simply means that the outcome is in line with one’s assumptions, while failure simply means that the outcome is not in line with one’s assumptions. But whether we succeed or fail, we can learn and reflect on our actions, and such experiences are the most valuable in R&D activities.
However, in many organizations, especially “elitist” product teams, responsibility and stress tend to break down from the top down. Because of their large organizations, pyramid structures tend to build institutions based on distrust. The system creates a culture of “better to do nothing than to do wrong.” Because the top is responsible for failure, all innovation can only stay at the top, it is difficult to implement and promote. In this case, the long-term cooperation of the organization has formed stable working habits and thinking patterns, and formed a balance of interests, which will make the whole organization “stuck” in the face of innovation.
When microservices are distributed from the top down as a political task, there is a lot of buck-passing within the team to avoid failure. Demonstrate the difficulty of this matter through continuous analysis, discussion and design. It seems to me that if you want to do something, you can find a way to do it instead of imagining a bunch of problems and responsibilities that you don’t have yet. Solving problems on the go is more efficient than designing and discussing.
The organization’s solution to this problem is to bring in a “bad guy” : a newly hired architect or outside consultant, for example, to do the job. You don’t have to take the blame if something goes wrong. This is a compromise that allows things to proceed without risk. However, this is a short-term effect, which cannot solve the innovation dilemma of the organization itself. Relying on external forces to solve the most valuable problems for a long time will not improve the organization, but form dependence on external forces. Not a good thing for Cohesion.
Only by breaking current working habits and thinking patterns, and fully recognizing the difficulties, risks and values of innovation, can we occupy the high point of innovation and attract talents.
Difficulty 4: “Selection difficulty” of microservice technology stack
Because “elitist” architects have a lot of responsibility and pressure. They must carefully select the technology stack for microservices architecture. So there’s a lot of experimentation between different technology stacks. For architects who are used to “designing, working overtime” in large r&d organizations. The “long design, slow feedback” pace seems to be a given.
The rapid growth of the microservices open source community has fostered “architect anxiety” : being despised by peers, by bosses who don’t understand technology, and even by subordinates if they adopt outdated technology. As a result, architects struggle to compare and learn from new technology stacks. In addition, not being familiar with the technology tends to increase the risk, requiring the architect to spend more time researching. Pick and choose the microservices stack with a “one step in place” architectural fantasy, rather than quickly iterating through existing low-cost solutions.
The core of microservice is to adopt the mechanism of “small scale, quick feedback” to reduce the complexity of software system and disperse risks through virtual and automation technology, so as to timely respond to various challenges brought by market changes, carry out rapid sales innovation and obtain market feedback. It’s not just taking advantage of new languages, programming frameworks or tools.
Learning and practice are complementary processes, learning in practice and applying the knowledge learned to practice. It’s not like preparing for a test: stop and study, then buckle down when you’re ready.
These four points can cause large organizations to get stuck in microservices implementations, which often leads to the most important and, IN my view, the core of microservices implementations.
Difficulty 5: Overestimate the technical change of microservices, but underestimate the organizational change brought by microservices
As an architect, never underestimate the power of Conway’s theorem: “Organizations that design systems produce designs and architectures that are equivalent to communication structures between organizations.”
From the perspective of institutional economics, software product itself is a computer program expression of communication system between internal organization (employees) and external organization (users). This system is bound to shorten the organization’s internal and external communication costs in the direction of development.
Therefore, the system architecture must be consistent with the organization architecture, and if it is not, it will impede the evolution of the organization.
This leads to a corollary: if the architecture of an enterprise organization is not unique, then neither is the architectural solution for microservices.
When the architecture and organizational structure are aligned, everything goes smoothly. Otherwise, all kinds of problems will arise.
This relationship is like the relationship between shoes and feet. Only when you wear the right shoes can you walk comfortably. Shoes that are too big or too small will not get you moving faster. Of course, you can choose to buy shoes (import products) that don’t quite fit, but will work, but you just have to stop to try them on. You can also pay a premium for a custom-made one that won’t make you walk faster any time soon, but will only fit better and better.
If everyone can run fast with new shoes but you, then it’s not the shoes, it’s your feet, and it’s not the shoes that can be fixed. You’ve got to get the foot out of the way first, and then the shoes. Of course, you can correct your feet with shoes. It takes some work, but it’s more effective than constantly changing your shoes.
Unfortunately, most organizations are not prepared for the organizational changes that microservices architectures bring. Still, “system architecture issues” and “organizational issues” are split into two separate domains: microservices are technical issues, and organizational issues are management issues.
A competitive organization is one that can achieve the effect of 1+1>2 by combining advantages. Rather than splitting the advantages so that 1+1<=2. Therefore, technical problems and management problems are not two problems, but two sides of the same problem.
Therefore, if your organization is a decentralized, small team structure, don’t worry, your application architecture will evolve in that direction. On the other hand, if you are not a decentralized small team structure, the microservices architecture will not fit the organizational structure.
How to solve these problems?
As a practitioner of micro services, micro services should not be “ye gonglong”, only stay at the level of research. Instead, start fast in an agile and lean way and solve problems as you go. Each organization has a different organizational structure and business structure, and the challenges of microservices implementation are very different. Rapid learning and improvement in the implementation process, there is no need for a long cycle of overall design.
For details on how to solve the five problems mentioned in this article, please refer to 7 Steps to Improve micro Services Implementation Efficiency.
For more exciting content, please pay attention to the wechat public number: Software Utopia
WeChat
Sina Weibo
Evernote
Pocket
Instapaper
Email
LinkedIn
Pinterest
