This year’s programmer festival is exactly the time when I have been working in Ali for 3 years. I take this opportunity to take stock of my work in the past 3 years and check my future development. Personal vision and thinking are always limited, especially in the field of research and technology. The more you know, the more you actually know how ignorant you are, and the more you are in awe of the unknown and excited by the vastness of the unknown.

I was born in 1976. I am a dragon. I am 41 years old, so I can be regarded as an old programmer. I really liked writing code, and I got my senior Programmer certificate in my junior year in ’96, so it was something to brag about.

My greatest pleasure since I graduated from PhD is learning and thinking deeply. Therefore, it is never confused by the simplicity or complexity of the project or business in the process of work. I have always had a clear guideline for my own development: take it one step at a time, improve my ability to solve problems, and set no limits on myself. I really thought about it about 10 years ago when I interviewed a 40-year-old guy, and I came to the conclusion: I love writing code, and I’m going to do it for the rest of my life.

Since the project is so important, let’s do it

Back to the topic, summarize the last 3 years of work in Ali. In the first two years, I mainly worked as the captain of the imperial Kitchen data engine team, talking about two key projects I experienced.

The first is 5K +, a universal big data platform. Just a month to catch up with this collection of Beijing and Hangzhou two big project, really quite lucky. I was impressed by several aspects of this project. One was that I participated in the discussion of the plan when the project was approved. Because it involved the communication between Beijing and Hangzhou, cross-department and cross-team, the eldest members of each team would inevitably fall in love and kill each other. In the early hours of the morning, during the free speech phase, I couldn’t resist jumping out and saying, since this project is so important, I think we’ll do it. It is really relying on their own accumulated in the startup company to jump out and say this sentence. I think it’s all about spelling.

As SOON as I said that, I realized that this could be causing my boss a huge problem. Fortunately, my boss has long been tired of pulling, stand up to take responsibility, for a time each boss out of the gun, respectively, a time of public excitement. The final assignment of responsibility was done in 20 minutes, and even an old Ali cried.

Thanks for that night, and thanks to Ali for giving me a great group of teammates and bosses. I will always remember that in the following 996 months, I was in charge of the project management of two small modules during the exchange business trip between Beijing and Hangzhou. I gave full play to my ability to solve problems quickly, and I would plug holes where there were holes. Of course, the whole team was very strong and hard-working. At the end of the project, I won the best firefighter award, which I really like.

Although I was also one of the Pmos of the project, I spent more time watching and learning from Ali’s project management, organization and coordination than just getting some soy sauce. This project was really difficult. We went through various compromises during the process. After the completion of the project, we paid off the technical debt on and off for a year. Later, I communicated with some classmates from other companies about the construction of data platform, and found that we had really gone far. Because it’s about my own feelings, I don’t praise other students, otherwise I could write a book.

Two, this is not my work alone, just trying to make it perfect

The second project also has a strong character of my own. We have been pressed very hard by business. But for the engine layer research and development, the team members also have their own demands, and the system should be gradually improved and improved. After the completion of the 5K+ project, an important module we relied on began to have frequent problems. With the deepening of the communication between teams, we found that the instability of the other team and the uncertain development direction led to a very high risk of this module in the future.

Our first thought was to deploy a new set as a transition. In the transition stage, in order to complete the business and do this at the same time, I took over part of the business work of two students in the team, freeing up manpower to do this work. Two students went to Hangzhou for a one-month business trip and basically took over the platform, which ensured the smooth operation of our business.

Subsequently, after investigation, we resolutely abandoned the original implementation of this module, and mobilized the technical force of the team to redesign and design a new module, in addition to replacement, more importantly, for development. After taking this step, I found that many things were alive in the future. Data services began to emerge from the whole platform as an important point, resulting in deeper interaction with the data team. Then, with the instability of the original data services, a new data service platform was spawned.

It’s not my job. I just try to make it perfect. At the beginning, I was very confused, and every step was shocking. Now I believe that every student who participated in this project has a good memory in their heart. Moreover, we have not only achieved ourselves through this project, but also achieved our brothers’ team and the development of the royal Kitchen. That big.

Technical challenges are prey, opportunities and achievements

During these three years, I don’t think I encountered great technical challenges. I thought of many things in advance and organized technical experts to discuss them in advance. When I was with the team, technical challenges were prey, opportunities and achievements.

Take a simple example to illustrate the process of my project. For example, we want to implement a stream-limited service, which is to allow a tenant the maximum QPS. First of all, we need to define the boundary of the problem, including whether to consider the network layer attacks (which may be handled by other modules), the scale of the business in the future, the stability of the system, the scalability of the architecture, etc.

After these problems are determined, a series of technical solutions will become the technology selection. So how to judge which technical solution to adopt, which is not the latest, coolest and best at that time, should consider the future deployment environment, upstream and downstream environment. Most importantly, this is a distributed requirement. To put it simply, N servers jointly maintain a QPS value. The distributed theory behind this, in plain English, is CAP. In the case that CAP three cannot be met at the same time, the goal should be reduced to ensure the business.

To this end, we refer to the distributed BASE model to reduce the requirement of consistency, and adopt the idea of combining partition and principal quota pool to solve the flow control of large tenants, while for long-tail tenants, another set of control is adopted to ensure accurate flow limiting.

At the same time, considering the emergency plan in the failure or degradation of third-party modules and non-critical modules, as well as the failure of some modules or machine room power failure caused by service unavailable, and some single point of service problems, a complete stability plan is designed. Of course, the last thing to consider is the scalability solution, if the demand scale suddenly becomes large, but the whole has a boundary.

To sum up, the entire project should have clear goals and phases and corresponding key indicators that are observable, evaluable, scalable, recoverable, and easily delivered to others for further development and maintenance.

I don’t think I’ve done it very well, but it’s really fun when you’re working on a system.

I don’t think it is necessary to change management at 35

Now that I think about it, there were very few people who gave me technical advice when I was growing up. More often than not, I preferred to learn from anyone I met. My purpose of learning was not to surpass others, but to surpass myself.

After I transferred to the iDST team, I sat next to the boss of iDST and benefited a lot from the work of the R&D and management experts. I have always felt that no one can teach programmers, they need their own research and attentively learning. In the cooperation and communication between the students of my former imperial Kitchen team and the students of iDST now, I can always find the advantages and bright points that make me very impressed. How can I not be happy about that?

In addition, I think it is very important to keep thinking and to communicate with other students in an open manner. I think my biggest technical advantage may still be in the engineering field, mainly in the server back-end research and development and data platform construction, mainly because of my thinking and experience. I will continue to enhance my ability in both theory and practice. At the same time, I’ve been building up my knowledge of ARTIFICIAL intelligence. Thanks to my deep thinking in the direction of information retrieval during my doctoral thesis, I discovered this fascinating field early on.

Here are a few words about the field of ARTIFICIAL intelligence, although some people say that it is also called tuning, feature engineering, training deep networks, etc. These are really layman’s words. What is really needed is rational thinking. To solve such a problem without a clear path, we need to think deeply and try hard. After numerous failures, we may have a small harvest, and then we need to explain this small harvest in the most accurate mathematical language to pave the way for subsequent research and development.

At present, I have just finished reading statistical Natural Language Processing, an overview of THE NLP field. Now I am rebuilding my knowledge system of probability theory and statistics, trying to acquire basic concepts at hand, and then looking for a small field for in-depth thinking and attempt. For me, there is little temptation to find a point to build a model and modify parameters to produce a paper. I hope to study the previous research results and make a breakthrough in theoretical depth.

This is an area where programmers who switch to management at 35 May not get it. When I meet a thing that I can continuously devote my energy to, and the more I study, the more difficult it becomes, for me, how lucky it is. What’s more, I don’t think it’s necessary to change management at 35. Management is not easy to do. Many people think that management is to assign work. Technicians are arrogant and arrogant. And real management, like writing code, is also a kind of knowledge, a combination of theory and practice, which requires both exploration and practice. When people ask you to lead, they entrust their own development to your hands, so it is a heavier responsibility.

When we chant the spring breeze but not the yumen Pass, yumen Pass is actually this small earth pit. So think critically about the popular mantra that you have to change management before you turn 35.

I mainly use the time on the way to work to do machine learning, about 3 hours a day, working time is mainly writing business code, I love writing code, tuning a function of the feeling is really cool!

For example, STATISTICAL Natural Language Processing, which I read on the way to work for almost a year, I was fascinated by the machine translation model invented by five IBM engineers. The process of inventing these models, the process of thinking about them, is what I learn.

Finally, I want to say that in Ali, I have experienced prosperity, confusion, loss and even neglect. The most important thing is to keep my technical heart and encourage everyone together.

The original article was published as: 2017-11-22

The author: Ink Jue