We may be surprised to see the title, this is not just the National Day, how began to write double 11 articles. Indeed, although the National Day is not over yet, this year’s Taobao Double 11 activity has already begun.
My last article was published on September 12. The main reason why I haven’t updated it until now is that I have been devoted to the research and development of the Double 11 project for half a month. The last half month is summed up as “tired”, not only physically tired, but also mentally tired.
In a short period of half a month, DOZENS of speechless requirements, endless meetings and endless code were crushing me crazily in the first half of the month. I did not know how many nights IT was the next day when I went home and woke up to repeat yesterday’s behavior, which always made me doubt the meaning of working here.
Fortunately, at 0 o ‘clock on October 1, the activity was on the line smoothly. Although there are still many functions that will be launched in succession, the continuous operation for half a month is finally a sign of suspension of pace, and I can finally take advantage of this National Day holiday to simply share my first Double Eleven on Taobao.
Double Eleven
November 11, 2009, was taobao’s first Double 11. Since then, November 11 has become a fixed holiday on Taobao every year (of course, now all kinds of platforms will gather the popularity of double 11). At that time, I was interested in some electronic products. At that time, there were great discounts for singles’ Day. However, this may be caused by my shopping habits. But the next day I always look at publicly disclosed data, such as trading peaks, QPS, etc., after all, they are professional. Last year, due to personal reasons, I did not participate in the Double 11 and left to go back to school. This July, AFTER I officially joined Taobao, I ushered in my first Double 11.
The activity of my department this year is called “Double Eleven Grass Machine”. Now this activity has been online, and you can experience it by scanning the above TWO-DIMENSIONAL code on mobile taobao. The pain point they need to solve is “they want to participate in singles’ Day, but they don’t know what to buy”. Seeing this, some people may say, why do so many bells and whistles? Just like PDD, won’t the subsidy of 10 billion yuan be directly reduced?
Well, at this point, I can only say that I’m a coder who does what the boss says. As for why not directly reduce like PDD 👇 as shown in the picture below
The function development
I am in charge of the function is the intervention of grass machine activity module, of things to do in fact not very complex, at different times operating personnel will be according to the result of the current situation of grass to recommend/do some intervention, followed by current affairs such as increasing exposure of some high quality content, or block a inferior star and its related content, sensitive word filtration, etc., general process as shown in the figure below.
The operator can fill in some configuration on one of the intervention back-end platforms. The front end students are responsible for building the background page. The server students need to store the configuration data filled in by the operator and then take out these data when the front end performs the contract. At first glance, it doesn’t seem difficult, just a simple CRUD function. Is Ali’s Singles’ Day that simple?
Background data falls into the database
In order to make the whole intervention module form a closed loop, we need to consider the implementation of operational intervention from two perspectives of “background data warehousing” and “foreground data fulfillment”.
For the background data database, it is indeed the CRUD of configuration data, but the difficulty lies in the abstraction of configuration data. The configuration requirements of operation are ever-changing, and it has become a difficult problem to realize it. If each type of configuration is used as a table, there will be a “data table explosion” in the future of the database, and the configuration and configuration are not independent, there may be some correlation, so the design of the data table must be able to achieve the universality and easy to expand for this kind of demand.
In fact, WHEN I originally wrote here, I prepared a simplified data model diagram, but considering the problem of data security, here is a simple description in words, the next part of the performance of the front desk is the same.
It should be noted that the core design philosophy for intervention configuration is “Everything is configuration”. In the design, we divided an operational configuration into two types of attributes: base attributes and form attributes. The basic properties are well understood, such as ID, name, operator, who to put on, when to put on, and so on, which are taken into account in every configuration. Form attributes are strange attributes presented by the operation background that need to be filled in by the operation, such as blocking the blacklist words in the configuration, and specifying the profile picture of the search term.
For basic attributes of operation configurations, data is stored in a tree structure. ParentId is used to specify the parent-child relationship between configurations, Group is used to specify configuration categories, and a configuration is specified by a unique primary key. Using a tree structure for storage will naturally support the business requirements of such an operational configuration and will be more natural to understand.
This area is the most variable and least common for form attributes in operational configurations, drawing on the schema-free design philosophy. Schema-free is considered because this type of form data does not need to be indexed and is so variable that it is almost impossible to describe it through an abstraction. Of course, there are pros and cons to this approach. The pros are extensibility, but the cons are high cost of change if indexing is required. So during development, I made the following jokes:
To do top level design without actual business needs is a loft in the sky, not practical.
Private thought do architecture design purpose is to change numerous for brief, while “abstract” can solve most of the coupling problem, but if on the premise of not clear design goal, in order to abstract and abstract, in order to design and design, it will make a framework originally is not complicated to introduce more puzzling concept which is complicated.
Secondly, any architectural design has its limitations, and different business/technical requirements at different stages will naturally lead to different design priorities. Therefore, for a system, there is no best architectural design, only the most suitable architectural design, it is unrealistic to try to solve all problems through one design.
Front desk data performance
The performance of the front desk data is one of the most frustrating things for me these days. Most of the weird requests come from this section, where we receive requests from users, read the configuration information from the repository, and intervene. The process is easy in theory, but there are a few things worth thinking about.
concurrency
No matter how insignificant the code is, once it is magnified to tens of millions or even hundreds of millions of people to execute it, it may be overturned at any time. In Ali development, concurrency is a problem that needs to be taken into account at any time. If you simply read configuration information from the database, the current QPS will crash the database at any time, so caching is necessary, but cache avalanches, cache breakdowns, data consistency problems can also occur.
The real time
Whether the intervention configuration can take effect in real time is a very important point, because it is related to the operation staff to grasp the rhythm of the Double Eleven activity. Considering the real-time performance of intervention configuration, I divide intervention into server-side intervention and algorithm engine intervention.
For example, if we need to block the search results for a term, we need to intervene directly before calling the algorithm engine results. The advantage of this is that we can reduce a call to the algorithm resource, and the effect is real-time. But if we need to block a piece of content below a search result (such as a sexually suggestive video), should that be done on a server or an algorithmic engine? Theoretically, both can work, but the intervention effects are different. The comparison is as follows:
- Server intervention, timely shielding, but after the algorithm return results are shielded, the foreground display will lack data (for example, only 9 data should be returned after 10 data intervention)
- Algorithm engine intervention, there is a risk of timeout, intervention effect has a delay, but the return result will not lack of data.
So, back to the previous section, there is no best architectural design, only the best architectural design.
Actually in actual development process, and involves a lot of problems, such as the algorithm of engine timeout, the service side of intervention and algorithm engine intervention such as strong coupling problem, in order to solve these problems, we introduced the “database”, but then again idempotent, problems such as system availability, here also is not convenient to make a detailed explanation.
summary
Column as a technology, this paper took a lot of space in the double tenth one some of my development work, but in fact this part of people think is not very important, because this one more or coding of live, if simply coding is a very happy thing, found the problem, think and solve problems these won’t make the heart tired. It was the atmosphere that really hurt, but I can’t talk much about it.
Overall, through this time put a activity, I learned a very, very much, also exposed many problems, now during the National Day long vacation, although as a first shot of the grass machine double tenth activity has been launched, but there are many functions without release, real double tenth also are yet to come, wait until all the end of the activity, A complete replay of my first Singles’ Day from a non-technical perspective.