Practice of multi-service modeling in Meituan search sorting

The search on the homepage of Meituan is the largest traffic distribution portal for various life services on meituan App, providing various services for tens of millions of users every day. Search sorting is a typical multi-service mixed sorting modeling problem, which has many challenges. This paper focuses on the modeling and optimization of multi-business sequencing in multi-business scenarios of stores and businesses, hoping to be helpful to students engaged in related work.

The introduction

The mission of Meituan is to “help people eat better and live better”. The business of Meituan App includes delivery, in-store catering, shopping, optimization, hotel, tourism, leisure and entertainment and other life services. The search on the homepage of Meituan is the largest traffic distribution portal for various life services on meituan App, providing various services for tens of millions of users every day. Meituan search sorting is a typical multi-service mixed sorting modeling problem. A typical multi-service search scenario is when the user searches for a location such as wangjing, but the user’s demand is not very clear. At this time, the search result page is shown in Figure 1 below. The list of businesses below will contain the results of various businesses such as catering, film, leisure and entertainment, and hotel near Wangjing, which is a mixed sorting scene of multiple businesses.

The multi-service scenario presents the following challenges:

As there are commonalities and features between services, how to make the model take into account these two features to achieve better data learning? For example, in-store catering is very sensitive to distance features, while tourist attractions are relatively insensitive to distance features.
Businesses naturally have high and low frequency characteristics (such as takeout and tourism), leading to an imbalance in the number of multi-business samples in the model’s training data.
Each business often has its own different main goals, how to meet the goals of different businesses, ultimately can improve the user experience of search.

This paper shares the modeling and optimization work of multi-business sorting in Meituan search. We mainly focus on the multi-business scenario of store-to-store merchants. The subsequent content will be divided into the following four parts: The first part is a brief introduction to the hierarchical architecture of Meituan search sorting; The second part will introduce multi-service fusion modeling on multi-channel fusion layer. The third part will introduce the multi-service sequencing modeling of the precision scheduling model. The last part is summary and prospect. Hope to be engaged in the relevant work of the students to inspire or help.

Sorting Process overview

The process of Meituan search system is shown in Figure 2 below. The overall process is divided into data layer, recall layer, sorting layer and display layer. The sorting layer is divided into the following sub-parts:

Rough ranking layer: a relatively simple model is used to perform preliminary filtering on recall candidate set to achieve trade-off of ranking effect and performance.
Multi-channel fusion layer: the quota model is constructed by using the features of query words and context scenarios to control the number of different business candidate sets and achieve accurate understanding of user needs.
Precision ranking: The deep learning model with billion-level features is used to capture all kinds of explicit and implicit signals to achieve accurate estimation of Item ranking scores.
Rearrangement layer: The use of small models and various mechanisms to reorder the results of fine arrangement, to achieve fine direction optimization.
Heterogeneous sorting layer: Deep learning model is used to sort heterogeneous clusters to achieve high load bearing of multiple services.

The multi-tier sorting architecture is designed to balance the sorting effect and performance. The subsequent multi-service modeling optimization work mentioned in this paper is mainly introduced from the multi-channel fusion layer and the fine scheduling layer.

Multi-business modeling practices

Multi-service Quota model (Multi-channel fusion layer)

With the development of Meituan business, Meituan search has access to catering, comprehensive, hotel, tourism and other businesses. For search terms with vague service intentions, such as Wudaokou, you need to judge the service intentions of users based on multiple factors, such as users, query terms, and scenarios. In order to integrate recall results of different businesses and refine a suitable candidate set for L2, we designed a multi-business quota model to balance the proportion of multi-business recall. This method of combining multi-way recall results based on quota is very common in search and recommendation scenarios, such as Taobao home page search and Meituan recommendation.

In order to provide flexible access to multi-way recall and adapt to the development of Meituan search business, we constantly iterate the search quota model. The iterative process of meituan’s multi-business Quota Model is described in detail below, and the multi-Business Quota Model is referred to simply as MQM in the rest of the article.

One-dimensional target multi-service quota

Considering that there are multiple recalls of different businesses in the big search results, in order to describe the strength of the intention of users’ search Query to recall the three-way businesses, we adopt the multi-objective modeling method to model whether each recall is clicked or ordered, and realize the multi-business quota initial version model MQM-V1. The model outputs the joint probability of click and order of each recall route as the final quota distribution. At the feature level, we use Query dimension feature, Context dimension feature, Cross dimension feature and User dimension feature to describe real-time personalized needs of users in different scenes. The mqM-V1 model structure is shown in Figure 4 below.

After the launch of MQM-V1, the overall online click rate is +0.53%, and the purchase rate of all businesses is basically flat.

Two-dimensional target multi-service quota

With the continuous iteration of Dasso recall strategy, dasso not only introduced the recall method divided by business, but also introduced heterogeneous recall methods across multiple businesses, such as vector retrieval and geographical proximity retrieval. As a result, dasso recall strategy kept increasing, and multi-business quota model also faced the problem of cold start caused by new recall sources. Meanwhile, in order to strengthen the individuation of multi-service quota model, we refer to the modeling method of user behavior sequence in [6]. In summary, the differences between mqM-V2 and MQM-V1 of this version are as follows:

Modeling targets are upgraded from one-dimensional targets clicked in the way of recall to two-dimensional targets of cross-product business in the way of recall, so that the granularity of multipath fusion is finer and the accuracy is higher.
The behavior sequence modeling module introduces Transformer Layer.
In order to solve the problem of cold start of new recall source access, we introduce the manual experience layer, including business prior and historical statistics, and determine the recall quota of each way by comprehensive model output.

After the launch of MQM-V2 version, the rates of all business indicators have been improved, among which the rates of tourism and restaurant visits are +2%, +0.57%, and the rates of comprehensive and hotel visits are flat.

Multi-service Ranking model (Fine Ranking Layer)

From meituan search refined model to DNN model, until the end of 2019, Meituan search refined model structure is the industry’s mainstream paradigm structure of Embedding&MLP, During this period, we also tried model structures proposed by the industry, such as PNN[1], DeepFM[2], DCN[3], AutoInt[4], FiBiNet[5] and so on.

As the iteration went on, we found that the optimization for specific business could not play a role in the refined model. In order to take into account the characteristics of each business and support more effective targeted iterative optimization of each business, we need to explore a model structure suitable for multi-business scenarios such as Meituan Search. The development history of precision model in multi-business modeling will be introduced in detail below. Multi-business Network will be referred to as MBN in the following part of the paper.

Split independent sub-network

Considering that hotels and tourism account for a small proportion of the traffic in meituan-based large search ranking strategy, and that relevant optimization for small traffic is difficult to be reflected in the current unified Embedding&MLP model structure, we try the manual customized multi-tower model MBN-V1 structure as shown in Figure 6: the main network reuses the current model structure. For specific information, refer to the behavioral sequence modeling part in [6] to add independent sub-networks for hotel and tourism; Subnet input including hotel ratings of the unique characteristics and the main network output, tourism subnet input includes the unique characteristics of tourism, the main network of the last layer of FC rated output, the main network, hotel and tourism tower input different is because the business logic to the data distribution difference is big, this is the result of practice, the final output is a weighted sum of three output.

For the weight part of the weighted sum, we adopt two ways to set the weight:

In the first way, hard segmentation is adopted, that is, the weight vector is a one-hot sparse vector: the hotel merchants are predicted, and only the outputs of the hotel sub-network are selected, and the rest are carried out by analogy.
Second, the output of the multi-service quota model is used as the weight value.

Online experiments show that the second method is better than the first one. We believe that hard segmentation will result in the parameters of subtower branches can only be updated by the data of corresponding businesses, and the uneven data proportion of each business will lead to poor learning, while soft segmentation will achieve a function of knowledge transfer. In the end, compared with the unified Embedding& MLP model, the overall tourism has a positive effect: the overall click rate is +0.17%, and the effect of the other businesses’ visit rate is basically flat.

Sub-network weight self-learning

Based on the initial positive effect of the first version of multi-service refinement model, we continue to add the food business sub-tower. Meanwhile, considering that MBN-V1 relies on the output of quota model, the change of quota model may have an impact on the effect of refinement model. In view of these factors, we launched the second version of multi-service model MBN-v2. The model structure is shown in Figure 7. Compared with MBN-V1, the differences are as follows:

Add a separate sub-network for the gourmet business.
Decoupled the refined model and the quota model, integrated the weight generation sub-network in the refined model, the input of the sub-network is mainly some Query dimensions, Context dimensions characteristics.

Online experimental results: Compared with MBN-V1, the click-through rate of MBN-V2 is +0.1%, and the effect of business purchase rate is basically the same.

The sub-network features are adaptive

On the basis of the second version of the model, we further added the integrated business subtower. As the number of subnetworks increases, the input of the subnetwork is manually designed at present, which requires a lot of time to conduct offline experiments. Considering that the current multi-service subtower structure is very similar to the multi-task learning structure in the industry, we try to introduce the multi-task learning structure in the industry. At the same time, we analyzed the weight baryon network output in MBN-V2 and found that the weight of its output was similar to the output of different business merchants, so the targeted optimization of business was not obvious. Based on the above part, we iterated the third edition of multi-service refinement MBN-V3, with the structure as shown in FIG. 8. The improvement points are as follows:

MMoE[7] multi-task learning structure is used to automatically learn the output of feature representation to the upper layer sub-network, so as to replace the input of manually designed sub-network.
In addition to the main LambdaLoss calculated by users’ online feedback, the Loss function of the fine model added the classification cross entropy Loss of the business, so as to achieve the purpose of maximizing the weight of the corresponding business subtower when predicting the score of a business Item.

Online experimental results: compared with MBN-V2, THE overall click-through rate of MMBN-V3 is flat, the visiting rate of gourmet business +0.36%, the visiting rate of comprehensive business +1.07%, the visiting rate of hotel business +0.27%, the visiting rate of tourism business +0.35%.

Optimization of multi-service feature expression

Although MMoE multi-task learning structure has been applied in many scenarios in the industry, and has been effectively verified in our multi-business modeling scenarios, we continue to follow the forefront of the industry and implement it in combination with business scenarios.

We tried the PLE[8] structure proposed by Tencent and iterated out mBN-V4 with multiple businesses. PLE can be regarded as an improved version of MMoE. It has its own specific expert layer for each task, and there is a shared expert layer between different tasks. Compared with MMoE, which is the weighted sum of all expert outputs, THE input of PLE subtask is the weighted sum of the unique experts and the shared expert outputs of the subtask, which makes it easier to learn the characteristics of the business. At the same time, considering performance, we selected single-layer PLE, also known as CGC structure, as shown in Figure 9 below:

Online experimental results: Compared with MBN-V3, THE overall click-through rate of MBN-V4 was +0.1%, the visiting rate of gourmet business was +0.53%, and the visiting rate of other businesses fluctuated flat. We visualized the expert weight of MMoE and CGC as shown in Figure 10 below. The analysis found that compared with MMoE, the expert weight variance of CGC structure was smaller and more stable among multiple samples of the same business, indicating that CGC had more advantages in feature representation compared with MMoE.

Summary and Outlook

Since the end of 2019, in order to solve the actual multi-service recall sequencing problem, Meituan search has carried out a lot of exploration, from engineering to algorithm to product form to enrich the multi-service support. The sorting algorithm is mainly optimized at the multipath recall and fusion layer and the precision layer.

The multi-channel fusion layer mainly completes the screening process of search results from result correlation to result quality, and needs to solve the fusion truncation problem of different recall methods (text recall, recommendation recall, vector recall) and recall results of different businesses, which directly determines the result candidate set that users can browse. Among them, the most important problem is to judge the strength of user demand for each business and recall quality of each business, and to determine the appropriate fine discharge access standards for each business result and recall result.

The multi-service quota model gives the proportion of each recall and each service that should be refined by integrating the real-time demand of users, the historical statistics of Query, the search context information and the quality of each recall source. The model ensures the diversity and high quality of the candidate set in different scenarios, achieves less intrusive access of new business and new recall methods, and reduces the cost of access of business and recall. At the same time, it also provides a priori weight to fuse the results of each service sub-network for the network structure of fine hierarchical service classification.

On the basis of the multi-channel fusion layer, the precision ranking layer further conducts fine ranking modeling and scoring for multi-service search results. The needs of users are as diverse as those of Meituan business. In order to fully model the needs in various scenarios, the refined multi-business ranking model has carried out several iterations from the underlying data (enriching the characteristics of sub-businesses), model structure and business target fusion. The model structure and the corresponding target fusion directly for the various size of business, scene and the corresponding business target fragment modeling, effectively alleviate the small business small scene in the unified modeling of large business sample submerged. At the same time, the model supports fast iteration of old and new businesses, and each business can easily iterate the characteristics, model structure and corresponding goals independently.

The above optimization covers all online traffic and has significantly improved search user experience and business value, but there is still a lot of work to continue to optimize.

Business-specific feature utilization: At present, we add business-specific features for some services, and give default values for these missing unique features for other services. However, this will bring a lot of redundant computing, which has room for optimization in terms of effect and performance.
Imbalanced sample Learning: the amount of data of different businesses varies greatly in meituan search. How to make the model better learn the distribution of small businesses, we are exploring methods such as transfer Learning and meta-learning.
Multi-objective optimization: Meituan search should not only take account of users’ search experience, but also serve the strategic objectives of each business of Meituan. Therefore, the main optimization indicators of each business may not be the same. Multi-objective optimization is also a direction of continuous exploration.

The work described in this paper focuses on the search ranking of multi-business merchants in Meituan. At the same time, with the development of commodity businesses such as preferential selection, buying vegetables, group of good goods, flash purchase and so on, we are also carrying out the work of multi-business mixed arrangement of commodity and heterogeneous multi-business mixed arrangement of merchant goods.

The resources

[1] Product-based neural networks for user response prediction
[2] DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
[3] Deep & Cross Network for Ad Click Predictions
[4] AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
[5] FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction
[6] Transformer’s practice in meituan search sorting
[7] Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
[8] Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations

Author’s brief introduction

Pei Hao, Xiao Yao, Xiao Jiang, Jia Qi, Chen Sheng, Yun Sen, Yong Chao, Li Qian, etc., are all from meituan platform search and NLP department.

Read more technical articles from meituan’s technical team

| in the public bar menu dialog reply goodies for [2020], [2019] special purchases, goodies for [2018], [2017] special purchases such as keywords, to view Meituan technology team calendar year essay collection.

| this paper Meituan produced by the technical team, the copyright ownership Meituan. You are welcome to reprint or use the content of this article for non-commercial purposes such as sharing and communication. Please mark “Content reprinted from Meituan Technical team”. This article shall not be reproduced or used commercially without permission. For any commercial activity, please send an email to [email protected] for authorization.