Welcome to Tencent cloud technology community, get more Tencent mass technology practice dry goods oh ~

Author: Su Bolan

Deep learning should be the hottest word in computing circles for a couple of years now. Based on deep learning, engineers have made exciting progress in imaging, speech, NLP and more. Deep learning itself is also being explored and developed, and the limits of its potential have not yet been seen.



Of course, deep learning is not a panacea. For example, features of many problems are easy to be extracted. We can directly use SVM and decision tree algorithm to obtain good results. Deep learning doesn’t help much. The other problem is that we don’t have enough data, and it’s hard to use deep learning algorithms to get usable models. In addition, some problems are demanding in computing resources and time, and they are not the preferred approach when deep learning miniaturization has not made a breakthrough.



The right posture to determine if a project is suitable for deep learning.

On the other hand, although deep learning has not made a great breakthrough in the fields of personalized recommendation and computational advertising at present, I believe that recommendation system is highly likely to be one of the most important application scenarios of deep learning. The reasons are as follows:

1. Nowadays, recommendation systems have to face massive data and extract features of tens of thousands or even billions of dimensions. Deep learning itself is a very good framework for representation learning. It is good at learning feature combinations that human beings cannot extract from massive data.



(Image credit: The Deep Learning Cheat Guide)

2. Data and features determine the upper limit of machine learning, and models and algorithms only approximate this upper limit. The existing recommendation systems depend on the effect of feature engineering. Feature engineering is built on a continuous deep understanding of the problem and access to additional data sources. However, the total category of features that can be abstracted by data people is limited, so it will be more and more difficult to obtain new data sources and features. With the development of artificial feature engineering, the input of manpower and time is getting longer and longer, but the new features are getting less and less improvement to the system. At this time, using deep learning to do feature expression may be a better choice in terms of cost.



Figure: The accuracy improvement curve of IBM Watson system. It can be seen that the results improved very quickly at the beginning, and the next little improvement required great efforts.

Therefore, we tried DNN in a certain business based on Tensorflow, ran through the whole process, and accumulated some experience. It is also more than 20% improvement over the original LR model online. I hope these codes can also help students to quickly achieve a set of in-depth model framework in their own business.



Figure: DNN in the general framework of CTR estimation

In general, we can choose a relatively simple framework to run through the process and gradually increase the complexity of the model. In general, we can use the framework shown below to divide the data we use into two categories: continuous and discrete features.

For continuous features, we need to do some normalization; For discrete features, a Embedding is generally done to transform a discrete feature into an N-dimensional vector. The length of this vector is generally proportional to the value space of this vector. The embedding process can be realized by FM. In our code, this is done via the Tensorflow automatic embedding_column.

Therefore, the user data and the recommended article data are divided into two categories together, and then the discrete features and continuous features after embedded are combined together as the input of neural network, and the output is [0,1] whether to click. Call the DNNClassifier of tensorFlow. This network can design the number of layers, the size of each layer, dropout, activation function, learning rate, and so on.

Opt = tf.train.AdamOptimizer(Learning_rate =0.01, beta1=0.9, beta2=0.999)# default 0.001 0.9 0.999 


        m = tf.contrib.learn.DNNClassifier(model_dir=model_dir,
                                           feature_columns=deep_columns,
                                           hidden_units=[1024, 512, 256],
                                           optimizer = opt,
                                           activation_fn=tf.nn.relu,        # defaultDropout = 0.05)Copy the code

So the following is a continuous tuning process, of course, this tuning is also a bit of skill. There are many online, here I will not say one.



Deep Learning Tuner (Image credit: Deep Learning Cheat Guide)

In general, deep learning is not that mysterious, it’s a very effective tool. There have been a lot of attempts by teams to personalize recommendations. Here, we present a simple and effective implementation based on Tensorflow, and hope to help some teams who want to try deep learning.



(Photo by Li Mulai, let’s learn deep learning together)


reading

[I am Jarvis] : Talk about the deep learning vision algorithm behind FaceID

Wang Zhijie: The AI intelligent cloud architecture has greatly improved the ability of intelligent speech recognition

Heterogeneous acceleration technology of deep learning (I) Efficiency is slow due to universal, and the architecture is born from AI

Has been authorized by the author tencent cloud community released, reproduced please indicate the article source The original link: https://cloud.tencent.com/community/article/603674