Abstract:With the exponential increase of data volume and the limitation of regular data types, the business scenario expansion of deep learning becomes more difficult. Graph neural network can make more accurate prediction, provide different personalized services for each user, and achieve accurate marketing, which is also the technological breakthrough for the second transformation of Internet enterprises today.
In the annual online shopping festival like 618, personalized and accurate marketing promotion is very powerful for e-commerce platforms. Therefore, how to select the products that consumers are most likely to buy from the mass of products has become the focus of many e-commerce platforms’ technology efforts. The essential ingredient behind this is the AI.
As a relatively mature AI technology, deep learning has been widely used in industrial-level production and enterprise development as an export of Internet dividend in the past. However, with the exponential increase of data volume and the limitation of regular data types, the expansion of business scenarios of deep learning has become more difficult.
Therefore, the market began to focus on the graph neural network (GNN) technology. Graph neural network can make more accurate prediction, provide different personalized services for each user, and achieve accurate marketing, which is also the technological breakthrough for the second transformation of Internet enterprises today.
At present, Huawei Cloud map neural network is greatly improving the overall computing efficiency by virtue of the efficient neural network training advantage of ModelArts, making the application of graph neural network including commodity recommendation more mature.
Industry application of graph neural network
At present, the mainstream deep learning is still CNN, RNN and other technologies (corresponding to image recognition, text mining and other fields). However, traditional deep learning technologies (CNN, RNN) cannot effectively deal with structural data, such as financial field, gene protein network, social network, product recommendation, etc. If deep learning is to be extended to more relational scenarios, graph neural network (GNN) technology will achieve better results for higher order learning on graph data.
Take the knowledge graph, for example. Its application as a graph neural network is better known for its scenarios than for the technology itself. There are many scenes in life with knowledge graph, such as semantic search engine, intelligent customer service, life little assistant, etc. The knowledge graph constructed by graph neural network can provide video/live subtitles, content review, intelligent customer service, insurance compensation, medical map, knowledge elimination and other services. With the help of knowledge graph, the exclusive industry knowledge can also be made into a graph network to analyze industry information and help enterprises to transform and upgrade.
In the future, ARTIFICIAL intelligence will operate more like the human brain, and the emergence of graph neural networks will allow ARTIFICIAL intelligence to begin to understand the world, to understand the world, rather than just perform statistical fitting. How to make graph deep learning fully exploit its application value and realize the implementation of application scenarios of high-dimensional sparse data will be the key to reshuffle homogenized enterprises in the next decade.
ModelArts2.0 marks the landing of Huawei Cloud map neural network
In last year’s Huawei Cloud Fully connected Conference, Huawei Cloud launched ModelArts2.0, a one-stop AI development and management platform. Announced that Huawei cloud has made a breakthrough in the field of deep learning, huawei cloud neural network officially landed.
This ModelArts2.0 release more than ten new features and services, including intelligent data screening, intelligent data annotation, intelligent data analysis, multiple model automatic search, ModelArts SDK, graph neural network, reinforcement learning, model evaluation/diagnosis, model compression/transformation, automatic difficult case discovery, online learning, etc. Covers the full life cycle of the AI model. It can be seen that Huawei Cloud ModelArts is playing a very big game. The landing of graph neural network is a breakthrough for ModelArts to achieve causal reasoning in the field of deep learning, and it is also an essential link for realizing automatic AI ability.
Huawei Cloud graph neural network is a new graph neural network technology jointly created by GES Graph Engine and ModelArts. Through the parallel construction of distributed graph computing platform and deep learning computing platform, a new architecture is constructed to achieve large-scale graph neural network analysis capability.
Huawei Cloud Network architect describes the principles for designing the Huawei Cloud network neural network (GNN) framework as follows: Clear responsibilities and unified architecture. For a single algorithm, sparse processing operations such as data preprocessing and field sampling are pushed down to the graph engine. The deep learning layer focuses on the optimization of operators. Multiple GNN algorithm frameworks are unified and unified operators are reused.
Distributed graph computing platform for large-scale graph network processing
In the calculation of enterprise-level graph deep learning, the scale of graph will reach tens of billions or even hundreds of billions according to business needs. Therefore, a mature graph deep learning will hand over the computation of super-large graph network to an independent distributed graph computing platform.
At present, most of the graph neural network frameworks are dealing with static graphs. This is because most of them treat the graph neural network algorithm as an offline computing task. The data of offline computing is unchanged (static), and the complete data need to be loaded once for each calculation, so it is not suitable for processing dynamic graphs. But it is often change figure data itself (dynamic) algorithm in the process of running need to constantly to traverse the graph, then put the figure data from memory calls for deep learning model, and then returned to constantly in the process of modeling, the problem on the picture is not very obvious, but in the hundred million map network becomes a serious performance issues, And the traversal time will increase exponentially, even causing outages.
Huawei maintains its own GES graph engine to maintain graph data and ensure that data can be dynamically added, deleted and modified. At the same time, execute a variety of different algorithms on a piece of data, without reloading data; Especially for large scale diagrams, there are significant end-to-end time savings. At present, the processing of dynamic graphs can be optimized. For example, data changes on dynamic graphs can be regarded as incremental data. The best practice is to design incremental algorithms to analyze incremental data, rather than neighborhood sampling, random walk, gradient calculation and other operations on the full data. The research of incremental graph neural network algorithm is still in the forefront and has not yet formed a complete theory.
GES graph engine currently has more than 20 graph scene algorithms and a large number of graph optimization algorithms, performance can be hundreds of millions of graph query in the second calculation. GES graph engine on the graph algorithm according to industry and enterprise needs, integrated PageRank and other more than 20 common algorithms, application scenarios cover urban industrial production, pipeline monitoring, commodity recommendation, social recommendation, project analysis, enterprise insight, knowledge map, financial risk management and control, enterprise IT application, relationship mining and other fields. And support point search, edge search, attribute filtering and other basic queries will query storage and other functions.
Take the Pixie algorithm as an example. The Pixie algorithm is an algorithm designed by Huawei Cloud to construct multiple data into the same graph and configure corresponding schema, point-edge attributes, and weights on the heterogeneous graph. Pixie algorithm is a new real-time recommendation algorithm, which overcomes the problem of data acquisition and fusion of heterogeneous graphs, supports comprehensive recommendation under multi-request nodes, and can meet the requirements of various complex, time-varying and diversified recommendation scenarios. With a large amount of data, the model can adapt to the dynamic changes of data without prior training, and achieve a better real-time recommendation effect, with strong scalability.
The new framework solves the problem of high frequency interaction between graph algorithms and deep learning
Improving the efficiency of data processing and unified algorithm framework based on native graph engine are the key and difficult points of current graph neural network platform research and development. However, graph data traversal and interaction with deep learning will greatly reduce graph operation efficiency, which is also one of the bottlenecks of graph deep learning.
Therefore, if graph deep learning wants to make a breakthrough in performance, it needs to redesign a new GNN framework. The following is the frame diagram of Huawei cloud map neural network authorized by AI Front.
(1) New GNN framework based on graph engine: In ModelArts efficient neural network training operator, on the basis of combining the GES image existing high-performance computing framework platform, use map engine high concurrency, the characteristics of low latency, will be highly parallelized geri weis-corbley training process, such as the jump probability estimates, at the edge of vertex neighborhood sampling, negative sample build, etc., to resolve to the local operation of each vertex; The system provides a dynamic scheduler so that these local operations can be executed in high parallelism, which can greatly improve the overall throughput of the system.
(2) Unification of various GNN algorithm frameworks: unified architecture is used to realize unsupervised large-scale graph embedding (such as DeepWalk, Node2Vec) and semi-supervised graph convolution (such as GCN, GraphSage) and other multi-class GNN algorithms, reducing the maintenance cost of the system.
Figure: Diagram of graph embedding and graph convolution calculation based on unified GNN architecture
(3) Integration of GNN and graph data management: enterprise-level GNN applications are usually not one-off calculations, and the data scale is large, so the data must be maintained and managed. However, existing GNNS usually do not have such considerations, and users can only build another database to maintain, and then export the data as a whole during calculation. It not only consumes large resources, but also introduces many problems such as data consistency. GES adopts Property Graph data model and ecologically compatible fact standard Gremlin Graph query language to manage and maintain distributed Graph data. When training is needed, all kinds of operators are called locally in the Graph engine and executed concurrently, which reduces end-to-end performance loss.
The r&d personnel compared the experimental performance of this product with multiple open source versions in data preprocessing and various sampling methods on the same platform (from huawei cloud internal data) :
Figure: (1) Performance comparison between the same platform and the open source version in data preprocessing and various sampling methods; (2) System scalability test results
Huawei Cloud map neural network greatly improves the overall computing efficiency of GNN by virtue of the efficient neural network training advantages of ModelArts and the high-performance graph computing advantages of GES. Taking node2VEC algorithm as an example, on PPI data set, Huawei cloud map neural network can complete sampling and training within 2 minutes. 20 times better than traditional open source implementations.
Tradeoffs between precision and resources
In terms of the accuracy of the graph neural network model, Huawei Cloud graph neural network adjusts the model accuracy by setting parameters and uses CPU or GPU to train the graph neural network algorithm.
Due to the particularity of graph data, the performance and effect of CPU training are generally not inferior to that of GPU for most types of data. At the same time, huawei cloud map neural network adopts different optimization methods to reduce resource occupancy rate and improve computing performance for graph embedding and graph convolution algorithms. The graph embedding algorithm uses parallel acceleration and storage design to optimize positive sampling and negative sampling. The convolution part of the graph focuses on optimizing the acceleration matrix due to its high complexity of mathematical changes between layers. In the future, Huawei Cloud will consider further improving the computing performance of graph neural network based on its own ARTIFICIAL intelligence chips from the hybrid hardware architecture.
The life cycle management of Huawei Cloud Map neural network model relies on ModelArts, huawei cloud one-stop AI development and management platform. The trained model can be deployed with one click, and the entire data-algorithm-model-reasoning life cycle can be viewed through the traceability diagram provided by the platform.
At present, the industry to implement a large-scale map neural network applications need a period of time, but huawei cloud neural network provides a reference for subsequent developers of the theory of experience and the social, financial, genes, image semantic relations more scenarios, the basis of the practice of the huawei cloud neural network has been in the world of machine learning and data mining kinds of academic conference paper, And won the “Zijin Longpan Award” of 2019 China Artificial Intelligence Summit.
conclusion
Graph neural network is a step for artificial intelligence to realize real intelligence, and it is also the beginning of ARTIFICIAL intelligence to solve the relational data difficult to deal with deep learning. From now on, artificial intelligence can recognize and learn the complex relationship of the world, and I believe that it will appear in our life in more gestures in the future, the most intuitive is the current promotion of various online e-commerce shopping.
Huawei cloud 618 promotion, AI development platform ModelArts also prepared a 10% discount package for users, students who are interested in graph neural network or AI development, everyone!
Click to follow, the first time to learn about Huawei cloud fresh technology ~