Abstract: Huawei Cloud community invited Yan Bo, the head of AI Gallery, to listen to him talk about the original intention of AI Gallery design, classic cases and future planning.
This article is from Huawei cloud community “AI experts talk: Reuse algorithms, models, cases, AIGallery take you quickly start application development”, by Huawei cloud community selection.
What interesting and useful AI development cases have you seen?
Ask a still photo of a character to sing, or an anime character to sing. For example, by identifying different types of wildlife and analyzing their population structure, we can protect them. Or intelligent detection of the standardization of wearing masks to help prevent the epidemic…
These scenarioalized AI cases can be found in huawei Cloud AI Gallery corresponding models, through continuous training, you can also achieve. The AI Gallery contains rich and high-quality AI assets such as algorithms, models, data and notebooks. Developers can directly reuse these assets to solve the problems of AI application development.
Throughout the whole development process of AI application, from data collection, annotation to the construction of algorithm model, many REUSABLE AI assets will be generated in every link. The purpose of AI Gallery is to give full play to the utility of these assets and improve the efficiency of AI development.
So, how does it converge these AI assets, and how does it maximize the utility of these assets to help developers develop AI efficiently?
Huawei Cloud community invited Dr. Yan, the director of AI Gallery, to talk about the original design intention, classic cases and future planning of AI Gallery.
Ai has three vehicles: data, algorithms, and computing power. From these three points, what is the current stage of AI application development?
Artificial intelligence is a field that human beings are constantly exploring and developing. Because the stages have not been defined in advance, it is difficult to answer which stage we are currently in. But there is a clear perception that AI is much more advanced than it was 10 years ago, with more and more applications.
This opportunity is the breakthrough of a class of algorithms represented by deep learning in 2012. Before this, we focused more on algorithms, and we used data reduction and some classifiers to do AI development related to machine learning, and the data volume of training was also very small.
At the turning point in 2012, we saw an order of magnitude increase in the accuracy of the AI development model with the help of computing power, algorithms and a large number of data iterations. As this magnitude increases, it will be able to apply AI technology to more industries and domains to improve productivity.
However, at this stage artificial intelligence is not able to make logical inferences by learning small amounts of data like humans. In essence, AI uses large amounts of data to fit and iterate, allowing it to “remember” the data and do some reasoning, but it doesn’t have the ability to make logical inferences. However, AI has become more accurate in the end than it has been in the past decade, and its applications have expanded further.
Further breakthroughs in computing power, combined with algorithms and data, will eventually lead to AI with higher accuracy and even the ability to make logical inferences like humans.
What are the steps involved in a complete AI application development process, and what are the challenges?
There are generally three processes.
The first step is data preparation, need to collect data, data cleaning, conversion work. Each has its own challenges. Take data as an example, there will be policy and legal restrictions in the collection stage, and it is difficult to break the data island. In addition, the data should be effectively marked, which requires a lot of manpower to complete, high economic cost.
The second stage is modeling. Based on the prepared data, appropriate algorithms are selected and relevant models are developed. Consider the application scenarios of the trained model. For example, whether the AI application is placed on the mobile terminal or on the cloud server, the requirements for reasoning delay and accuracy are different. Therefore, in the modeling process of AI development, it is necessary to comprehensively understand the scenarios of AI application, and then choose the appropriate algorithm and process architecture. It is different from the academic field that only pursues accuracy or reasoning speed. We need to consider comprehensively, so the challenge is relatively high.
The third stage is the development of specific AI applications based on models. IT focuses on specific application scenarios and develops some IT systems, software and UI interactions. For example, algorithm engineer is responsible for modeling development, and application engineer may assume the role of application engineer when it comes to application development. As an application development engineer, I received the developed model, but the delay and precision of reasoning of this model may not reach the ideal state. At this point, we need to optimize it further by compression and distillation. If not, it’s up to you to see if the application can eventually be designed to circumvent these problems.
Does the AI Gallery address some of the problems mentioned above? What was it designed for?
Many development processes are now platform-based, and each stage of AI development generates digital assets: algorithms, models, data sets, and possibly processing functions, methods, and the like. We hope that there is a place where these things can be deposited and accumulated, so that subsequent developers can reuse some of the previous achievements. This is also the original intention of our design of AI Gallery.
As more and more developers share AI assets in various scenarios, the AI Gallery can contain experiments with various precision across the entire scenario, and other developers can use these assets directly based on the final development scenario.
For example, the three stages of AI development may involve different roles. What if an application engineer wants to be involved in AI development but lacks the data and algorithm engineer? The AI Gallery has trained models that application engineers can use. From this perspective, it can improve overall development efficiency.
What is a developer’s primary concern when choosing an algorithm or model? How does AI Gallery respond at this point?
If the data is selected, it is generally based on its industry and field scene to see if there is appropriate data, which is strongly related to the field and industry. Currently we provide a mechanism for data sharing, and many developers have shared data sets of open source standard scenarios so that people can quickly verify their ideas in ModelArts.
In terms of algorithms, developers should first consider whether the model generated by the algorithm is what they want, the input data format of the algorithm during training, the cost required for training, and the environment for running the algorithm.
On the model side, it is important to clarify whether the application development will be deployed on the cloud, on the edge, or on the end, for the final application scenario. The second is the delay of reasoning. For example, the data of medical scene will be very large, and its reasoning is asynchronous, but some scenes require real-time reasoning, which may have high requirements on the reasoning response time. Finally, accuracy, the sensitivity of the application scenario to accuracy.
To sum up, at each stage of AI development, there are many metrics and dimensions to consider. What we need to do is standardize these dimensions and indicators so that developers who publish AI assets can fill in these metrics so that people who use them can browse, sift, and retrieve them quickly to find what they want.
What are the classic cases of AI Gallery that can be introduced to developers?
For some classical algorithms in the field of vision, such as YOLO and ResNet50, the authorities have made a lot of adaptation, but these algorithms have not actually precipitated into this field and industry. Therefore, based on some internal projects, we also did some AI practice cases. Water meter readings, helmet tests, etc. These cases may use the same specific algorithm, but applied in different domains and industry scenarios.
Later, we will let our partners, teachers from universities and developers share their cases together, so that other developers can read these cases and quickly reproduce them, speeding up the whole end-to-end development.
Here are some classic examples of AI Gallery:
Hard hat detection, water meter reading recognition, steel inventory in the construction site scene, using PPO algorithm to play Super Mario Brothers, and Chinese chess AI against.
Such as industrial helmet testing, water meter readings, are based on the precipitation of some of huawei’s projects in the industry. These cases produce models that meet industrial-grade requirements and can be deployed directly. The only difference is data. Currently, we only provide one sample data. If you can collect more and better data, the precision of the trained model will be very good.
What benefits do developers get after AI assets are published in the AI Gallery?
For ISV partners, AI Gallery is connected with Huawei Cloud Market, so they can sell asset models on the cloud market and directly obtain commercial benefits.
For developers right now, it’s more about personal achievements and accolades. In the follow-up, we are also actively introducing individual developer programs, so that ordinary individual developers can participate in the whole project and truly enter the actual combat link, which can not only get practice exercise, but also get economic returns.
How does AI Gallery help Pratt & Whitney AI?
One is the accumulation of assets and cases. There are already many developers contributing mainstream open source datasets on the AI Gallery that others can use to validate algorithms directly. In terms of algorithms and models, the authorities have also pre-integrated many commonly used algorithms. Universities are also Posting algorithms for some classic papers to the AI Gallery to share.
Second, the sharing mechanism. Developers can share algorithms and models in the AI Gallery, and then we are trying to put some incentive mechanisms in place to make them more motivated to share.
Third, we launched a case library for end-to-end case scenarios. Although there are not many current cases, huawei officials, ISVs, partners and individual developers will release the summary of project cases developed and delivered in the future, so that the majority of developers can learn these cases to accelerate the application development process.
What are the future plans for AI Gallery?
The first direction is to accelerate the adoption of AI in industries and enterprises. The first is to improve AI development efficiency through asset precipitation. The second is the demand square of the projects we are working on and the certification mechanism of developers. By reducing the links in the development process, more developers and partners can develop and deliver AI projects through the AI Gallery, which will ultimately help the industry and enterprises solve problems and accelerate the implementation of applications.
The other direction is mainly oriented towards learning and education scenarios. At present, we do iterative training and development based on a large amount of computing power and data, but many universities may not be able to keep up with the hardware, so we need to carry out teaching practice on the cloud. So for the education industry, for the individual developer learning scenario, we are also going to do some things, including teaching courses, paper interpretation and so on.
Ultimately, we hope to bridge the two lines, providing one-stop solutions for teaching, training and learning, as well as enabling developers to put what they have learned into practice in real delivery scenarios.
Click to follow, the first time to learn about Huawei cloud fresh technology ~