The paper

AI has already invaded our lives, such as face recognition, language recognition, smart audio and so on. However, data processing and model reasoning of AI in the early days are all in the cloud. With the change of time and demand, smart researchers have begun to put AI reasoning training into terminals (mobile phones, IoT devices, etc.). MNN framework is the end – side inference engine launched by Ali

The main purpose of this article is to introduce the process of bottom-up computing, understand what is bottom-up computing, provide ideas for mobile developers to start learning, let’s discuss learning together, if it is useful to you remember to like encouragement ❤

Why do on-end calculations

The process of traditional intelligent computing

Just briefly describe the basic process

  1. The terminal collects data to the server
  2. The server uses algorithms to train the model based on the collected data, resulting in a commercially available model that can then be deployed
  3. This step is called the inference stage, in which the terminal sends the user’s data to the server, the server uses the model to perform inference calculation, and then returns the result to the terminal
  4. The terminal performs operations based on the results

As you can see from the above process, there are several problems

  1. Data privacy, data needs to be reported to the server, and can not be collected if the user does not authorize it
  2. The terminal must wait for the network request to complete before responding to the network delay, which is not suitable for tasks with strong real-time operation
  3. Computing power centralized, completely dependent on server computing power

End up computing advantage

On-end computing, which involves reasoning and training at the end, has the following advantages

  1. It has fast response speed, does not rely on cloud service requests, and can make decisions quickly. At present, many applications have been used, such as Taobao, Xianyu, Douyin, etc
  2. Data privacy security, more and more strict policy control of data privacy, it is more and more difficult to upload data to the server
  3. Save server resources
  4. While the computing power of a single mobile phone /IoT device is limited, if all devices are put together, it will far surpass the computing power of cloud services, but this direction is still developing

The installation

This paper takes the iOS platform as an example for installation. Compared with other libraries, MNN installation is a little complicated, and the official documentation is not particularly sound (you can refer to the tutorial on the official website according to your personal preference). In short, if you encounter problems, solve them

downloadMNNThe source code

git clone https://github.com/alibaba/MNN.git
Copy the code

Install the operating

The first step:

Access the main directory of the warehouse: MNN

The second step:

Check for basic dependencies: cmake, protobuf, C++ compiler (GCC \clang), if not installed to find their own installation

The official requirements and check commands for dependent versions are as follows:

dependency Recommended version Command to check whether it is installed
cmake The theory relies on 3.0 or higher, but the proposed version and the version in the test environment are 3.10+ cmake -v
protobuf Version >= 3.0 is required protoc --version
C + + compiler GCCorClangVersion >= 4.9 is installed by default for macOSClang clang -vorgcc -v

Step 3:

Compile model converter: MNNConvert, the whole compilation process takes a few minutes

What is the main function of MNNConvert? Model training can be carried out through different frameworks, such as TensorFlow and Caffe, which generate different model formats. To run in MNN, they need to be converted into MNN supported formats

1. cd MNN/
2. ./schema/generate.sh
3. mkdir build
4. cd build
5. cmake .. -DMNN_BUILD_CONVERTER=true && make -j4
Copy the code

Step 4 :(optional)

Download the model required by the demo and convert it to the model supported by MNN. This model is mainly used by the demo project and can be skipped

Note in particular: Two problems are often encountered in this step:

  1. Forgetting to compile the model converter first:MNNConvert(Step 3)
  2. When downloading the demo model, it is easy to download the failure, maybe because of the wall or other reasons. If the failure continues, you can download the model manually, and then convert it (roughly operation is to modify./tools/script/get_model.shScript, comment out the download logic, and then manually download after completion, then execute the following command, unsuccessful operation can comment or add wechat communication)

Run the following command in the MNN home directory

./tools/script/get_model.sh
Copy the code

Step 5:

Open the project with Xcode/ios/MNN. Xcodeproj, click compile, compile a complete library of MNN out just the way they are

Conclusion:

The installation process does three things

  1. Compile model converterMNNConvert, is mainly used to convert models generated by other frameworks (such as TF and caffe Del) intoMNNAvailable models
  2. Download the demo model and use itMNNConvertConverted toMNN modelThis step is optional because it is used to provide the model for the demo project
  3. compileMNNlibrary

In general, if you are developing and debugging the source code, MNNConvert should be compiled and the MNN library should be compiled as well. If you just want to see the demo project, then you don’t need to do the above. Download the following demo project and run it

The demo project

A direct way to learn a framework is to see how it is used in the demo. The installation steps above mentioned, because the demo uses a third-party open source model, so you must first complete the above installation operation of the fourth step, and then find the path MNN/ Demo under the Demo project, open the direct run can be

In the demo project can be source debugging, API usage, etc., go to learn!

The process of performing reasoning

Looking at MNN from a higher perspective, regardless of the technical details, MNN is mostly about reasoning (and maybe training later), so guess what you need to do: load the model, input data, output results, etc. We take demo project as an example to study the reasoning process

Model loading: Interpreter & Session

The whole process is likened to a browser loading a web page

A model Interpreter, like a browser’s engine, that interprets a web page, analyzes its structure, and so on for rendering

Session: Session. When webpage rendering is completed, Session operations, such as requesting a picture or clicking button, are always carried out, which is a communication process. In MNN, Session is an inference process and holds inference data

So the Interpreter loads the model, the Session is responsible for infering the data, the Session is created through Interpreter, and a single Interpreter can create multiple sessions. Let’s look at source code parsing

The Interpreter and Session object

The statement

@interface Model : NSObject { std::shared_ptr<MNN::Interpreter> _net; // Interpreter used to hold model data MNN::Session *_session; // Session, reasoning data holder}Copy the code

In the Model class, you declare the interpreter object _net and the session object _session, which is a base class because different types of models create their own Models and inherit from Model classes

Create a Interpreter

NSString *model = [[NSBundle mainBundle] pathForResource:@"mobilenet_v2.caffe" ofType:@" MNN "]; / / from the disk load model, of course, is also the support from memory read _net = STD: : from < MNN... Interpreter > (MNN: : Interpreter: : createFromFile (model. UTF8String));Copy the code

Create a session

- (void)setType:(MNNForwardType)type threads:(NSUInteger)threads { MNN::ScheduleConfig config; // Ignore config. Type = type; config.numThread = (int)threads; If (_session) {// releaseSession net->releaseSession(_session); } // create a new session _session = _net->createSession(config); }Copy the code

Now that the model is loaded and ready, how do you enter data

Input and run data

Model loading is completed, it is time to start reasoning, in fact, do not want to be too complicated, just a few interfaces, as a starter to understand the skeleton and then see the details

Data containers

So the Tensor class in MNN is designed to be a container for the data that you put in, as explained below

/** * data container. * Data for host tensor is saved in 'host' field. Its memory is allocated malloc directly. * Data for device tensor is saved in `deviceId` field. its memory is allocated by session's backend. * usually, device tensors are created by engine (like net, session). * meanwhile, host tensors could be created by engine or user. */
class MNN_PUBLIC Tensor {
public:
    struct InsideDescribe;

    /** dimension type used to create tensor */
    enum DimensionType {
        /** for tensorflow net type. uses NHWC as data format. */
        TENSORFLOW,
        /** for caffe net type. uses NCHW as data format. */
        CAFFE,
        /** for caffe net type. uses NC4HW4 as data format. */
        CAFFE_C4
    };

    /** handle type */
    enum HandleDataType {
        /** default handle type */
        HANDLE_NONE = 0./** string handle type */
        HANDLE_STRING = 1}; . }Copy the code

Input data & run

You’ll get a Tensor object back from the input, and that’s the code for that.

  • Input data functiongetSessionInputThe definition of
Tensor* Interpreter::getSessionInput(const Session* session, const char* name){}
Copy the code
  • Demo project example code
Tensor = _net->getSessionInput(_session, nullptr); MNN::Tensor tensorCache(input); input->copyToHostTensor(&tensorCache); for (int i = 0; i < cycles; i++) { input->copyFromHostTensor(&tensorCache); // start run_net ->runSession(_session); }...Copy the code

To get the results

So once we’ve done the inference run, we need to get the result, and we’ll get the Tensor object back

  • The function that gets the resultgetSessionOutputdefine
Tensor* Interpreter::getSessionOutput(const Session* session, const char* name) {}
Copy the code
  • Demo project example code
MNN::Tensor *output = _net->getSessionOutput(_session, nullptr); MNN::Tensor *output = _net->getSessionOutput(_session, nullptr) MNN::Tensor copy(output); output->copyToHostTensor(&copy); float *data = copy.host<float>(); .Copy the code

summary

This process is designed to introduce the use of the entire framework process, combined with the demo project to understand faster, and not too in-depth, I hope to quickly start

MNNKit

Sometimes I think about some problems: in fact, for intelligent computing, we only need a suitable model and then use it for reasoning calculation, then the model can be reused, there is no need for everyone to retrain a set of models

For this problem, Ali put out the super mature model for everyone to use, these models have been practiced on Taobao and other apps, it is MNNKit, based on the end of the inference engine MNN provided a series of application layer solutions, currently open three models, developers can directly use to play

Kit SDK function
FaceDetection Face recognition
HandGestureDetection Gesture recognition
PortraitSegmentation Like splitting

MNNKit organizational structure

(Direct reference to GitHub library description)

MNN layer: The most core layer, providing model loading and reasoning

MNNKit Core: mainly for MNN C++ interface encapsulation and abstraction, into a higher level OC or Java implementation of the API, convenient for the caller to use

Business layer: mainly for the encapsulation of specific algorithm model, mainly for business use

conclusion

What do you need to do on – end computing? For mobile developers, it is very strange to get to know AI development for the first time. How to start AI development is very strange. I am also learning now.

If you are familiar with mathematics is of certain words, don’t be one by one, to start from scratch to learn mathematics, it really is too inefficient and Suggestions from the developers most familiar code level first, understand the basic workflow of AI, such as learn how to use MNN, you will know the original is take model reasoning and get the result feedback to the user

Then in learning how to train the model, it is suggested to start with a simple machine learning algorithm, according to the algorithm needs to use the mathematical formula, and then check the use of these formulas, while learning to better understand

The whole down introduced what is the end of the calculation and advantages, the MNN library has a basic introduction, to this I believe that we have a basic understanding. Feel useful to help like + pay attention to it, this is the biggest encouragement to the author!

Finally, welcome to pay attention to the public number: [code on the work], this public number is committed to a simple and easy to understand the way to explain mobile/full stack development, interesting algorithms, computer basic knowledge, etc., to help you build a complete knowledge system, together to become a top developer. There are a lot of learning materials on public account 12345:

  1. IOS ebooks (Get organized)
  2. Full Stack Development Video Tutorial (full set)
  3. Machine Learning Video (full set)
  4. Blockchain Video Tutorial (full set)