Serverless TensorFlow function in the public cloud

For software developers and students, AI is paid.

In 2021, the average annual salary of the developer of Tensorflow, the most popular AI framework, was $148,508, which translates into nearly one million yuan.

Ai skills are now required for even entry-level programming jobs. In fact, it’s pretty easy to find a tutorial to train your Tensorflow model for tasks like image recognition and natural language processing. You only need some basic Python knowledge to train the model and run it to demonstrate it.

It’s just that knowing how to train models in simple Python doesn’t bring much benefit.

It is much more difficult to train a model as a reliable Web service for others to use than to train a model. There are significant challenges for developers to deploy the TensorFlow model in a production environment. Companies pay a high price for people who can overcome these challenges.

  • The Python language and framework are inefficient in preparing and processing input and output data for models. It is estimated that 90% of AI computing effort is spent on data preparation. The Python language is simply too cumbersome and slow.
  • It is difficult to scale services on demand. Due to the computational requirements of AI reasoning, the server computer may be temporarily suspended even with only a few requests. Scaling the number of servers up and down as needed is critical.

However, there is a simple solution to this problem. With the WebAssembly VIRTUAL machine SSVM and Tencent Cloud Serverless, you can deploy the Tensorflow model as a service in a production environment using simple Rust code of less than 50 lines.

Simple means that the Rust function syntax used is very simple and does not use life ownership.

With this template function, you can deploy the TensorFlow model on Tencent Cloud for free for use in production environments!

If you follow the steps in this article and deploy the Tensorflow Serverless function on the Tencent cloud, you will get a series of peripheral products. Details of the event can be found here. Please fill out this form and we will send you a gift as soon as possible.

👉 Build video for better viewing

How it works

To address the challenges mentioned above, the technologies selected are as follows:

  • The Rust programming language is fast and memory safe, making it ideal for high-performance data processing and model execution logic.

  • WebAssembly acts as a bridge between modern Rust programs, the latest Tensorflow libraries, and enhanced public cloud operating systems. WebAssembly is a compatibility sandbox that is critical to the developer experience.

  • Tencent Cloud ServerLess provides an extensible infrastructure to run Rust and WebAssembly functions for TensorFlow reasoning. Combined with the Serverless framework, Tencent Cloud Serverless provides excellent developer experience for deploying simple software.

Now, let’s see how it works. First, fork the template project from GitHub and do all the preparatory work. You can use GitHub Codespaces IDE or Docker images, or you can install Rust, SSVMUp, serverless Framework on your computer.

Here is image recognition AI as a service. It uses trained TensorFlow models to identify food in images. With less than 50 lines of simple Rust code, it can be deployed on Tencent Cloud Serverless. Tencent Cloud Serverless can scale on demand and is charged based on actual usage.

View the deployed web page

code

Rust code for loading an input image loads and executes the Tensorflow model to identify the content on that image. The model here is in Tensorflow Lite format and recognizes the food on the input image.

// Load the trained TensorFlow Lite model. let model_data: &[u8] = include_bytes! ("lite-model_aiy_vision_classifier_food_V1_1.tflite"); // The uploaded image format is Base64 encoding, and encapsulated in JSON object through Tencent cloud API gateway. let mut buffer = String::new(); io::stdin().read_to_string(&mut buffer).expect("Error reading from STDIN"); let obj: FaasInput = serde_json::from_str(&buffer).unwrap(); let img_buf = base64::decode_config(&(obj.body), base64::STANDARD).unwrap(); // Load the uploaded image and adjust it to 192x192, which is the required size for this MobileNet model. let flat_img = ssvm_TensorFlow_interface::load_jpg_image_to_rgb8(&img_buf, 192, 192); // Run the model with the image as the input tensor and get the model output tensor. let mut session = ssvm_TensorFlow_interface::Session::new(&model_data, ssvm_TensorFlow_interface::ModelType::TensorFlowLite); session.add_input("input", &flat_img, &[1, 192, 192, 3]) .run(); let res_vec: Vec<u8> = session.get_output("MobilenetV1/Predictions/Softmax");Copy the code

The RES_VEc vector contains a list of probabilities for each object in the image (for example, the probability of a cake in this image is 0.8). The Rust code below reads the labels of these objects and prints them out with the highest probability from the Tensorflow model output.

let labels = include_str! ("aiy_food_V1_labelmap.txt"); let mut i = 0; let mut max_index: i32 = -1; let mut max_value: u8 = 0; while i < res_vec.len() { let cur = res_vec[i]; if cur > max_value { max_value = cur; max_index = i as i32; } i += 1; } let mut label_lines = labels.lines(); for _i in 0.. max_index { label_lines.next(); } let class_name = label_lines.next().unwrap().to_string(); if max_value > 50 && max_index ! = 0 { println! ("The image {} contains a <a href='https://www.google.com/search?q={}'>{}</a>", confidence.to_string(), class_name, class_name); } else { println! ("No food item is detected"); }Copy the code

Deploy your own TensorFlow AI reasoning function

On the back end, Rust code is compiled into WebAssembly bytecode and executed in the SSVM WebAssembly runtime. The SSVM has been pre-configured to access the high-performance TensorFlow native library in multiple operating system environments, including Tencent Cloud’s Serverless container. In turn, Tencent Cloud Serverless provides a simple solution for extending Tensorflow reasoning functions.

Open a Terminal window in Codespaces IDE and run the following command from Docker or command line to build the cloud function.

$ssvmup build - enable - aotCopy the code

In the Terminal window, run the following command line to deploy the TensorFlow cloud function to Tencent Cloud.

$ cp pkg/scf.so scf/ $ sls deploy ... . website: [https://sls-website-ap-hongkong-kfdilz-1302315972.cos-website.ap-hongkong.myqcloud.com](https://sls-website-ap-hongkong -kfdilz-1302315972.cos-website.ap-hongkong.myqcloud.com/)Copy the code

Open your deployed Website URL in your browser and see what you’ve eaten.

summary

In this article, we discussed how to create simple, secure and high-performance Rust functions to run the Tensorflow model, and how to deploy these functions to the public cloud as scalable and on-demand AI services.

Now, you can deploy the Tensorflow Serverless function on Tencent Cloud for free and get exquisite peripheral.