How to use TensorFlow Mobile in an Android app

• How to Use TensorFlow Mobile in Android Apps
• By Ashraff Hathibelagal
• Translation from: The Gold Project
• This article is permalink: github.com/xitu/gold-m…
• Translator: luochen
• Proofreader: ALVINYEH LeeSniper

TensorFlow is one of the most popular machine learning frameworks available today, allowing you to easily create and train deep models — often called deep feedforward neural networks — that can solve complex problems such as image classification, target detection, and natural language understanding. TensorFlow Mobile is a library designed to help you leverage these models in your Mobile applications.

In this tutorial, I’ll show you how to use TensorFlow Mobile in an Android Studio project.

preparation

To be able to follow the tutorial, all you need to do is:

• Android Studio 3.0 or higher
• TensorFlow 1.5.0 or higher
• An Android device capable of running API Level 21 or higher
• And a basic understanding of the TensorFlow framework

1. Create the model

Before we can start using TensorFlow Mobile, we need a TensorFlow model that has been trained. So let’s create one.

Our model will be very basic, similar to an XOR gate, which takes two inputs, which can be zero or one, and then has an output. If the two inputs are the same, the output is zero. Also, because it will be a deep model, it will have two hidden layers, one with four neurons and the other with three. You are free to change the number of hidden layers and the number of neurons they contain.

To keep this tutorial simple, we’ll use TFLearn, a popular TensorFlow encapsulation framework that provides a more straightforward and concise API than the low-level TensorFlow API. If you have not already installed it, use the following command to install it in the TensorFlow virtual environment:

pip install tflearn
Copy the code

To start creating models, it’s a good idea to create a new Python script called create_model.py in an empty directory, and then open it with your favorite text editor.

In the file, the first thing we need to do is import the TFLearn API.

import tflearn
Copy the code

Next, we must create the training data. For our simple model, there are only four possible inputs and outputs, similar to the contents of the XOR gate truth table.

X = [
    [0.0],
    [0.1],
    [1.0],
    [1.1]
]
 
Y = [
    [0].# Desired output for inputs 0, 0
    [1].# Desired output for inputs 0, 1
    [1].# Desired output for inputs 1, 0
    [0]   # Desired output for inputs 1, 1
]
Copy the code

When assigning initial weights to all neurons in a hidden layer, it is usually best to use random numbers generated from a uniform distribution. You can generate these values using the Uniform () method.

weights = tflearn.initializations.uniform(minval = - 1, maxval = 1)
Copy the code

At this point, we can start building the neural network layer. To create an input layer, we must use the input_data() method, which allows us to specify the number of inputs that the network will accept. Once the input layer is ready, we can call the fully_connected() method multiple times to add more layers to the network.

# input layer
net = tflearn.input_data(
        shape = [None.2],
        name = 'my_input'
)
 
# hidden layer
net = tflearn.fully_connected(net, 4,
        activation = 'sigmoid',
        weights_init = weights
)
net = tflearn.fully_connected(net, 3,
        activation = 'sigmoid',
        weights_init = weights
)
 
# output layer
net = tflearn.fully_connected(net, 1,
        activation = 'sigmoid', 
        weights_init = weights,
        name = 'my_output'
)
Copy the code

Notice that in the code above, we gave meaningful names to the input and output layers. This is important because we need them when we use networks in Android apps. Also note that the sigmoid activation function is used for the hidden and output layers. You can try other activation functions, such as SoftMax, tanh, and relu.

As the final layer of our network, we must create a regression layer using the regression() function, which requires some hyperparameters as its parameters, such as the learning rate of the network and the optimizer and loss functions it should use. The following code shows you how to use stochastic gradient descent (SGD) as the optimizer function and mean square error as the loss function:

net = tflearn.regression(net,
        learning_rate = 2,
        optimizer = 'sgd',
        loss = 'mean_square'
)
Copy the code

Next, in order to let the TFLearn framework know that our network model is actually a deep neural network model, we need to call the DNN() function.

model = tflearn.DNN(net)
Copy the code

The model is now ready. What we’re going to do now is use the training data that we created earlier. Therefore, the fit() method of the model is called and the training data and training cycle are specified. Since the training data are very small, our model will need thousands of iterations to reach reasonable accuracy.

model.fit(X, Y, 5000)
Copy the code

Once the training is complete, we can call the predict() method of the model to check whether it generates the expected output. The following code shows how to check the output for all valid inputs:

print("1 XOR 0 = %f" % model.predict([[1.0]]).item(0))
print("1 XOR 1 = %f" % model.predict([[1.1]]).item(0))
print("0 XOR 1 = %f" % model.predict([[0.1]]).item(0))
print("0 XOR 0 = %f" % model.predict([[0.0]]).item(0))
Copy the code

If you run the Python script now, you should see something like this:

Note that the output will not be exactly 0 or 1. But floating-point numbers close to zero or one. Therefore, when using output, you might want to use Python’s round() function.

Unless we explicitly save the model after training, we will lose it as soon as the program ends. Fortunately, with TFLearn, you can save the model simply by calling the save() method. However, in order to be able to use the saved model in TensorFlow Mobile, we must make sure that all training related operations are removed before saving. These operations are in the tf.Graphkeys.train_OPS collection. The following code shows how to remove it:

# Remove training related operations
with net.graph.as_default():
    del tf.get_collection_ref(tf.GraphKeys.TRAIN_OPS)[:]
 
# Save model
model.save('xor.tflearn')
Copy the code

If you run the script again, you’ll find that it generates checkpoint files, metadata files, index files, and data files, all of which, when used together, can quickly rebuild our trained model.

2. Curing model

In addition to saving the model, we had to solidify it before we could use it with TensorFlow Mobile. As you may have guessed, the process of curing a model involves converting all of its variables to constants. In addition, the solidified model must be a single binary that conforms to the Serialization format of Google Protocol Buffers.

Create a new Python script named freeze_model.py and open it with a text editor. We will write the solidified model code in this file.

Since TFLearn does not have any model curing capabilities, we must now use the TensorFlow API directly. Import them by adding the following lines to the file:

import tensorflow as tf
Copy the code

Throughout the script, we will use a single TensorFlow session. We use the Session class constructor to create a Session.

with tf.Session() as session:
    The rest of the code is here
Copy the code

At this point, we must create the Saver object by calling the import_meta_graph() function and passing it the name of the model’s metadata file. In addition to returning the Saver object, The import_meta_graph() function also automatically adds the graph definition of the model to the graph definition of the session.

Once the saver is created, we can initialize all variables that exist in the diagram definition by calling the restore() method, which needs to contain the directory path of the model’s latest checkpoint file.

my_saver = tf.train.import_meta_graph('xor.tflearn.meta')
my_saver.restore(session, tf.train.latest_checkpoint('. '))
Copy the code

At this point, we can call the convert_variables_to_constants() function to create a solidifiable graph definition where all variables of the model are replaced with constants. As its input, the function needs the current session, a graph definition for the current session, and a list containing the name of the model’s output layer.

frozen_graph = tf.graph_util.convert_variables_to_constants(
    session,
    session.graph_def,
    ['my_output/Sigmoid'])Copy the code

Calling the SerializeToString() method defined by the solidified graph gives us a binary Protobuf representation of the model. By using Python’s basic file I/O, I recommend that you save it as a file called frozen_model.pb.

with open('frozen_model.pb'.'wb') as f:
    f.write(frozen_graph.SerializeToString())
Copy the code

You can now run the script to generate the solidified model.

We now have everything we need to get started with TensorFlow Mobile.

3. Android Studio project Settings

The TensorFlow Mobile library is available on JCenter, so we can add it directly as an implementation dependency in the app module build.gradle file.

implementation 'org. Tensorflow: tensorflow - android: 1.7.0'
Copy the code

To add a solidified model to your project, place the frozen_model.pb file in the assets folder of your project.

Initialize the TensorFlow interface

TensorFlow Mobile provides a simple interface that we can use to interact with our solidified model. To create the interface, please use the TensorFlowInferenceInterface constructor of a class, the class needs a AssetManager instance and solidification model file name.

thread {
    val tfInterface = TensorFlowInferenceInterface(assets,
                                        "frozen_model.pb")
     
    // More code here
}
Copy the code

In the code above, you can see that we are generating a new thread. This is to ensure that the application’s UI remains responsive and is not necessary but recommended.

To ensure that TensorFlow Mobile reads our model’s file correctly, let’s now try printing the names of all the operations in the model diagram. To get a reference to the graph, we use the graph() method of the interface and get all the operations, the operations() method of the graph. The following code tells you what to do:

val graph = tfInterface.graph()
graph.operations().forEach {
    println(it.name())
}
Copy the code

If you run the application now, you should be able to see a dozen operation names printed in the Logcat window of Android Studio. If we solidify the model without error, we can find the names of the input and output layers in these names: my_INPUT /X and my_output/Sigmoid.

5. Use model

To make predictions with the model, we put data into the input layer and get data at the output layer. Entering data into the input layer requires the interface’s feed() method, which requires the name of the layer, the array containing the input data, and the dimension of the array. The following code shows how to enter the numbers 0 and 1 into the input layer:

tfInterface.feed("my_input/X".floatArrayOf(0f, 1f), 1, 2)
Copy the code

Once the data is loaded into the input layer, we must infer using the run() method, which requires the name of the output layer. Once the operation is complete, the output layer contains the model’s predictions. To load the prediction results into the Kotlin array, we can use the fetch() method. The following code shows how to do this:

tfInterface.run(arrayOf("my_output/Sigmoid"))
 
val output = floatArrayOf(-1f)
tfInterface.fetch("my_output/Sigmoid", output)
Copy the code

You can now run the application to see if the model’s predictions are correct.

You can change the numbers entered into the input layer to confirm that the model’s predictions are always correct.

conclusion

You now know how to create a simple TensorFlow model and use it on your Android app with TensorFlow Mobile. But don’t stick to your own model, and with what you’ve learned today, you should have no problem working with a larger model. Examples include MobileNet and Inception, which can be found in the model campus of TensorFlow. Note, however, that these models can make the APK larger, causing problems for users with low-end devices.

To learn more about TensorFlow Mobile, see the official documentation.

---

Diggings translation project is a community for translating quality Internet technical articles from diggings English sharing articles. The content covers the fields of Android, iOS, front end, back end, blockchain, products, design, artificial intelligence and so on. For more high-quality translations, please keep paying attention to The Translation Project, official weibo and zhihu column.