preface
The classification model used in this paper is from CNN-RNN Chinese text classification, based on TensorFlow, thanks to open source.
Recently, I needed to use such a function as Chinese text classification, so I immediately thought of Create ML, but after my own attempts, I found that Create ML does not support Chinese text classification (you can try it yourself).
Recently, I found that Youdao Dictionary has such a function of offline translation. I guess this is to download the model to local use, so it is theoretically feasible to deploy the model to the mobile terminal. However, I only knew about TensorFlow in various deep learning frameworks, so I went back to The pit of TensorFlow. At the end of last year, I said THAT I would never use TensorFlow again.
The tensorFlow-trained model actually works best on the back end, but SINCE I didn’t want to maintain a robust back end for my APP, I stuck with deploying the model to the mobile end. This is the Demo.
Without further ado, I can generalize a few steps to deploying a model from scratch
- Train and test the model and save it in CKPT format
- CKPT model was solidified into PB model
- Convert the PB model to the TFLite model using the method provided by TensorFlow Lite
- Import TensorFlow Lite using Cocoapods and import the model into the project
- Encapsulate call model logic for text categorization
Note: This blog is only deployed according to the open source project above, and other network architectures are subject to case-by-case analysis.
Principle of general classification
If you want to deploy from scratch, you must have some knowledge of TensorFlow, because it is basically impossible to proceed without reading the project’s source code.
This project maps each character in the text to a number (ID) and, through a series of metaphysical operations, yields a one-dimensional array in which the first 10 are the values we care about because there are only 10 labels.
The data processing
We need to understand how data is processed — inputs and outputs — so that we can write code to make predictions in iOS apps.
The input
The open source project maps each character (Chinese character) to an ID from a row in the dataset, meaning that the first row corresponds to a character ID of 0, the second row corresponds to a character ID of 1, and so on. So we’ve got an array of ids. In addition, this ID array needs to be processed into a fixed length. In this paper, in iOS, if there is not enough, add 0 after the array, and if there is more, remove the end of the array.
The output
The output is an array with more than 10 entries, but since there are only 10 categories in the dataset, we only need to focus on the first 10 entries in the array. These first 10 arrays correspond to the subscripts in the label array, and the values of the array are the predicted probabilities. So the subscript of the output array 0-10 corresponds to the possibility of a specific classification of 0-10 in the label array.
The deployment of
Training model
This article uses CNN network in open source project. TensorFlow Lite supports limited operators, so not all operators in TensorFlow are supported. If not, errors like the following will occur in the conversion:
Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.contrib.lite.toco_convert(). Here is a list of operators for which you will need custom implementations: RandomUniform
Copy the codeIn this error, you can see that the unsupported operator is RandomUniform. Of the tf search after CNN. Contrib. The layers. The dropout is not supported, but the problem is not big, we can replace it with L2 regularization to prevent over fitting.
Here is the modified reference code:
# coding: utf-8
from functools import partial
import tensorflow as tf
class TCNNConfig(object):
"""CNN Configuration Parameters"""
embedding_dim = 64 # word vector dimensions
seq_length = 600 # sequence length
num_classes = 10 # category number
num_filters = 256 # number of convolution kernels
kernel_size = 5 # Convolution kernel size
vocab_size = 5000 # Vocabulary is small
hidden_dim = 128 # Fully connected layer neuronsDropout_keep_prob = 0.5Dropout retention ratio
learning_rate = 1e-3 Vector #
batch_size = 64 # Training size per batch
num_epochs = 10 Total number of iterations
print_per_batch = 100 Output the result every number of rounds
save_per_batch = 10 # Every number of rounds is deposited into tensorboardScale = 0.01 class TextCNN(object)"""Text categorization, CNN model."""
def __init__(self, config):
self.config = config
# Three data to be entered
self.input_x = tf.placeholder(tf.int32, [None, self.config.seq_length], name='input_x')
self.input_y = tf.placeholder(tf.float32, [None, self.config.num_classes], name='input_y')
self.keep_prob = tf.placeholder(tf.float32, name='keep_prob')
self.cnn()
def cnn(self):
"""CNN model"""
my_dense_layer = partial(
tf.layers.dense, activation=tf.nn.relu,
The L2 regularization function is passed here, and regularization coefficients are passed in the function.
kernel_regularizer=tf.contrib.layers.l2_regularizer(self.config.scale)
)
# word vector mapping
with tf.device('/cpu:0'):
embedding = tf.get_variable('embedding', [self.config.vocab_size, self.config.embedding_dim])
embedding_inputs = tf.nn.embedding_lookup(embedding, self.input_x)
with tf.name_scope("cnn") :# CNN layer
conv = tf.layers.conv1d(embedding_inputs, self.config.num_filters, self.config.kernel_size, name='conv')
# global max pooling layer
gmp = tf.reduce_max(conv, reduction_indices=[1], name='gmp')
with tf.name_scope("score") :Full connection layer
fc = my_dense_layer(gmp, self.config.hidden_dim, name='fc1')
# fc = tf.layers.dense(gmp, self.config.hidden_dim, name='fc1')
# fc = tf.contrib.layers.dropout(fc, self.keep_prob)
# fc = tf.nn.relu(fc)
# classifier
self.logits = tf.layers.dense(fc, self.config.num_classes, name='fc2')
self.y_pred_cls = tf.argmax(tf.nn.softmax(self.logits), 1) # Prediction category
with tf.name_scope("optimize") :# Loss function, cross entropy
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=self.logits, labels=self.input_y)
reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
self.loss = tf.add_n([tf.reduce_mean(cross_entropy)] + reg_losses)
# self.loss = tf.reduce_mean(cross_entropy)
# the optimizer
self.optim = tf.train.AdamOptimizer(learning_rate=self.config.learning_rate).minimize(self.loss)
with tf.name_scope("accuracy") :# accuracy
correct_pred = tf.equal(tf.argmax(self.input_y, 1), self.y_pred_cls)
self.acc = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
Copy the codeAfter training in run_cnn.py, we can get the following CKPT model:
CKPT model was solidified into PB model
In the solidified model section, you will need to read through the open source project, otherwise you will not understand its network structure and its inputs and outputs. This is also very unfriendly to iOS developers.
We know from the source code that self.logits in TextCNN is the output that we need to worry about, so we can print out the tensor here, and then find the output name that we need
ops = sess.graph.get_operations()
for op in ops:
print(op)
Copy the codeSo the name we need here is theta
output_node_names = "score/fc2/BiasAdd"
Copy the codeReference source:
def freeze_graph(input_checkpoint):
""" :param input_checkpoint: :return: """
# checkpoint = tf.train.get_checkpoint_state(model_folder) # checkpoint = tf.train.get_checkpoint_state(model_folder
# input_checkpoint = checkpoint.model_checkpoint_path
The output node name must be a node that exists in the original model
output_node_names = "score/fc2/BiasAdd"
saver = tf.train.import_meta_graph(input_checkpoint + '.meta', clear_devices=True)
with tf.Session() as sess:
saver.restore(sess, input_checkpoint) # Restore the graph and get data
output_graph_def = tf.graph_util.convert_variables_to_constants( # model persistence, which fixes variable values
sess=sess,
input_graph_def=sess.graph_def, # is equal to: sess. Graph_def
output_node_names=output_node_names.split(","))# If there are multiple output nodes, separate them with commas
with tf.gfile.GFile(output_graph, "wb") as f: # Save the model
f.write(output_graph_def.SerializeToString()) # serialize output
Copy the codeInput_checkpoint is your CKPT model path
Convert pb model to TFLite model
Here are the annotations for the from_frozen_graph method. The documentation for TensorFlow Lite says that the incoming tensor is a tensor, then you have a misplaced misplaced sequence, and then you have to check the source code for your library at the break point and you need the tensor name.
As long as the operator does not not support the situation is very simple, directly on the source code is finished
def convert_to_tflite():
input_tensors = [
"input_x"
]
output_tensors = [
"score/fc2/BiasAdd"
]
converter = tf.lite.TFLiteConverter.from_frozen_graph(
output_graph,
input_tensors,
output_tensors)
converter.target_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
open(output_tflite_model, "wb").write(tflite_model)
Copy the codeInput_x is the input name
Introduce TensorFlow Lite using Cocoapods
TensorFlow Lite has several libraries, and you need to write C++ native. After a while I gave up. OC and SWIFT. Since my project is written by Swift, I use swift’s TensorFlow Lite library directly
Follow their README
pod 'TensorFlowLiteSwift'
Copy the codeimport TensorFlowLite
Copy the codeI’m going to introduce, which is kind of friendly, rather than compiling TensorFlow directly into an iOS project, which is as simple as it could be.
Encapsulate call model logic for text categorization
When we feed data to make predictions, we also need to do things the way we feed data in open source projects. Call logic we can refer to the official Example
Into the model
We need to import models, categories, and character ids, as shown in the demo provided in the introduction to this article.
Initialize the Interpreter
private init() {
let options = InterpreterOptions()
do {
// Create the `Interpreter`.
let modelPath = Bundle.init(for: TextClassifier.self).path(forResource: "model", ofType: "tflite")!
interpreter = try Interpreter(modelPath: modelPath, options: options)
// Allocate memory for the model's input `Tensor`s. try interpreter.allocateTensors() } catch { print("Failed to create the interpreter with error: \(error.localizedDescription)") } }Copy the codeLoad labels, ids, and convert characters to ids
private func loadLabels() {
if let path = Bundle.init(for: TextClassifier.self).path(forResource: "labels", ofType: "txt") {
let fileManager = FileManager.default
let txtData = fileManager.contents(atPath: path)!
let content = String.init(data: txtData, encoding: .utf8)
letrowArray = content? .split(separator:"\n")?? []for row in rowArray {
labels.append(String(row))
}
}
}
private func loadTextId() {
if let path = Bundle.init(for: TextClassifier.self).path(forResource: "text_id", ofType: "txt") {
let fileManager = FileManager.default
let txtData = fileManager.contents(atPath: path)!
let content = String.init(data: txtData, encoding: .utf8)
letrowArray = content? .split(separator:"\n")?? [] var i = 0for row in rowArray {
textIdInfo[String(row)] = i
i += 1
}
}
}
private func transformTextToId(_ text: String) -> [Int] {
var idArray: [Int] = []
for str intext { idArray.append(textIdInfo[String(str)]!) } // According to the input Settings in the Python project, if it exceeds, it will be truncated before, and if it is insufficient, it will be filled with 0while idArray.count < 2400 {
idArray.append(0)
}
while idArray.count > 2400 {
idArray.removeLast()
}
return idArray
}
Copy the codeTo make predictions
public func runModel(with text: String, closure: @escaping(InferenceReslutClosure)) {
DispatchQueue.global().async {
let idArray = self.transformTextToId(text)
let outputTensor: Tensor
do {
_ = try self.interpreter.input(at: 0)
let idData = Data.init(bytes: idArray, count: idArray.count)
try self.interpreter.copy(idData, toInputAt: 0)
try self.interpreter.invoke()
outputTensor = try self.interpreter.output(at: 0)
} catch {
print("An error occurred while entering data: \(error.localizedDescription)")
return
}
let results: [Float]
switch outputTensor.dataType {
case .uInt8:
guard let quantization = outputTensor.quantizationParameters else {
print("No results returned because the quantization values for the output tensor are nil.")
return
}
let quantizedResults = [UInt8](outputTensor.data)
results = quantizedResults.map {
quantization.scale * Float(Int($0) - quantization.zeroPoint)
}
case .float32:
results = outputTensor.data.withUnsafeBytes( { (ptr: UnsafeRawBufferPointer) in[Float32](UnsafeBufferPointer.init(start: ptr.baseAddress? .assumingMemoryBound(to: Float32.self), count: ptr.count)) }) default:print("Output tensor data type \(outputTensor.dataType) is unsupported for this app.")
return
}
let resultArray = self.getTopN(results: results)
DispatchQueue.main.async {
closure(resultArray)
}
}
}
Copy the codeFirst we need to render the [Int] type to Data to Interpreter as follows
let idData = Data.init(bytes: idArray, count: idArray.count)
Copy the codeThe invoke()** method makes a prediction for the call model
Once we’ve got the output outputTensor, the float32 type in its dataType is the output that we need, because the output in open source projects is float32. Here we need to use the swift pointer to change the Data type to [Float] as follows:
results = outputTensor.data.withUnsafeBytes( { (ptr: UnsafeRawBufferPointer) in[Float32](UnsafeBufferPointer.init(start: ptr.baseAddress? .assumingMemoryBound(to: Float32.self), count: ptr.count)) })Copy the codeUInt8 = UInt8 = UInt8 = UInt8 = UInt8 = UInt8 = UInt8 = UInt8 = UInt8 = UInt8
Finally, we get the top 3 most likely tags (predicted values) by getTopN method.
Private func getTopN(results: [Float]) -> [Inference] {// Create tuple array [(labelIndex: Int, confidence: Float)]letZippedResults = zip(allelage. indices, results)let sortedResults = zippedResults.sorted { $01 >.The $1.1}. Prefix (resultCount) // Returns the previous resultCount label and predicted valuereturn sortedResults.map { result in Inference.init(confidence: result.1, label: labels[result.0]) }
}
Copy the codeThe logic here is just like the above, we just output the first 10 elements of the one-dimensional array, and then order them to take the maximum three values, the three values of the index directly in the label array to get the corresponding prediction classification.
The last
The blog is just a preview, the detailed logic or need to see the Demo directly
reference
TensorFlow For Poets 2 TensorFlow for Poets 2 TFLite iOS 【 iOS /Android】TensorflowLite Mobile deployment TensorFlow Lite Swift Example TensorFlow Convert PB file to TFLite using python