TensorFlow Tutorial #09 – Video Data

This article mainly demonstrates how to create a classifier using your own image data.

01 – a simple linear model | 02 – convolution neural network | 03 – PrettyTensor | 04 – save & restore 05-06 – CIFAR integrated learning | | | 07 – Inception 10 model 08 – migration study

By Magnus Erik Hvass Pedersen/GitHub/Videos on YouTube 英 文翻译

If reproduced, please attach a link to this article.

---

introduce

The previous tutorial #08 described how to use a pre-trained Inception model to do Transfer Learning on a CIFAR-10 dataset. This article will show you how to use your own images.

To demonstrate, we use a new dataset called Knifey-Spoony, which contains thousands of images of knives, spoons, and forks in different contexts. There are 4170 images in the training set and 530 images in the test set. Categories knifey, Sppony and Forky are references to the Simpsons.

The images in the Knifey-Spoony dataset are retrieved from video files using a simple Python script that runs on Linux (it requires an AVConv program to convert video into images). This allows you to quickly create data sets containing thousands of images from just a few minutes of video.

This article builds on the previous tutorial. You need to familiarize yourself with transfer learning in tutorial #08, as well as the previous tutorial on how to create and train neural networks in TensorFlow.

The flow chart

The following figure shows the flow of data for transfer learning using the Inception model. First, we enter and process an image in the Inception model. Before the final classification layer of the model, the so-called transfer-values are saved to a cache file.

This is similar to what we did in Tutorial #08, except now we use the Knifey-Spoony dataset instead of CIFAR-10, which means that we send jpeg images to the Inception model instead of using a NUMpy array containing the image data.

After all images in the new data set have been processed with Inception and the resulting transfer-values have been saved to a cache file, we can use these transfer-values as input to other neural networks. The second neural network is then trained with categories in the new data set, so the network learns how to classify images based on transfer-values of the Inception model.

Thus, the Inception model extracts useful information from the image, and then uses additional neural networks to do the actual classification.

from IPython.display import Image, display
Image('images/09_transfer_learning_flowchart.png')Copy the code

The import

%matplotlib inline
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import time
from datetime import timedelta
import os

# Functions and classes for loading and using the Inception model.
import inception

# We use Pretty Tensor to define the new classifier.
import prettytensor as ptCopy the code

Developed using Python3.5.2 (Anaconda), the TensorFlow version is:

tf.__version__Copy the code

'0.12.0 - rc0'Copy the code

PrettyTensor version:

pt.__version__Copy the code

‘0.7.1’

Load the data

import knifeyCopy the code

Data dimensions are already defined in the Knifey module, so we can just import them if we need them.

from knifey import num_classesCopy the code

Set the path to save the data set on your computer.

# knifey.data_dir = "data/knifey-spoony/"Copy the code

Set the folder to save the cached files in this tutorial.

data_dir = knifey.data_dirCopy the code

The Knifey-Spoony dataset is about 22MB and will be downloaded automatically if no files are found in a given path.

knifey.maybe_download_and_extract()Copy the code

Data has apparently already been downloaded and unpacked.

Now load the dataset. The program iterates through the subfolders to get all the *.jpg images, and then places the file names into two lists of training sets and test sets. The image is not actually loaded at this time, and the loading will be performed after transfer-values are calculated.

The list of file names will be saved to hard disk, and we must ensure that they are in the order in which the dataset will be reloaded later. This is important so that we know which images correspond to which transfer-values.

dataset = knifey.load()Copy the code

Creating dataset from the files in: data/knifey-spoony/

• Data loaded from cache-file: data/knifey-spoony/knifey-spoony.pkl

Your data

You can replace the Knifey-Spoony dataset with your own images. You need to create a dataset object in the dataset. Py module. The best way to do this is to use the load_cache() wrapper function, which automatically saves the list of images to the cache file, so you need to make sure that the list order is the same as the transfer-values that will be generated later.

Images for each category need to be organized in their own folders. The documentation in the dataset. Py module has more details.

# This is the code you would run to load your own image-files.
# It has been commented out so it won't run now.

# from dataset import load_cached
# dataset = load_cached(cache_path='my_dataset_cache.pkl', in_dir='my_images/')
# num_classes = dataset.num_classesCopy the code

Training set and test set

Gets the category name.

class_names = dataset.class_names
class_namesCopy the code

[‘forky’, ‘knifey’, ‘spoony’]

Get the test set. It returns the image’s file path, integer category number, and one-Hot encoded array of category numbers, called labels.

image_paths_train, cls_train, labels_train = dataset.get_training_set()Copy the code

Print the first image address to see if it is correct.

image_paths_train[0]Copy the code

‘/home/magnus/development/TensorFlow-Tutorials/data/knifey-spoony/forky/forky-05-0023.jpg’

Get the test set.

image_paths_test, cls_test, labels_test = dataset.get_test_set()Copy the code

Print the first image address to see if it is correct.

image_paths_test[0]Copy the code

‘/home/magnus/development/TensorFlow-Tutorials/data/knifey-spoony/forky/test/forky-test-01-0163.jpg’

The Knifey-Spoony dataset is now loaded with 4700 images and the corresponding labels (categories of images). The dataset is manually divided into two subsets, the training set and the test set.

print("Size of:")
print("- Training-set:\t\t{}".format(len(image_paths_train)))
print("- Test-set:\t\t{}".format(len(image_paths_test)))Copy the code

Size of:

• Training-set: 4170
• Test-set: 530

A help function to draw an image

This function is used to draw nine images in a 3×3 grid and write the real category and the predicted category under each image.

def plot_images(images, cls_true, cls_pred=None, smooth=True):

    assert len(images) == len(cls_true)

    # Create figure with sub-plots.
    fig, axes = plt.subplots(3.3)

    # Adjust vertical spacing.
    if cls_pred is None:
        hspace = 0.3
    else:
        hspace = 0.6
    fig.subplots_adjust(hspace=hspace, wspace=0.3)

    # Interpolation type.
    if smooth:
        interpolation = 'spline16'
    else:
        interpolation = 'nearest'

    for i, ax in enumerate(axes.flat):
        # There may be less than 9 images, ensure it doesn't crash.
        if i < len(images):
            # Plot image.
            ax.imshow(images[i],
                      interpolation=interpolation)

            # Name of the true class.
            cls_true_name = class_names[cls_true[i]]

            # Show true and predicted classes.
            if cls_pred is None:
                xlabel = "True: {0}".format(cls_true_name)
            else:
                # Name of the predicted class.
                cls_pred_name = class_names[cls_pred[i]]

                xlabel = "True: {0}\nPred: {1}".format(cls_true_name, cls_pred_name)

            # Show the classes as the label on the x-axis.
            ax.set_xlabel(xlabel)

        # Remove ticks from the plot.
        ax.set_xticks([])
        ax.set_yticks([])

    # Ensure the plot is shown correctly with multiple plots
    # in a single Notebook cell.
    plt.show()Copy the code

Help function for loading images

The data set does not load the actual image, there is a list of image (address) in the training set and test set respectively. The following helper functions load some image files.

from matplotlib.image import imread

def load_images(image_paths):
    # Load the images from disk.
    images = [imread(path) for path in image_paths]

    # Convert to a numpy array and return it.
    return np.asarray(images)Copy the code

Draw some images to see if the data is correct

# Load the first images from the test-set.
images = load_images(image_paths=image_paths_test[0:9])

# Get the true classes for those images.
cls_true = cls_test[0:9]

# Plot the images and labels using our helper-function above.
plot_images(images=images, cls_true=cls_true, smooth=True)Copy the code

Download Inception model

Download the Inception model from the Web. This is the default folder where you save your data files. The folder is created automatically if it does not exist.

# inception.data_dir = 'inception/'Copy the code

If the Inception model does not exist in the folder, it is downloaded automatically. It is 85 MB.

See tutorial #07 for more details.

inception.maybe_download()Copy the code

Downloading Inception v3 Model ...
Data has apparently already been downloaded and unpacked.Copy the code

Loading Inception model

Load the model to prepare for image classification.

Pay attention to this warning. The program may fail to run in the future.

model = inception.Inception()Copy the code

Calculate Transfer – Values

Import the help function used to get transfer-values from the Inception model.

from inception import transfer_values_cacheCopy the code

Sets the directory for training and test set cache files.

file_path_cache_train = os.path.join(data_dir, 'inception-knifey-train.pkl')
file_path_cache_test = os.path.join(data_dir, 'inception-knifey-test.pkl')Copy the code

print("Processing Inception transfer-values for training-images ...")

# If transfer-values have already been calculated then reload them,
# otherwise calculate them and save them to a cache-file.
transfer_values_train = transfer_values_cache(cache_path=file_path_cache_train,
                                              image_paths=image_paths_train,
                                              model=model)Copy the code

Processing Inception transfer-values for training-images ...
- Data loaded from cache-file: data/knifey-spoony/inception-knifey-train.pklCopy the code

print("Processing Inception transfer-values for test-images ...")

# If transfer-values have already been calculated then reload them,
# otherwise calculate them and save them to a cache-file.
transfer_values_test = transfer_values_cache(cache_path=file_path_cache_test,
                                             image_paths=image_paths_test,
                                             model=model)Copy the code

Processing Inception transfer-values for test-images ...
- Data loaded from cache-file: data/knifey-spoony/inception-knifey-test.pklCopy the code

Check the array size of transfer-Values. There were 4170 images in the training set, and each image had 2048 transfer-values.

transfer_values_train.shapeCopy the code

(4170, 2048)

Again, there were 530 images in the test set, with 2048 transfer-values per image.

transfer_values_test.shapeCopy the code

(530, 2048)

Draw the help function for Transfer-Values

def plot_transfer_values(i):
    print("Input image:")

    # Plot the i'th image from the test-set.
    image = imread(image_paths_test[i])
    plt.imshow(image, interpolation='spline16')
    plt.show()

    print("Transfer-values for the image using Inception model:")

    # Transform the transfer-values into an image.
    img = transfer_values_test[i]
    img = img.reshape((32.64))

    # Plot the image for the transfer-values.
    plt.imshow(img, interpolation='nearest', cmap='Reds')
    plt.show()Copy the code

plot_transfer_values(i=100)Copy the code

Input image:

Transfer-values for the image using Inception model:

plot_transfer_values(i=300)Copy the code

Input image:

Transfer-values for the image using Inception model:

PCA analysis results of Transfer-Values

Principal component analysis (PCA) in SciKit-Learn was used to reduce the dimensions of transfer-Values from 2048 dimensions to 2 dimensions for easy drawing.

from sklearn.decomposition import PCACopy the code

Create a new PCA-Object with the target array dimension set to 2.

pca = PCA(n_components=2)Copy the code

It takes a while to calculate PCA. The data set for this article is not very large, otherwise you would need to pick a small part of the training set to speed up the calculation.

# transfer_values = transfer_values_train[0:3000]
transfer_values = transfer_values_trainCopy the code

Gets the category number of the sample you selected.

# cls = cls_train[0:3000]
cls = cls_trainCopy the code

The protected array has 4170 samples, each with 2048 transfer-values.

transfer_values.shapeCopy the code

(4170, 2048)

PCA was used to reduce transfer-value from 2048 dimensions to 2 dimensions.

transfer_values_reduced = pca.fit_transform(transfer_values)Copy the code

The array now has 4170 samples with two values per sample.

transfer_values_reduced.shapeCopy the code

(4170, 2)

The help function is used to draw transfer-values after dimensionality reduction.

def plot_scatter(values, cls):
    # Create a color-map with a different color for each class.
    import matplotlib.cm as cm
    cmap = cm.rainbow(np.linspace(0.0.1.0, num_classes))

    # Create an index with a random permutation to make a better plot.
    idx = np.random.permutation(len(values))

    # Get the color for each sample.
    colors = cmap[cls[idx]]

    # Extract the x- and y-values.
    x = values[idx, 0]
    y = values[idx, 1]

    # Plot it.
    plt.scatter(x, y, color=colors, alpha=0.5)
    plt.show()Copy the code

Draw transfer-values after dimensionality reduction with PCA. Different categories in the Knifey-Spoony dataset are represented in 3 different colors. The colors have a lot of overlap. This may be because PCA cannot properly separate transfer-values.

plot_scatter(transfer_values_reduced, cls=cls)Copy the code

T-sne analysis results of Transfer-Values

from sklearn.manifold import TSNECopy the code

Another method of dimensionality reduction is t-SNE. Unfortunately, T-SNE is slow, so we first reduced the dimensions from 2048 to 50 using PCA.

pca = PCA(n_components=50)
transfer_values_50d = pca.fit_transform(transfer_values)Copy the code

Create a new T-SNE object to do the final dimension reduction, setting the target dimension to 2.

tsne = TSNE(n_components=2)Copy the code

The final dimension reduction was performed using t-SNE. The t-SNE currently implemented in SciKit-Learn may not be able to handle many samples of data, so the program may crash if you use the entire training set.

transfer_values_reduced = tsne.fit_transform(transfer_values_50d)Copy the code

Ensure that the array has 4170 samples, each with two transfer-values.

transfer_values_reduced.shapeCopy the code

(4170, 2)

Draw the transfer-values reduced to 2d by T-SNE, which has better separation than the above PCA results.

This means that transfer-values obtained from the Inception model seem to contain enough information to classify knifey-Sponny images, but there is still some overlap, indicating that the separation is not perfect.

plot_scatter(transfer_values_reduced, cls=cls)Copy the code

New classifier in TensorFlow

We will create a new neural network in TensorFlow. The network takes transfer-values from the Inception model as input and outputs the predicted categories of knifey-Spoony images.

It is assumed that you are already familiar with how to build neural networks in TensorFlow, otherwise please read tutorial #03.

Placeholder variables

You first need to find the array length of Transfer-Values, which is a variable stored in the Inception model object.

transfer_len = model.transfer_lenCopy the code

Now create a placeholder variable for the input transfer-values into our new network. The variable’s shape is [None, transfer_len]. None means that its input array contains any number of samples, each with 2048 elements, namely transfer_len.

x = tf.placeholder(tf.float32, shape=[None, transfer_len], name='x')Copy the code

Define another placeholder variable for the input image’s real type tag. This is a one-Hot encoded array of 10 elements, each representing One of the possible categories in the dataset.

y_true = tf.placeholder(tf.float32, shape=[None, num_classes], name='y_true')Copy the code

Computes the integer representing the real category. This could also be a placeholder variable.

y_true_cls = tf.argmax(y_true, dimension=1)Copy the code

The neural network

Create a neural network for classification on knifey-Spoony dataset. It takes transfer-values derived from the Inception model as input and stores them in placeholder variable X. Network output prediction category y_pred.

Tutorial #03 has more details on constructing neural networks using Pretty Tensor.

# Wrap the transfer-values as a Pretty Tensor object.
x_pretty = pt.wrap(x)

with pt.defaults_scope(activation_fn=tf.nn.relu):
    y_pred, loss = x_pretty.\
        fully_connected(size=1024, name='layer_fc1').\
        softmax_classifier(num_classes=num_classes, labels=y_true)Copy the code

An optimization method

Create a variable to record the number of current optimization iterations.

global_step = tf.Variable(initial_value=0,
                          name='global_step', trainable=False)Copy the code

Methods for optimizing new neural networks.

optimizer = tf.train.AdamOptimizer(learning_rate=1e-4).minimize(loss, global_step)Copy the code

Classification accuracy

The network output y_pred is an array of three elements. The category number is the index of the largest element in the array.

y_pred_cls = tf.argmax(y_pred, dimension=1)Copy the code

Create a Boolean vector that indicates whether the true category of each image is the same as the predicted category.

correct_prediction = tf.equal(y_pred_cls, y_true_cls)Copy the code

The accuracy of the classification is calculated by converting the Boolean vector type to a floating-point vector, where False becomes 0 and True becomes 1, and averaging the values.

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))Copy the code

Run TensorFlow

Creating TensorFlow sessions (Session)

Once the TensorFlow diagram is created, we need to create a TensorFlow session to run the diagram.

session = tf.Session()Copy the code

Initialize a variable

We need to initialize the weights and biases variables before we can start optimizing them.

session.run(tf.global_variables_initializer())Copy the code

Get the help function for random training Batch

The training set has 4,170 images (and an array to hold Transfer-values). It takes a lot of time to calculate the gradient of the model with these transfer-vlues. Therefore, we use only a small number of transfer-vlues in each iteration of the optimizer.

If running out of memory causes your computer to crash or become slow, you should try to reduce those numbers, but more optimized iterations may be needed in the meantime.

train_batch_size = 64Copy the code

The transfer-Vlues function is used to select a random batch of transfer-Vlues from the training set.

def random_batch(a):
    # Number of images (transfer-values) in the training-set.
    num_images = len(transfer_values_train)

    # Create a random index.
    idx = np.random.choice(num_images,
                           size=train_batch_size,
                           replace=False)

    # Use the random index to select random x and y-values.
    # We use the transfer-values instead of images as x-values.
    x_batch = transfer_values_train[idx]
    y_batch = labels_train[idx]

    return x_batch, y_batchCopy the code

Perform the help function for the optimization iteration

Function is used to perform a number of optimization iterations to gradually improve the network layer variables. In each iteration, a new batch of data is selected from the training set, and TensorFlow performs optimization on these training samples. Progress is printed every 100 iterations.

def optimize(num_iterations):
    # Start-time used for printing time-usage below.
    start_time = time.time()

    for i in range(num_iterations):
        # Get a batch of training examples.
        # x_batch now holds a batch of images (transfer-values) and
        # y_true_batch are the true labels for those images.
        x_batch, y_true_batch = random_batch()

        # Put the batch into a dict with the proper names
        # for placeholder variables in the TensorFlow graph.
        feed_dict_train = {x: x_batch,
                           y_true: y_true_batch}

        # Run the optimizer using this batch of training data.
        # TensorFlow assigns the variables in feed_dict_train
        # to the placeholder variables and then runs the optimizer.
        # We also want to retrieve the global_step counter.
        i_global, _ = session.run([global_step, optimizer],
                                  feed_dict=feed_dict_train)

        # Print status to screen every 100 iterations (and last).
        if (i_global % 100= =0) or (i == num_iterations - 1) :# Calculate the accuracy on the training-batch.
            batch_acc = session.run(accuracy,
                                    feed_dict=feed_dict_train)

            # Print status.
            msg = "Global Step: {0:>6}, Training Batch Accuracy: {1:>6.1%}"
            print(msg.format(i_global, batch_acc))

    # Ending time.
    end_time = time.time()

    # Difference between start and end-times.
    time_dif = end_time - start_time

    # Print the time-usage.
    print("Time usage: " + str(timedelta(seconds=int(round(time_dif)))))Copy the code

Help function to show results

Help function for drawing error samples

The draw () function is used to draw misclassified samples in the test set.

def plot_example_errors(cls_pred, correct):
    # This function is called from print_test_accuracy() below.

    # cls_pred is an array of the predicted class-number for
    # all images in the test-set.

    # correct is a boolean array whether the predicted class
    # is equal to the true class for each image in the test-set.

    # Negate the boolean array.
    incorrect = (correct == False)

    # Get the indices for the incorrectly classified images.
    idx = np.flatnonzero(incorrect)

    # Number of images to select, max 9.
    n = min(len(idx), 9)

    # Randomize and select n indices.
    idx = np.random.choice(idx,
                           size=n,
                           replace=False)

    # Get the predicted classes for those images.
    cls_pred = cls_pred[idx]

    # Get the true classes for those images.
    cls_true = cls_test[idx]

    # Load the corresponding images from the test-set.
    # Note: We cannot do image_paths_test[idx] on lists of strings.
    image_paths = [image_paths_test[i] for i in idx]
    images = load_images(image_paths)

    # Plot the images.
    plot_images(images=images,
                cls_true=cls_true,
                cls_pred=cls_pred)Copy the code

Help function to draw the confusion matrix

# Import a function from sklearn to calculate the confusion-matrix.
from sklearn.metrics import confusion_matrix

def plot_confusion_matrix(cls_pred):
    # This is called from print_test_accuracy() below.

    # cls_pred is an array of the predicted class-number for
    # all images in the test-set.

    # Get the confusion matrix using sklearn.
    cm = confusion_matrix(y_true=cls_test,  # True class for test-set.
                          y_pred=cls_pred)  # Predicted class.

    # Print the confusion matrix as text.
    for i in range(num_classes):
        # Append the class-name to each line.
        class_name = "({}) {}".format(i, class_names[i])
        print(cm[i, :], class_name)

    # Print the class-numbers for easy reference.
    class_numbers = ["({0})".format(i) for i in range(num_classes)]
    print("".join(class_numbers))Copy the code

Compute the help function for classification

This function computes the predicted categories of images and returns a Boolean array representing whether each image is correctly classified.

Because the calculations may consume too much memory, batch them. If your computer crashes, try lowering batch-size.

# Split the data-set in batches of this size to limit RAM usage.
batch_size = 256

def predict_cls(transfer_values, labels, cls_true):
    # Number of images.
    num_images = len(transfer_values)

    # Allocate an array for the predicted classes which
    # will be calculated in batches and filled into this array.
    cls_pred = np.zeros(shape=num_images, dtype=np.int)

    # Now calculate the predicted classes for the batches.
    # We will just iterate through all the batches.
    # There might be a more clever and Pythonic way of doing this.

    # The starting index for the next batch is denoted i.
    i = 0

    while i < num_images:
        # The ending index for the next batch is denoted j.
        j = min(i + batch_size, num_images)

        # Create a feed-dict with the images and labels
        # between index i and j.
        feed_dict = {x: transfer_values[i:j],
                     y_true: labels[i:j]}

        # Calculate the predicted class using TensorFlow.
        cls_pred[i:j] = session.run(y_pred_cls, feed_dict=feed_dict)

        # Set the start-index for the next batch to the
        # end-index of the current batch.
        i = j

    # Create a boolean array whether each image is correctly classified.
    correct = (cls_true == cls_pred)

    return correct, cls_predCopy the code

Calculates the prediction categories on the test set.

def predict_cls_test(a):
    return predict_cls(transfer_values = transfer_values_test,
                       labels = labels_test,
                       cls_true = cls_test)Copy the code

Help function to calculate classification accuracy

This function calculates the classification accuracy of a given Boolean array indicating whether each image is correctly classified. For example, cls_accuracy([True, True, False, False, False]) = 2/5 = 0.4.

def classification_accuracy(correct):
    # When averaging a boolean array, False means 0 and True means 1.
    # So we are calculating: number of True / len(correct) which is
    # the same as the classification accuracy.

    # Return the classification accuracy
    # and the number of correct classifications.
    return correct.mean(), correct.sum()Copy the code

Show the help function for classification accuracy

The class accuracy () function prints the classification accuracy on the test set.

It will take a while to calculate the classification for all the images on the test set, so we call the above functions directly from this function so that we don’t have to recalculate the classification for each function.

def print_test_accuracy(show_example_errors=False, show_confusion_matrix=False):

    # For all the images in the test-set,
    # calculate the predicted classes and whether they are correct.
    correct, cls_pred = predict_cls_test()

    # Classification accuracy and the number of correct classifications.
    acc, num_correct = classification_accuracy(correct)

    # Number of images being classified.
    num_images = len(correct)

    # Print the accuracy.
    msg = "Accuracy on Test-Set: {0:.1%} ({1} / {2})"
    print(msg.format(acc, num_correct, num_images))

    # Plot some examples of mis-classifications, if desired.
    if show_example_errors:
        print("Example errors:")
        plot_example_errors(cls_pred=cls_pred, correct=correct)

    # Plot the confusion matrix, if desired.
    if show_confusion_matrix:
        print("Confusion Matrix:")
        plot_confusion_matrix(cls_pred=cls_pred)Copy the code

The results of

Optimize previous performance

The accuracy on the test set is very low because the model is only initialized, not optimized, so it just randomly classifies the images.

print_test_accuracy(show_example_errors=False,
                    show_confusion_matrix=True)Copy the code

Accuracy on test-set: 30.0% (159/530) Confusion Matrix: [151 0 0] (0) forky [122 3 12] (1) knifey [237 0 5] (2) spoony (0) (1) (2)

Performance after 1000 optimization iterations

After 1000 iterations of optimization, the accuracy on the test set was about 70%.

optimize(num_iterations=1000)Copy the code

Global Step: 100, Training Batch Accuracy: 95.3% Global Step: 200, Training Batch Accuracy: 96.9% 300, Training Batch Accuracy: 98.4% Global Step: 400, Training Batch Accuracy: 100.0% Global Step: 500, Training Batch Accuracy: 100.0% Global Step: 600, Training Batch Accuracy: 100.0% Global Step: 700, Training Batch Accuracy: 100.0% Global Step: 800, Training Batch Accuracy: 100.0% Global Step: 900, Training Batch Accuracy: 100.0% Global Step: 1000, Training Batch Accuracy: 100.0% Time usage: 0:00:02

print_test_accuracy(show_example_errors=True,
                    show_confusion_matrix=True)Copy the code

Accuracy on test-set: 71.3% (378/530) Example errors:

Confusion Matrix:

[35 36 80] (0) forky

[ 6 101 30] (1) knifey

[ 0 0 242] (2) spoony

(0) (1) (2)

Close the TensorFlow session

We have now completed the task with TensorFlow, closing the session and freeing resources. Note that we need to close two Tensorflow-Sessions, one for each model object.

# This has been commented out in case you want to modify and experiment
# with the Notebook without having to restart it.
# model.close()
# session.close()Copy the code

conclusion

This tutorial shows us how to do transfer learning with your own images on the Inception model. The thousands of images used in the tutorial were generated using a Python script from a few minutes of video footage.

However, the classification accuracy on the Knifey-Spoony dataset is not very high, especially for the fork image. Probably because Inception model is trained on ImageNet data set with only 16 fork images, but it includes over 1200 spoon images and over 1300 table knife images. Thus Inception models are likely to fail to correctly identify forks.

So we need another technique to fine-tune the Inception model so that it can better recognize forks.

practice

Here are some suggested exercises that might help you improve your TensorFlow skills. In order to learn how to use TensorFlow more appropriately, practical experience is important.

Before you can make changes to this Notebook, you may want to make a backup.

• Try changing the neural network for the new classifier. What happens if you remove full connection layers or add more full connection layers?
• What happens if you do more or fewer iterations?
• Try deleting some images of spoons from the training set so that there are about the same number of images per category (make a backup first). You also need to remove all file names with them*.pklAnd rerun the Notebook. Will this improve classification accuracy? Compare the confusion matrix before and after the change.
• withconvert.pyThe script builds your own data set. For example, record videos of cars and motorcycles and create a classification system.
• Need to remove any ambiguous images from the training set you created? How does the classification accuracy change after you delete these images?
• Change the Notebook so that you can enter a single image instead of the entire dataset. You do not need to save transfer-values from the Inception model.
• Can you create a better or faster neural network than using the Inception model for transfer learning?
• Explain to a friend how the program works.