In this Keras tutorial, you’ll discover how easy it is to get started with deep learning and Python. You will use the Keras deep learning library to train your first neural network on a custom image data set, and you will also implement your first convolutional neural network (CNN).

The inspiration for this guide came from PyImageSearch reader Igor, who emailed me a few weeks ago and asked:

Hey, Adrian, thanks
PyImageSearchWonderful blog on. I have found that almost every Keras or image classification “how-to guide” uses MNIST or CIFAR-10 datasets built into Keras. I just call one of these functions and the data loads automatically.

But how do I use my image data set on Keras? What steps must I take?



Igor is right — most of the Keras tutorials you encounter attempt to use image classification datasets such as MNIST (handwriting recognition) or CIFAR-10 (Basic object recognition) to teach you the basics of the library.

These image datasets are the standard benchmarks in the computer vision and deep learning literature and, of course, they will definitely get you started with Keras…

. But they’re not necessarily practical, because they don’t teach you how to work with a set of images on disk. Instead, you simply call helper functions to load the precompiled dataset.

I’m going to bring you a new and different Keras tutorial.

I’m going to teach you how to take advantage of these pre-compiled data sets, rather than teach you how to use custom data sets to train your first neural networks and convolutional neural networks, to be honest, your goal is to apply deep learning to your own data sets, not the built-in Keras, am I right?

Learn how to get started with Keras, deep learning, and Python. Read on!

The introduction

Today’s Keras tutorial is designed for practitioners — this is how practitioners apply deep learning.

This meant that we learned by doing, wrote some Keras code by hand, and then trained our network on our custom data set.

This tutorial is not meant to be an in-depth study of theories surrounding deep learning.

If you want to delve deeper into deep learning, including hands-on implementations and theoretical discussions, I suggest you check out my book deep Learning for Computer Vision with Python.

Overview: What does it include

It doesn’t take a lot of code to train the first simple neural network with Keras, but we’ll start slowly, step-by-step, making sure you understand how to train the network on your own custom data set.

The steps we will discuss today include:

  1. Install Keras and other dependencies on your system
  2. Load data from disk
  3. Create training and test branches
  4. Define your Keras model architecture
  5. Compile your Keras model
  6. Train your training data model
  7. Evaluate the model on test data
  8. Predictions were made using trained Keras models

I’ve also added a section to train your first convolutional neural network.

This may seem like a lot of steps, but I promise you that once we start getting into the examples, you’ll see that the examples are linear, make intuitive sense, and will help you understand the fundamentals of training neural networks with Keras.

Our sample data set

Figure 1: We will not use CiFoE10 or MNIST for our data set in this KARAS tutorial. Instead, I’ll show you how to organize your own image data sets and train neural networks using Keras’ deep learning

Most of the Keras tutorials you’ll encounter to solve image classification problems will use MNIST or CIFAR-10 — I’m not going to do that.

First, MNIST and CIFAR-10 are hardly new examples.

These tutorials don’t really cover how to use your own custom image data set. Instead, they simply invoke the built-in Keras utility, which miraculously returns MNIST and CIFAR-10 data sets as Numpy arrays. In fact, your training and testing splits are already pre-marked for you!

Second, if you want to use your own custom data set but really don’t know where to start, you may be frustrated and wondering:

  1. Where does the help function load data from?
  2. What format should the data set on disk be?
  3. How do I load a data set into memory?
  4. What pre-processing steps do I need to perform?

Honestly, your goal in learning Keras and deep learning is not to work with these pre-baked datasets.

Instead, you want to use your own custom data set.

The introductory tutorials you encounter will only take you so far.

That’s why in this Keras tutorial we will use a custom Dataset called “Animals Dataset” that I created for my book deep Learning for Computer Vision with Python:

Figure 2: In this Keras tutorial, we will use a sample animal data set directly from my deep learning book. The dataset consisted of dogs, cats and pandas

The purpose of this data set is to correctly classify images containing:

  • cats
  • dogs
  • panda

Containing only 3000 images, the animal dataset is meant to be an introductory dataset that we can quickly train an in-depth learning model using our CPU or GPU (and still get reasonable accuracy).

Additionally, using this custom data set enables you to understand:

  1. How do I allocate data sets on disk
  2. How do I load images and class labels from disk
  3. How do you divide the data into training and testing branches
  4. How to train the first Keras neural network on training data
  5. How to evaluate the model of test data
  6. How do you reuse your training model with all new data and split it outside of training and testing

By following the steps in this Keras tutorial, you will be able to replace my animal dataset with any dataset, provided you can use the project/directory structure detailed below.

Need data? If you need to scour the web for Images to create a dataset, look at the Easy Way with Bing Image Search, or slightly More involved way with Google Images.

The project structure

There are many files associated with this project. Grab the zip from the download section, then use the tree command to display the project structure in the terminal (I’ve provided two command-line parameter flags for tree to make the output nice and clean) :

$tree -- DirsFirst -- Filelimit 10. ├─ Animals │ ├─ cats [1000 entries entries exceed Filelimit, Not opening Dir] │ ├─ dogs [1000 entries entries exceeds filelimit, Not opening dir] │ ├ ─ panda [1000 entries exceed filelimit, Not opening dir] ├ ─ ─ images │ ├ ─ ─ the JPG │ ├ ─ ─ t JPG │ └ ─ ─ please JPG ├ ─ ─ the output │ ├ ─ ─ simple_nn. Model │ ├ ─ ─ │ ├─ Bass School. Model │ ├─ Bass School. │ ├─ Bass School Smallvggnet_plot. PNG ├ ─ ─ pyimagesearch │ ├ ─ ─ just set py │ └ ─ ─ smallvggnet. Py ├ ─ ─ predict. Py ├ ─ ─ train_simple_nn. Py └ ─ ─ train_vgg.py 7 directories, 14 filesCopy the code

As mentioned earlier, today we will be using animal data sets. Notice how animals are assigned in the project tree. Inside Animals, there are three classes: cat/, dogs/, and Panda /. Each of these directories is 1000 images corresponding to the corresponding class.

If you use your own data set, allocate it the same way! Ideally, you should collect at least 1000 images per category. It’s not always possible, but you should at least have category balance. Having too many images in a class folder can lead to model bias.

Next up is the image/ directory. This directory contains three images for test purposes, which we will use to demonstrate how to load trained models from disk and then classify input images that are not part of the original data set.

The output/ folder contains three types of files that are generated by training:

  • .model : Generates a serialized Keras model file after training, which can be used in future inference scripts.
  • .packle : serialized label binarization file. This file contains an object containing the class name. It comes with a model file.
  • .png: I always put my training/validation drawing image in the output folder because it is the output of the training process.

The Pyimagesearch directory is a module. Contrary to many of the questions I receive, Pyimagesearch is not an installable PIP package. Instead, it resides in the project folder, where the classes contained can be imported into your script. It is available in the downloads section of the Keras tutorial.

Today we’ll review four.py files:

  1. In the first half of this blog post, we will train a simple model. The training script istrain_simple_nn.py.
  2. We’re going to use train_vgg.pyThe script trains SmallVGGNET.
  3. smallvggnet.py The file contains our SmallVGGNET class and a convolutional neural network.
  4. What good is the serialization model unless we can deploy it? in predict.py, I provided you with a sample code to load a serialized model + label file and reason about the image. Predictive scripts are useful and we have successfully trained a model with reasonable accuracy. It is always useful to run this script to test images that are not included in the dataset.

1. Install Keras on the system

Figure 3: We will use Kras and the TysFLOW back end to introduce Kras’s deep learning blog post

For today’s tutorial, you’ll need to install Keras, TensorFlow, and OpenCV.

If you don’t have the software on your system, don’t go overboard yet! I’ve written some easy-to-follow installation instructions. I also update them regularly. Here’s what you need:

  • OpenCV Installation Guide – This launcher links to tutorials to help you install OpenCV on Ubuntu, MacOS, or Raspberry Pi.
  • Install Keras and TensorFlow- You’ll be running with Keras and TensorFlow in less than two minutes, thankspip. You can install these packages on Raspberry Pi, but I don’t recommend using your Pi for training. Pre-trained and appropriately sized models (like the two we introduced today) can easily run on Pi, but be sure to train them first!
  • The installationimutil,scikit-learningandmatplotlibMake sure these packages are installed as well (ideally in a virtual environment). The installation pipIt is easy to:
$ workon <your_env_name> # optional
$ pip install --upgrade imutils
$ pip install --upgrade scikit-learn
$ pip install --upgrade matplotlib
Copy the code

2. Load data from the disk

Figure 4: Step 2 of our Keras tutorial is to load the image from disk into memory

Now that we have Keras installed on our system, we are ready to use Keras to implement our first simple neural network training script. We’ll implement a full convolutional neural network later, but let’s get started.

Open train_simple_nn.py and insert the following code:

# set the matplotlib backend so figures can be saved in the background
import matplotlib
matplotlib.use("Agg")
 
# import the necessary packages
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from keras.models import Sequential
from keras.layers.core import Dense
from keras.optimizers import SGD
from imutils import paths
import matplotlib.pyplot as plt
import numpy as np
import argparse
import random
import pickle
import cv2
import os
Copy the code

Lines 2-19 import the packages we need. As you can see, this script has a lot of tools at work. Let’s review the important things:

  1. matplotlib This is the Python drawing package. That said, it does have its nuances, and if you run into problems, seeThis blog post. In line 3, we indicate matplotlibuse“agg”The back end enables us to save the drawing to disk — your first nuance!
  2. sk learnThe sciKit-Learning library will help us binarize tags, split data for training/testing, and generate training reports in our terminals.
  3. Keras: You’re reading this tutorial to learn about KerAS — it’s our advanced front end to TensorFlow and other deep learning backends.
  4. imutils: my set of function extensions. We will use the path module to generate a list of image file paths for training.
  5. Numpy: Numeric processing in Python. This is another way of packaging. If you have installed OpenCV for Python and Skiti-Learn, then you have Numpy because it is a dependency.
  6. cv2: OpenCV. Usually use 2, but you’ll most likely use OpenCV 3 or higher.
  7. The rest of the imports are built into the Python installation!

Well, that’s a lot, but as we walk through these scripts, it helps to have an idea of what each import is used for.

Let’s parse command-line arguments with the argparse module:

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-d", "--dataset", required=True,
	help="path to input dataset of images")
ap.add_argument("-m", "--model", required=True,
	help="path to output trained model")
ap.add_argument("-l", "--label-bin", required=True,
	help="path to output label binarizer")
ap.add_argument("-p", "--plot", required=True,
	help="path to output accuracy/loss plot")
args = vars(ap.parse_args())
Copy the code

Our script dynamically processes the additional information provided from the command line when executing the script. Additional information comes in the form of command-line arguments. The Argparse module is built into Python and will handle parsing the information you provide in the command string. For further explanation, see this blog post.

We have four command-line argument parses:

  • dataset: Path to the image data set on disk.
  • model : Our model will be serialized and output to disk. This parameter contains the path to the output model file.
  • label-binThe: Dataset tag is serialized to disk for easy traceback in other scripts. This is the path to the output label binarizer file.
  • plot: Output the path to the training drawing image file. We will examine this graph to see if the data is over/under fit.

Using the dataset information, let’s load our image and class tags:

# initialize the data and labels print("[INFO] loading images..." ) data = [] labels = [] # grab the image paths and randomly shuffle them imagePaths = sorted(list(paths.list_images(args["dataset"]))) random.seed(42) random.shuffle(imagePaths) # loop over the input images  for imagePath in imagePaths: # load the image, resize the image to be 32x32 pixels (ignoring # aspect ratio), flatten the image into 32x32x3=3072 pixel image # into a list, and store the image in the data list image = cv2.imread(imagePath) image = cv2.resize(image, (32, 32)).flatten() data.append(image) # extract the class label from the image path and update the # labels list label = imagePath.split(os.path.sep)[-2] labels.append(label)Copy the code

Here we are:

  • Initialize our data 和 label“(lines 35 and 36). These will then become Numpy arrays.
  • grab imagepathsAnd shuffle the cards randomly (lines 31-41). In sorting, and shuffleBefore, paths.list_image Function will be easily found–datasetAll paths to all input images in the directory. Set up the seedSo that random reordering is repeatable.
  • Start looping through all of the data set (line 44) imagepaths .

For each image path, we proceed with:

  • will imageLoad into memory (line 48).
  • willimageAdjusted for32 x 32Pixels (ignore aspect ratio) and make the imageflatten (Line 49). properlyresize Our image is crucial, because this neural network needs these dimensions. Each neural network requires different dimensions, so be aware of this. Flattening data allows us to easily transfer raw pixel intensity to input layer neurons. As you will see later, for VGGNET, we pass the volume to the network, starting with the convolution. Keep in mind that this example is just a simple unconvolutional network – we’ll see a more advanced example later in the blog.
  • Appends the resized image todata(Line 50).
  • A class that extracts images from a pathlabels(Line 54) and add it to labelsList (line 55).labels The list contains classes corresponding to each image in the data list.

Now we can apply array manipulation to data and labels:

# scale the raw pixel intensities to the range [0, 1] data = np.array(data, Dtype ="float") / 255.0 labels = Np.array (labels)Copy the code

In line 58, we scale pixel intensity from the range [0, 255] to [0, 1] (a common preprocessing step).

We also convert the Labels list to a Numpy array (line 59).

3. Build your training and testing branch

Figure 5: Before assembling a deep learning or machine learning model, you must split your data into training and test sets. Use SciKit-learn in this blog post to split our data

Now that we have loaded the image data from disk, we need to build our training and test segmentation:

# partition the data into training and testing splits using 75% of
# the data for training and the remaining 25% for testing
(trainX, testX, trainY, testY) = train_test_split(data,
	labels, test_size=0.25, random_state=42)
Copy the code

Typically, a portion of the data is allocated for training and a smaller portion is tested. Scikit-learn provides a handy train_test_split function that will split the data for us.

TrainX and testX are both image data themselves, while trainY and testY constitute labels.

Our class tag is currently represented as a string, however, Keras will assume both:

  • The tag is encoded as an integer.
  • In addition, a heat coding is performed on these labels so that each label is represented as a vector instead of an integer.

To do this, we can use the LabelBinarizer class:

# convert the labels from integers to vectors (for 2-class, binary
# classification you should use Keras' to_categorical function
# instead as the scikit-learn's LabelBinarizer will not return a
# vector)
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)
Copy the code

At line 70, we initialize the LabelBinarizer object.

Call fit_Transform to find all the unique class tags in trainY and convert them to a hot coded label.

The call to.transform in testY requires only a hot coding step — the unique set of possible class tags is already determined by calling FIT_transform.

Here’s an example:

[1, 0, 0] # corresponds to cats
[0, 1, 0] # corresponds to dogs
[0, 0, 1] # corresponds to panda
Copy the code

Note that only one of the array elements is “hot”, which is why we call it the “one hot” encoding.

4. Define your Keras model architecture

The next step is to define our neural network architecture using Keras. Here, we will use a network with one input layer, two hidden layers, and one output layer:

# define the 3072-1024-512-3 architecture using Keras
model = Sequential()
model.add(Dense(1024, input_shape=(3072,), activation="sigmoid"))
model.add(Dense(512, activation="sigmoid"))
model.add(Dense(len(lb.classes_), activation="softmax"))
Copy the code

Since our model is very simple, we continue to define it in this script (usually I like to create separate classes in a separate file of the model architecture).

The input layer and the first hidden layer are defined on line 76. There will be 3072 input_shape because there are 32×32×3 = 3072 pixels in the translation input image. The first hidden layer will have 1024 nodes.

The second hidden layer will have 512 nodes (line 77).

Finally, the number of nodes in the final output layer (line 78) will be the number of possible class tags — in this case, the output layer will have three nodes, each for our class tags (” cats “, “dogs”, and “panda”).

5. Compile your KRAS model

Figure 7: Step 5 of our KARAS tutorial requires us to compile the model with optimizer and loss functions

Once we have defined our neural network architecture, the next step is to “compile” it:

# initialize our initial learning rate and # of epochs to train for INIT_LR = 0.01 epochs = 75 # compile the model using  SGD as our optimizer and categorical # cross-entropy loss (you'll want to use binary_crossentropy # for 2-class classification) print("[INFO] training network..." ) opt = SGD(lr=INIT_LR) model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])Copy the code

First, we initialize our learning rate and total number of calendar elements to train (lines 81 and 82).

Then, taking “categorical_crossentropy” as the loss function, random gradient descent (SGD) optimizer is used to compile the model.

Classification cross entropy is used as a loss for almost all networks trained to perform classification. The only exception is the 2-class classification, where there are only two possible class labels. In this case, you will want to swap “categorical_crossentropy” for “binary_crossentropy”.

To be continued!