Abstract: This paper mainly implements a classification learning case through Keras, and introduces the MNIST script recognition data set in detail.

A case study of Keras building a Classification Neural Network and MNIST Digital Images by Eastmount

What is classification learning

1.Classification

Regression problems, which predict a continuous distribution of values, such as the price of a house, the speed of a car, the price of a Pizza, etc. However, when we need to determine whether a picture is a cat or a dog, we can no longer use regression to solve the problem. At this time, we need to divide it into the category that the computer can recognize (cat or dog) through classification learning.

As you can see above, computers generally process things differently from humans, whether it’s sounds, pictures or words, which can only appear in a computer neural network as a zero or a one. The pictures that the neural network sees are actually a bunch of numbers, and the processing of numbers eventually generates another bunch of numbers, which has certain cognitive significance. Through a little processing, the computer can determine whether the picture is a cat or a dog.

Classification, a kind of supervised learning, is an important research field in data mining, machine learning and data science. The classification model is similar to human learning. It obtains an objective function through learning historical data or training set, and then uses the objective function to predict the unknown attributes of the new data set. The classification model mainly includes two steps:

  • Training. Given a data set, each sample contains a set of features and a category of information, and then the classification algorithm is called to train the model.

  • Predictions. The generated model is used to classify and predict the new data set (test set) and judge its classification results.

Checksets are usually used to verify the performance of learning models. The data set is divided into disjoint training set and test set, the training set is used to construct the classification model, and the test set is used to verify how many class tags are correctly classified.

So, what’s the difference between regression and classification?

Classification and regression both belong to supervised learning. The difference between them lies in that regression is used to predict continuous real values. For example, if the house area is given to predict the house price, the returned result is the house price. Classification, on the other hand, is used to predict limited discrete values, such as whether a person has diabetes, returning a “yes” or “no.” That is, it is clear which predefined target class the object belongs to, which is classified if the predefined target class is discrete value, and which is regression if the continuous value.

2.MNIST

MNIST is a handwritten recognition dataset, which is a very classic example of a neural network. MNIST image data set contains a large number of digital handwritten images, as shown in the figure below, which we can try to use for classification experiments.

The MNIST dataset contains annotated information, and the figure above represents the numbers 5, 0, 4 and 1 respectively. This dataset consists of three parts:

  • Training data set: 55,000 samples, mnist.train

  • Test data set: 10,000 samples, mnist.test

  • Validation dataset: 5,000 samples, mnist.validation

Usually, the training data set used to train model, validation data set used for inspection and the correctness of the model is trained and whether fitting, test set is not visible (equivalent to a black box), but our ultimate goal is to make the trained model on the test set of accuracy (here) to achieve the best effect.

As shown in the figure below, the data is read by the computer in this form, such as 28*28=784 pixels, white is 0, black is digital, there are 55,000 pictures in total.

A sample data in the MNIST dataset contains two parts: handwritten image and corresponding label. Here, XS and ys are used to represent pictures and corresponding labels respectively. Both training data set and test data set have XS and ys. Mnist.train. images and mnist.train.labels are used to represent picture data and corresponding label data in training data set.

As shown in the figure below, it represents a picture composed of 2828 pixel point matrix. The number 784 (2828) here, if placed in our neural network, is the size of x input. Its corresponding matrix is shown in the figure below, and the class label is 1.

Ultimately, MNIST’s training data set has a tensor of 55000*784 digits, which is a multidimensional array, with the first dimension representing the index of the image and the second dimension representing the index of the pixels in the image (the pixels in the tensor are between 0 and 1).

The y value here is actually a matrix that has 10 positions, and if it’s a 1, it puts a 1 in position 1 (the second digit) and a 0 everywhere else; If it is 2, it writes 1 in position 2 (the third digit) and 0 elsewhere. In this way, the numbers in different positions are classified, for example, [0,0,0,1,0,0,0,0,0,0,0] is used to represent the number 3, as shown in the figure below.

Mnist.train. labels is a 55000*10 two-dimensional array, as shown in the figure below. It represents 55,000 data points. The first data point y represents 5, the second data point Y represents 0, the third data point Y represents 4, and the fourth data point Y represents 1.

Now that you know what the MNIST dataset consists of and what x and Y mean, let’s start writing Keras!

Keras implements MNIST classification

In this paper, Keras is used to build a classification neural network and retrain MNIST data sets. Where X represents the picture, 28*28, and y corresponds to the label of the image.

The first step is to import the extension package.

import numpy as np from keras.datasets import mnist from keras.utils import np_utils from keras.models import Sequential  from keras.layers import Dense, Activation from keras.optimizers import RMSpropCopy the code

The second step is loading MNIST data and preprocessing.

  • X_train.reshape(X_train.shape[0],-1) / 255

Each pixel point is normalized and converted from 0-255 to 0-1.

  • np_utils.to_categorical(y_train,nb_classes=10)

The up_utils call converts the class label to a value of 10 lengths, and if the number is 3, it is marked as 1 in the corresponding place and 0 elsewhere, i.e. {0,0,0, 0,0,0,0,0}.

Since the MNIST dataset is sample data from Keras or TensorFlow, we only need the following line of code to read the dataset. If the dataset does not exist it will be downloaded online, if the dataset has already been downloaded it will be called directly.

Shape (X_train, y_train), (X_test, X_test) X_train = X_train. Shape [0], 0 0 -1) / 255 # normalize X_test = X_test.reshape(X_test.shape[0], - 1) / 255 # # normalize vector into a class of matrix for converting 5 0 0 0 0 0 1 0 0 0 0 matrix y_train = np_utils. To_categorical (y_train, num_classes=10) y_test = np_utils.to_categorical(y_test, num_classes=10)Copy the code

The third step is to create a neural network layer.

The previous method of creating a neural network layer is to add the neural layer with add() after the definition.

  • model = Sequential()

  • model.add(Dense(output_dim=1,input_dim=1))

Here we take another approach, adding neural layers by list at the time Sequential() is defined. Note also that the neural network excitation function is added and RMSprop is invoked to accelerate the neural network.

  • from keras.layers importDense, Activation

  • from keras.optimizersimport RMSprop

The neural network layer is:

  • The first layer is Dense(32, input_DIM =784), which converts the incoming 784 into 32 outputs

  • This data is loaded with an Activation(‘ RELu ‘) excitation function and converted into non-linear data

  • The second layer is Dense(10), which outputs 10 units. And the Keras definition neural layer defaults its input to the output of the previous layer, which is 32 (omitted)

  • Activation(‘ softmax ‘), an excitation function, is then loaded for classification

    Another way to build your neural net

    Model = Sequential(Dense(32, input_DIM =784), # input 784(28*28) => output 32 Activation(‘relu’), # excitation function conversion into nonlinear data Dense(10), # Activation(‘softmax’) # Activation function calling softmax for classification])

    Another way to define your optimizer

    Rmsprop = RMSprop(LR =0.001, RHO =0.9, Epsilon = 1E-08, Decay =0.0

    We add metrics to get more results you want to see

    Activating neural network

    Model.compile (optimizer = rmsprop, # loss = ‘categorical_crossentropy’, # metrics = [‘accuracy’], # Calculation error or accuracy)

The fourth step, neural network training and prediction.

Print ("Training") model.fit(X_train, y_train, nb_epoch=2, batch_size=32) print("Training") model.fit(X_train, y_train, nb_epoch=2, batch_size=32) accuracy = model.evaluate(X_test, y_test) print("loss:", loss) print("accuracy:", accuracy)Copy the code

Complete code:

# -* -coding: UTF-8 -*- """ Created on Fri Feb 14 16:43:21 2020 @author: Eastmount CSDN YXZ O(∩_∩)O Wuhan Fighting!! """ import numpy as np from keras.datasets import mnist from keras.utils import np_utils from keras.models import Sequential from keras.layers import Dense, Activation from keras. Optimizers import RMSprop # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- load data and the pretreatment of -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # # X shape(60000, 28*28) y shape(10000,) (X_train, y_train) X_train = X_train. Shape [0], 0 0 -1) / 255 # normalize X_test = X_test.reshape(X_test.shape[0], - 1) / 255 # # normalize vector into a class of matrix for converting 5 0 0 0 0 0 1 0 0 0 0 matrix y_train = np_utils. To_categorical (y_train, num_classes=10) y_test = np_utils.to_categorical(y_test, Num_classes = 10) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- to create a neural network layer -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # Another way to build your neural net Model = Sequential(Dense(32, input_DIM =784), # input 784(28*28) => output 32 Activation('relu'), # excitation function conversion into nonlinear data Dense(10), # Output as result Activation for 10 units ('softmax') # excitation function calls softmax for classification]) # Another way to define your Optimizer RMSprop = RMSprop (lr = 0.001, rho = 0.9, e - 08 epsilon = 1. Decaying =0.0) # We add metrics to get more results you want to see # # loss = 'categorical_crossentropy', # loss function metrics = ['accuracy'], # # calculation error or accuracy) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the Training and predicting -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- print (" Training ") model. The fit (X_train, Print ("Testing") loss, accuracy = model. Evaluate (X_test, y_test) print("loss:", loss) print("accuracy:", accuracy)Copy the code

To run the code, the MNIT dataset is downloaded first.

Using TensorFlow backend.
Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz
11493376/11490434 [==============================] - 18s 2us/step
Copy the code

Then output the results of the two training, we can see that the error is decreasing and the accuracy is increasing. The error loss of the final test output is “0.185575” and the accuracy rate is “0.94690”.

If readers want a more intuitive view of our figures, they can define functions and display them.

The complete code at this point looks like this:

# -* -coding: UTF-8 -*- """ Created on Fri Feb 14 16:43:21 2020 @author: Eastmount CSDN YXZ O(∩_∩)O Wuhan Fighting!! """ import numpy as np from keras.datasets import mnist from keras.utils import np_utils from keras.models import Sequential from keras.layers import Dense, Activation from keras.optimizers import RMSprop import matplotlib.pyplot as plt from PIL import Image # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- load data and the pretreatment of -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - # # download MNIST data X shape (28), 60000, 28 * y shape (10000, ) (X_train, y_train), (X_test, Y_test) = mnist. Load_data # () -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- display images -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- def show_mnist(train_image, train_labels): n = 6 m = 6 fig = plt.figure() for i in range(n): for j in range(m): Subplot (n,m, I *n+j+1) index = I *n+j # img_array = train_image[index] img = image.fromarray (img_array) plt.title(train_labels[index]) plt.imshow(img, cmap='Greys') plt.show() show_mnist(X_train, X_train = X_train. Shape [0], 0 0 -1) / 255 # normalize X_test = X_test.reshape(X_test.shape[0], - 1) / 255 # # normalize vector into a class of matrix for converting 5 0 0 0 0 0 1 0 0 0 0 matrix y_train = np_utils. To_categorical (y_train, num_classes=10) y_test = np_utils.to_categorical(y_test, Num_classes = 10) # -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- to create a neural network layer -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- # Another way to build your neural net Model = Sequential(Dense(32, input_DIM =784), # input 784(28*28) => output 32 Activation('relu'), # excitation function conversion into nonlinear data Dense(10), # Output as result Activation for 10 units ('softmax') # excitation function calls softmax for classification]) # Another way to define your Optimizer RMSprop = RMSprop (lr = 0.001, rho = 0.9, e - 08 epsilon = 1. Decaying =0.0) # We add metrics to get more results you want to see # # loss = 'categorical_crossentropy', # loss function metrics = ['accuracy'], # # calculation error or accuracy) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- the Training and predicting -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- print (" Training ") model. The fit (X_train, Print ("Testing") loss, accuracy = model. Evaluate (X_test, y_test) print("loss:", loss) print("accuracy:", accuracy)Copy the code

Click to follow, the first time to learn about Huawei cloud fresh technology ~