In order to facilitate readers to understand the one-dimensional convolution application, please eat it together with the author’s practical article:

1.[Deep application]· Baseline of the first China ECG Intelligence Contest (Based on Keras Val_ACC: 0.88)

·DC race Bearing fault detection Baseline (based on Keras1D convolution VAL_ACC :0.99780)

3. github.com/xiaosongshi…

4. “Take you to Learn AI” takes you to learn AI and TensorFlow2 combat introduction: How to fast deep Learning development

An overview of the

Many technical articles have focused on the use of 2D convolutional neural networks (2D CNN), especially in image recognition. However, one-dimensional convolutional neural networks (1D CNNs) are only involved to a certain extent, such as the application in natural language processing (NLP). There are few articles about how to construct a one-dimensional convolutional neural network to solve some of the machine learning problems you might be facing.

This paper tries to fill such a short board.

When to apply 1D CNN?

CNN can do a good job of identifying simple patterns in the data and then using these simple patterns to generate more complex patterns at a higher level. 1D CNN is very effective when you want to get a feature of interest from a short (fixed-length) piece of the overall data set, and the location of the feature in that piece of data is not highly correlated.

1D CNN can be well applied to time series analysis of sensor data (such as gyroscope or accelerometer data). It is also good for analyzing signal data with fixed length periods (such as audio signals). In addition, it can be applied to natural language processing tasks (since word proximity may not always be a good indicator of trainable patterns, the application of LSTM networks in NLP is more promising).

What’s the difference between 1D CNN and 2D CNN?

Whether one-dimensional, two-dimensional or three-dimensional, convolutional neural networks (CNNs) have the same characteristics and the same processing methods. The key difference lies in the dimension of the input data and how the feature detector (or filter) slides between the data:

Problem description

In this article, we will focus on the processing of time-slice based accelerometer data from the user’s belt-style smartphone device. Based on accelerometer data from the X, Y, and Z axes, 1D CNN is used to predict the type of activity the user is doing (such as “walking,” “jogging,” or “standing”). You can find more information on my other two articles here and here. The data looks similar at each time interval for various activities.

Sample time series from accelerometer data

How to construct a 1D CNN in PYTHON?

There are many standard CNN models available. I took a model described on the Keras website and fine-tuned it to fit the problems described earlier. The following image provides a high-level overview of the model built. Each of these layers will be further explained.

Let’s first look at the corresponding Python code to build this model:

model_m.add(Reshape((TIME_PERIODS, num_sensors), input_shape=(input_shape,))) model_m.add(Conv1D(100, 10, activation='relu', input_shape=(TIME_PERIODS, num_sensors))) model_m.add(Conv1D(100, 10, activation='relu')) model_m.add(MaxPooling1D(3)) model_m.add(Conv1D(160, 10, activation='relu')) model_m.add(Conv1D(160, 10, Activation = 'relu) model_m. Add (GlobalAveragePooling1D ()) model_m. Add (Dropout (0.5)) model_m. Add (Dense (num_classes, activation='softmax'))Copy the code

Running this code yields the following deep neural network:

_________________________________________________________________ Layer (type) Output Shape Param ================================================================= reshape_45 (Reshape) (None, 80, 3) 0 _________________________________________________________________ conv1d_145 (Conv1D) (None, 71, 100) 3100 _________________________________________________________________ conv1d_146 (Conv1D) (None, 62, 100) 100100 _________________________________________________________________ max_pooling1d_39 (MaxPooling (None, 20, 100) 0 _________________________________________________________________ conv1d_147 (Conv1D) (None, 11, 160) 160160 _________________________________________________________________ conv1d_148 (Conv1D) (None, 2, 160) 256160 _________________________________________________________________ global_average_pooling1d_29 (None, 160) 0 _________________________________________________________________ dropout_29 (Dropout) (None, 160) 0 _________________________________________________________________ dense_29 (Dense) (None, 6) 966 ================================================================= Trainable params: 520486 _________________________________________________________________Copy the code

Let’s dive into each layer to see what’s going on:

  • Input data: after the data is preprocessed, each data record contains 80 time slices (data is recorded at a sampling frequency of 20Hz, so each time interval contains 4 seconds of accelerometer data). At each time interval, three data of the accelerometer’s x, Y, and Z axes are stored. So you get an 80 x 3 matrix. Since I usually use neural networks on iOS, the data has to be spread out to a 240 degree vector and fed into the neural network. The first layer of the network had to be morphed back to its original 80 x 3 shape.

  • The first 1D CNN layer: The first layer defines a filter (also called a feature detector) of height 10 (also called convolution kernel size). Only if a filter is defined can the neural network learn a single feature in the first layer. That might not be enough, so we’ll define 100 filters. So we have trained 100 different features in the first layer of the network. The output of the first neural network layer is a 71 x 100 matrix. Each column of the output matrix contains the weight of a filter. When defining the kernel size and taking into account the length of the input matrix, each filter will contain 71 weight values.

  • Second 1D CNN layer: The output of the first CNN will be input into the second CNN layer. We will again define 100 different filters on this network layer for training. Following the same logic as the first layer, the output matrix is 62 x 100 in size.

  • Maximum pooling layer: Pooling layer is often used after CNN layer in order to reduce output complexity and prevent data overfitting. In our example, we chose a pooling layer of size 3. This means that the output matrix for this layer is only one-third the size of the input matrix.

  • Third and fourth 1D CNN layers: To learn higher-level features, two additional 1D CNN layers are used here. The output matrix after these two layers is a 2 x 160 matrix.

  • Average pooling layer: add one more pooling layer to further avoid the occurrence of over-fitting. Instead of taking the maximum, the pooling takes the average of the two weights in the neural network. The size of the output matrix is 1 x 160. Each feature detector has only one weight left in this layer of the neural network.

  • Dropout layer: The Dropout layer randomly assigns zero weights to neurons in the network. Since we chose a ratio of 0.5, 50% of the neurons would be zero-weighted. By doing this, the network becomes less responsive to small changes in data. Therefore, it can further improve the accuracy of invisible data processing. The output of this layer is still a 1 x 160 matrix.

  • Using Softmax activated fully connected layer: The last layer will reduce a vector of length 160 to a vector of length 6 because we have 6 categories to predict (i.e., “jog”, “sit”, “walk”, “stand”, “upstairs”, “downstairs”). The dimensional descent here is done by another matrix multiplication. Softmax is used as the activation function. It forces the sum of all six output values of the neural network to be one. Therefore, the output value will represent the probability of occurrence in each of the six categories.

Train and test the neural network

Here is Python code to train the model in a batch size of 400, where the training set and validation set are split 80/20.

    keras.callbacks.ModelCheckpoint(

        filepath='best_model.{epoch:02d}-{val_loss:.2f}.h5',

        monitor='val_loss', save_best_only=True),

    keras.callbacks.EarlyStopping(monitor='acc', patience=1)

model_m.compile(loss='categorical_crossentropy',

                optimizer='adam', metrics=['accuracy'])

history = model_m.fit(x_train,

                      callbacks=callbacks_list,
Copy the code

The accuracy of the model on training data can reach 97%.

16694/16694 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 16 s 973 us/step - loss: 0.0975 acc: 0.9683 - val_loss: 0.7468 - val_acc: 0.8031 16694/16694 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 17 s 989 us/step - loss: 0.0917 acc: 0.9715 - val_loss: 0.7215 - val_acc: 0.8064 16694/16694 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 17 s 1 ms/step - loss: 0.0877-ACC: 0.9716 - val_loss: 0.7233 - val_ACC: 0.8040 16694/16694 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 17 s 1 ms/step - loss: 0.0659 acc: 0.9802 - val_loss: 0.7064 - val_acc: 0.8347 16694/16694 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 17 s 1 ms/step - loss: 0.0626 acc: 0.9799 - VAL_loss: 0.7219 - val_ACC: 0.8107Copy the code

Based on the test set data, the accuracy rate is 92%.

Accuracy on test data: 0.92
Copy the code

conclusion

This paper introduces how to use 1D CNN to train the network by taking the accelerometer data of smart phone to predict the user’s behavior as an example.

Additional reading

1.[Deep application]· Baseline of the first China ECG Intelligence Contest (Based on Keras Val_ACC: 0.88)

·DC race Bearing fault detection Baseline (based on Keras1D convolution VAL_ACC :0.99780)

3. github.com/xiaosongshi…

Welcome to Fork+Star ><

Based on the sharing of theoretical learning and application development technology of deep learning, the author will often share the dry contents of deep learning. When learning or applying deep learning, you can also communicate with me on this page if you have any questions.