One, foreword

Today is the 10th day, we will use LSTM to complete the prediction of stock opening price, and the final R2 can reach 0.74, which is two percentage points higher than the 0.72 of traditional RNN.

My environment:

• Language: Python3.6.5
• Compiler: Jupyter Notebook
• Deep learning environment: TensorFlow2.4.1

From Column: 100 Examples of deep Learning

If you are still a little white, you can have a look at my special column for you: “White entry deep learning”, to help you with zero basic entry deep learning.

Second, what is LSTM

The basic flow of neural network program

LSTM is an advanced version of RNN. If the maximum limit of RNN is to understand a sentence, then the maximum limit of LSTM is to understand a paragraph. Details are as follows:

LSTM, Short for Long Short Term Memory Networks, is a special RNN that can learn long-term dependencies. LSTM was proposed by Hochreiter & Schmidhuber (1997). Many researchers improved and carried forward LSTM through a series of works. LSTM works very well for many problems and is now widely used.

All recurrent neural networks have the form of a chain of repeated neural network modules. In ordinary RNN, the repeating module structure is very simple, and its structure is as follows:

LSTM avoids the problem of long-term dependency. Remember long-term information! LSTM has a complex structure inside. It can select and adjust the transmitted information through the gated state, remember the information that needs to be remembered for a long time, and forget the unimportant information. Its structure is as follows:

Three, preparation

1. Set the GPU

Comment out this part of the code if you are using a CPU.

import tensorflow as tf

gpus = tf.config.list_physical_devices("GPU")

if gpus:
    tf.config.experimental.set_memory_growth(gpus[0].True)  Set GPU memory usage as required
    tf.config.set_visible_devices([gpus[0]],"GPU")
Copy the code

2. Set related parameters

import pandas            as pd
import tensorflow        as tf  
import numpy             as np
import matplotlib.pyplot as plt
# Support Chinese
plt.rcParams['font.sans-serif'] = ['SimHei']  # used to display Chinese labels normally
plt.rcParams['axes.unicode_minus'] = False  # is used to display the minus sign normally

from numpy                 import array
from sklearn               import metrics
from sklearn.preprocessing import MinMaxScaler
from keras.models          import Sequential
from keras.layers          import Dense,LSTM,Bidirectional


# Make sure the results are reproduced as much as possible
from numpy.random          import seed
seed(1)
tf.random.set_seed(1)

# Set parameters
n_timestamp  = 40    # timestamp
n_epochs     = 20    # Number of training rounds
# = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
# Select model:
# 1: Single-layer LSTM
# 2: Multilayer LSTM
#            3: 双向 LSTM
# = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
model_type = 1
Copy the code

3. Load data

data = pd.read_csv('./datasets/SH600519.csv')  # Read the stock file

data
Copy the code

"" the opening price of the first (2426-300=2126) days is the training set, and the opening price of the last 300 days is the test set.
training_set = data.iloc[0:2426 - 300.2:3].values  
test_set     = data.iloc[2426 - 300:, 2:3].values
Copy the code

4. Data preprocessing

1. The normalization

Normalize the data, ranging from 0 to 1
sc  = MinMaxScaler(feature_range=(0.1))
training_set_scaled = sc.fit_transform(training_set)
testing_set_scaled  = sc.transform(test_set) 
Copy the code

2. Timestamp function

# select * from 'X'; N_timestamp +1 Day data is Y.
def data_split(sequence, n_timestamp) :
    X = []
    y = []
    for i in range(len(sequence)):
        end_ix = i + n_timestamp
        
        if end_ix > len(sequence)-1:
            break
            
        seq_x, seq_y = sequence[i:end_ix], sequence[end_ix]
        X.append(seq_x)
        y.append(seq_y)
    return array(X), array(y)

X_train, y_train = data_split(training_set_scaled, n_timestamp)
X_train          = X_train.reshape(X_train.shape[0], X_train.shape[1].1)

X_test, y_test   = data_split(testing_set_scaled, n_timestamp)
X_test           = X_test.reshape(X_test.shape[0], X_test.shape[1].1)
Copy the code

5. Build a model

# Construct LSTM model
if model_type == 1:
    # monolayer LSTM
    model = Sequential()
    model.add(LSTM(units=50, activation='relu',
                   input_shape=(X_train.shape[1].1)))
    model.add(Dense(units=1))
if model_type == 2:
    # multilayer LSTM
    model = Sequential()
    model.add(LSTM(units=50, activation='relu', return_sequences=True,
                   input_shape=(X_train.shape[1].1)))
    model.add(LSTM(units=50, activation='relu'))
    model.add(Dense(1))
if model_type == 3:
    # two-way LSTM
    model = Sequential()
    model.add(Bidirectional(LSTM(50, activation='relu'),
                            input_shape=(X_train.shape[1].1)))
    model.add(Dense(1))
    
model.summary() # Output model structure
Copy the code

WARNING:tensorflow:Layer lstm will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm (LSTM) (None, 50) 10400 _________________________________________________________________ dense (Dense) (None, 1) 51 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = Total params: 10451 Trainable params: 10451 Non - trainable params: 0 _________________________________________________________________Copy the code

Activate the model

# This application only measures the loss value, not the accuracy, so the option of metrics is deleted and only the Loss value will be displayed in each epoch iteration
model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
              loss='mean_squared_error')  # Loss function uses mean square error
Copy the code

7. Training model

history = model.fit(X_train, y_train, 
                    batch_size=64, 
                    epochs=n_epochs, 
                    validation_data=(X_test, y_test), 
                    validation_freq=1)                  Number of epoch intervals to test

model.summary()
Copy the code

Epoch 1/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 5 s 107 ms/step - loss: 0.1049 - val_loss: 0.0569 Epoch 2/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 86 ms/step - loss: 0.0074 - val_loss: 1.1616 Epoch 3/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 83 ms/step - loss: 0.0012 - val_loss: 0.1408 Epoch 4/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 78 ms/step - loss: e-04 val_loss - 5.8758: 0.0421 Epoch 5/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 84 ms/step - loss: e-04 val_loss - 5.3411: 0.0159 Epoch 6/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 81 ms/step - loss: e-04 val_loss - 3.9690: 0.0034 Epoch 7/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 84 ms/step - loss: e-04 val_loss - 4.3521: 0.0032 Epoch 8/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 85 ms/step - loss: e-04 val_loss - 3.8233: 0.0059 Epoch 9/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 81 ms/step - loss: e-04 val_loss - 3.6539: 0.0082 Epoch 10/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 81 ms/step - loss: e-04 val_loss - 3.1790: 0.0141 Epoch 11/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 82 ms/step - loss: e-04 val_loss - 3.5332: 0.0166 Epoch 12/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 86 ms/step - loss: e-04 val_loss - 3.2684: 0.0155 Epoch 13/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 80 ms/step - loss: e-04 val_loss - 2.6495: 0.0149 Epoch 14/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 84 ms/step - loss: e-04 val_loss - 3.1398: 0.0172 Epoch 15/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 80 ms/step - loss: e-04 val_loss - 3.4533: 0.0077 Epoch 16/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 81 ms/step - loss: e-04 val_loss - 2.9621: 0.0082 Epoch 17/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 83 ms/step - loss: e-04 val_loss - 2.2228: 0.0092 Epoch 18/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 86 ms/step - loss: e-04 val_loss - 2.4517: 0.0093 Epoch 19/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 86 ms/step - loss: e-04 val_loss - 2.7179: 0.0053 Epoch 20/20 33/33 [= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - 3 s 82 ms/step - loss: e-04 val_loss - 2.5923:0.0054 the Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm (LSTM) (None, 50) 10400 _________________________________________________________________ dense (Dense) (None, 1) 51 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = Total params: 10451 Trainable params: 10451 Non - trainable params: 0 _________________________________________________________________Copy the code

Visualization of results

1. Draw a Loss diagram

plt.plot(history.history['loss']    , label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.title(Training and Validation Loss by K)
plt.legend()
plt.show()
Copy the code

2. The prediction

predicted_stock_price = model.predict(X_test)                        # Test set input model for prediction
predicted_stock_price = sc.inverse_transform(predicted_stock_price)  Restore the forecast data - reverse normalization from (0,1) to the original range
real_stock_price      = sc.inverse_transform(y_test)Restore real data - reverse normalization from (0,1) to the original range

# Draw a comparison between the real data and the predicted data
plt.plot(real_stock_price, color='red', label='Stock Price')
plt.plot(predicted_stock_price, color='blue', label='Predicted Stock Price')
plt.title('Stock Price Prediction by K)
plt.xlabel('Time')
plt.ylabel('Stock Price')
plt.legend()
plt.show()
Copy the code

(3) assessment

"" MSE: mean square error -----> Predicted value minus true value squared RMSE: root mean square error -----> root mean square error MAE: mean absolute error -----> predicted value minus true value squared calculated mean R2: Decision coefficient, can understand simple model to reflect the goodness of fit of important statistic articles detailing can refer to: https://blog.csdn.net/qq_38251616/article/details/107997435 "" "
MSE   = metrics.mean_squared_error(predicted_stock_price, real_stock_price)
RMSE  = metrics.mean_squared_error(predicted_stock_price, real_stock_price)**0.5
MAE   = metrics.mean_absolute_error(predicted_stock_price, real_stock_price)
R2    = metrics.r2_score(predicted_stock_price, real_stock_price)

print('Mean square error: %.5f' % MSE)
print('Root mean square error: %.5f' % RMSE)
print('Mean absolute error: %.5f' % MAE)
print('R2: %.5f' % R2)
Copy the code

Mean square error: 2688.75170 Root mean square error: 51.85317 Mean absolute error: 44.97829 R2: 0.74036Copy the code

In addition to changing the model, the fitting degree can also be improved by adjusting the parameters. Here is mainly to introduce LSTM, not to make a detailed introduction to the adjustment of parameters.

Please follow, like, and bookmark this article if you find it helpful

---

Finally, I will send you a copy of the data structure brush notes to help you get the offer from BAT and other first-line big factories. It is written by the big guys of Google and Ali. It is very useful for students who have weak algorithms or need to improve (extraction code: 9GO2) :

Google and Ali big guy’s Leetcode brush notes

As well as the 7K+ open source ebooks I organized, there is always one that can help you 💖 (extract code: 4eg0)

7K+ This open source ebook