Basic knowledge of MXNET and GLUon is required to read this manual. This article uses CPU training code. Zmkwjx.github. IO Github address: github.com/zmkwjx/Gluo… GluonTS official website: Gluon-ts.mxnet. IO

1. Introduction to DeepAR

In GluonTS, DeepAR implemented an RN-based model that uses an autoregressive recursive network for probability prediction, a method for generating accurate probability predictions based on training the autoregressive recursive network model on a large number of correlated time series. The accuracy is about 15 percent better than the latest technology. Probabilistic prediction (that is, estimating the future probability distribution of a time series based on its past) is a key factor in optimizing business processes.

  • Note: The code for this model is independent of the implementation behind SageMaker’s DeepAR prediction algorithm

2. DeepAR input/output

DeepAR supports two data channels. The required train channel describes the training data set. The optional test channel describes the data set that the algorithm uses to evaluate the model’s accuracy after training. You can provide training and test data sets in JSON row format. When specifying a path for training and test data, you can specify a file or a directory containing multiple files that can be stored in subdirectories. If you specify a directory, DeepAR uses all files in the directory as input for the appropriate channel. By default, the DeepAR model uses.json files to input data.

  • A method to load a data set
# This method will be described later
common.FileDataset("Fill in here the absolute path to the training data folder", freq="H")
Copy the code

2.1 Input data field format

  • Start – The value is a string in the format of YYY-MM-DD HH:MM:SS. The start timestamp cannot contain time zone information.
  • Target – represents an array of floating point values or integers for a time series. You can encode the missing value as NULL, or as a “NAN” string in JSON:
{"start": "The 2009-11-01 00:00:00"."target": [5."NAN".7.12]}
Copy the code
  • Feat_dynamic_real (Optional) – An array of floating-point values or integers representing a custom element time series (dynamic element) vector. If this field is set, all records must have the same number of internal arrays (the same number of characteristic time series). In addition, each internal array must have the same length as the associated target value. For example, if the target time series represents requirements for different products, feat_dynamic_real might be a Boolean time series that indicates whether a promotion has been applied to a particular product:
{"start":... ."target": [5."NAN".7.12]."dynamic_feat": [[1.0.0.1]]}
Copy the code
  • Feat_static_cat (Optional) – An array of classified characteristics that can be used to encode the groups that records belong to. The classification element must be encoded as a sequence of positive integers based on 0. For example, the classification field {R, G, B} can be encoded as {0,1,2}. All values from each classification field must be represented in the training data set.

If you are using a JSON file, the file must be in JSON Lines format. Such as:

{"start": "The 2009-11-01 00:00:00"."target": [4.3."NaN".5.1. ] ."feat_static_cat": [0.1]."feat_dynamic_real": [[1.1.1.2.0.5. ] ]} {"start": "The 2012-01-30 00:00:00"."target": [1.0, -5.0. ] ."feat_static_cat": [2.3]."feat_dynamic_real": [[1.1.2.05. ] ]} {"start": "The 1999-01-30 00:00:00"."target": [2.0.1.0]."feat_static_cat": [1.4]."feat_dynamic_real": [[1.3.0.4]]}
Copy the code

In this example, each time series has two associated classification features and one time series feature.

If the algorithm is trained unconditionally, it learns a “global” model that is independent of the specific identity of the target time series in reasoning and is constrained only by its shape.

If the model is conditional on feat_STATIC_CAT and Feat_dynamic_REAL feature data provided for each time series, the prediction is likely to be influenced by time series features with corresponding CAT features. For example, if the Target time series represents a clothing item requirement, you can associate a two-dimensional CAT vector that codes the item type in the first component (for example, 0 = shoes, 1 = dress) and the item color in the second component (for example, 0 = red, 1 = blue). Example input is as follows:

{ "start":... ."target":... ."feat_static_cat": [0.0],... }# red shoes
{ "start":... ."target":... ."feat_static_cat": [1.1],... }# blue dress
Copy the code

When deducing, you can ask to predict a target whose FEAT_STATIC_CAT value is a combination of the FEAT_STATIC_CAT values observed in the training data, for example:

{ "start":... ."target":... ."feat_static_cat": [0.1],... }# blue shoes
{ "start":... ."target":... ."feat_static_cat": [1.0],... }# red dress
Copy the code

2.2 Training data criteria

  • Time series can start at different times and lengths. For example, in marketing, products often enter retail catalogs on different dates, so their start dates are naturally different. However, all series must have the same frequency, number of classification features, and number of dynamic features.
  • The training files are randomly sorted according to the position of time series in the files. In other words, time series appear in a file in random order.
  • Make sure the start field is set correctly. The algorithm uses the start timestamp to derive internal characteristics.
  • If you use classification features (Feat_STATIC_CAT), all time series must have the same number of classification features.
  • If your data set contains the Feat_DYNAMic_REAL field, the algorithm automatically uses that field. All time series must have the same number of characteristic time series. The time point in each characteristic time series corresponds to the time point in the target. Additionally, entries in the FEAT_DYNAMic_REAL field should have the same length as target. If you have used the Feat_dynamic_REAL field to train your model, you must provide this field for reasoning. In addition, each feature must have the length of the supplied target plus prediction_length.

2.3 Load the dataset of JSON Lines files contained in the path

Common.filedataset Dataset of JSON Lines files contained in the loading path.

class gluonts.dataset.common.FileDataset(path: pathlib.Path, freq: str, one_dim_target: bool = True)

  • Path: The path to the file containing the dataset. Every file is considered a training data source, except for files that start with “.” and end with “_SUCCESS”. The effective line in the file can is: {” start “:” 2014-09-07 “, “target” : [0.1, 0.2]}
  • Freq: Frequency of observations in time series
  • One_dim_target: whether to accept only univariate target time series

2.4 Construct DeepAR network

class gluonts.model.deepar.DeepAREstimator(freq: str, prediction_length: int, trainer: Trainer = gluonts.trainer._base.Trainer(batch_size=32, clip_gradient=10.0, CTX =None, epochs=100, Blotting ZE =True, init=” Xavier “, LEARning_rate =0.001, LEARning_rate_decay_factor =0.5, minimum_learning_rate= 5E-05, num_batches_per_epoch=50, patience=10, weight_decay=1e-08), context_length: Optional[int] = None, num_layers: Int = 2, num_cells: int = 40, cell_type: STR = ‘LSTM ‘, dropout_rate: float = 0.1, use_feat_dynamic_real: bool = False, use_feat_static_cat: bool = False, use_feat_static_real: bool = False, cardinality: Optional[List[int]] = None, embedding_dimension: Optional[List[int]] = None, distr_output: gluonts.distribution.distribution_output.DistributionOutput = gluonts.distribution.student_t.StudentTOutput(), scaling: bool = True, lags_seq: Optional[List[int]] = None, time_features: Optional[List[gluonts.time_feature._base.TimeFeature]] = None, num_parallel_samples: int = 100)

  • Freq: Frequency of observations in time series
  • Prediction_length: Indicates the length of the prediction range
  • Context_length: Number of steps to expand for RNN before calculating prediction (default: None, in this case, context_length = projection_length)
  • Num_layers: RNN Number of layers (default: 2)
  • Num_cells: Number of RNN cells per layer (default: 40)
  • Cell_type: type of loop cell to use (available: “LSTM” or “GRu”; Default value: LSTM)
  • Dropout_rate: drop regularization parameter (default: 0.1)
  • Use_feat_dynamic_real: Whether to use fields from feat_dynamic_real data (default: False)
  • Use_feat_static_cat: Whether to use fields in the FEAT_STATIC_CAT data (default: False)
  • Use_feat_static_real: Whether to use fields in the FEAT_STATIC_REAL data (default: False)
  • Cardinality: The number of values for each classification feature. If use_FEAT_STATIC_cat == True, this must be set (default: None)
  • Scaling: Whether to automatically scale the target value (default: True)
  • Lags_seq: Index used as lag target value for RNN input (default: None, in which case these values will be determined automatically based on frequency)
  • Time_features: Time features used as RNN input (default: None, in which case they are automatically determined based on frequency)
  • Num_parallel_samples: Number of evaluated samples for each time series to increase parallelism during reasoning. This is a model optimization without compromising accuracy (default: 100)

2.5 example

train_data = common.FileDataset("Fill in here the absolute path to the training data folder", freq="H")
test_data  = common.FileDataset("Fill in here the absolute path of the data folder you want to predict.", freq="H")

estimator = deepar.DeepAREstimator(
    prediction_length=24,
    context_length=100,
    use_feat_static_cat=True,
    use_feat_dynamic_real=True,
    num_parallel_samples=100,
    cardinality=[2.1],
    freq="H",
    trainer=Trainer(ctx="cpu", epochs=200, learning_rate=1e-3)
)
predictor = estimator.train(training_data=train_data)

for test_entry, forecast in zip(test_data, predictor.predict(test_data)):
    to_pandas(test_entry)[-100:].plot(figsize=(12.5), linewidth=2)
    forecast.plot(color='g', prediction_intervals=[50.0.90.0])
plt.grid(which='both')
plt.legend(["past observations"."median prediction"."90% prediction interval"."50% prediction interval"])
plt.show()

prediction = next(predictor.predict(test_data))
print(prediction.mean)
prediction.plot(output_file='graph.png')
Copy the code