directory

Loss Functions

Comparison of loss function loss and accuracy

Loss in regression task mainly includes

Loss Functions

  • The loss function is used to evaluate the difference between the predicted value and the real value of the model. When deep learning trains the model, it is to calculate the loss function and update the model parameters, so as to reduce the optimization error until the loss function value drops to the target value or reaches the training times.
  • Different models generally use different loss functions. The better the loss function is set, the better the performance of the model in general.
  • The Loss function can be customized

Comparison of loss function loss and accuracy

  • In classification problems, accuracy is more intuitive and interpretable. For regression problems, accuracy is not available, only loss can be used
  • Accuracy is not differentiable and cannot be directly used for network training, while the back propagation algorithm requires the loss function to be differentiable
  • Loss function (Loss) can be differentiated, gradient can be calculated, and parameters can be updated by back propagation.

Loss in regression task mainly includes

  • Mean squareerror (MSE) : MSE represents the sum of squares of the difference between the predicted value and the target value and then averages it

  • L2 norm: THE L2 loss represents the sum of the squares of the difference between the predicted value and the target value and then the greater square. L2 represents the Euclidean distance.

  • MSE and L2 have the same curve. The difference is that one is the mean np.mean(), the other is the square np.sqrt().

Tensorflow and Keras code:

Comments #tensorflow tf.losses. Mean_squared_error (Labels, predictions, weights=1.0, scope=None, loss_collection=tf.GraphKeys.LOSSES, reduction=Reduction.SUM_BY_NONZERO_WEIGHTS ) tf.metrics.mean_squared_error( labels, predictions, weights=None, metrics_collections=None, updates_collections=None, name=None ) #keras mean_squared_error(y_true, y_pred)Copy the code
  • Meanabsolute Error (MAE) : MAE represents the absolute value of the difference between the predicted value and the target value and then averages it

  • L1 norm: L1 represents the absolute value of the difference between the predicted value and the target value, also known as the Manhattan distance

The difference between MAE and L1 is that one calculates the np.mean() and the other does not calculate the NP.sum (). The curve trend of the two is also exactly the same.

  • Tensorflow and Keras code:
#tensorflow
tf.metrics.mean_absolute_error(
    labels,
    predictions,
    weights=None,
    metrics_collections=None,
    updates_collections=None,
    name=None
)

#keras
mean_absolute_error(y_true, y_pred)
Copy the code
  • Comparison of MSE and MAE:

MAE loss is more robust for outliers, but its derivative discontinuity makes the process of finding the optimal solution inefficient. MSE loss is sensitive to outposts, but is more stable and accurate during optimization.

  • Other available loss functions refer to:

zhuanlan.zhihu.com/p/58883095

Supplement knowledge

MAE (mean Absolute Error) and RMSE (root Mean Squared Error) are the two most commonly used indexes to measure the accuracy of variables, and they are also two important scales for evaluating models in machine learning.

Definition 1.

MAE (mean Absolute error) is the average of absolute errors, which is actually the more general form of the mean of errors.

or

Root mean squared Error RMSE (root mean squared Error), also known as RMSD, can also measure the average size of the error, which is the square root of the average square difference between the predicted value and the actual observation.

or

 

2. Compare

The root mean square error (RMSE) is more affected by outliers.

3. The application of

When we work with large data sets, we cannot examine each value to see if there is one or more outliers, or if all errors are systematically higher

Solution: Looking at MAE and RMSE ratios can help us understand whether there are large but uncommon errors.

 

References:

【 1 】 zhuanlan.zhihu.com/p/58883095

【 2 】 blog.csdn.net/u014421797/…

【 3 】 blog.csdn.net/qq_14845119…

[4] blog.csdn.net/nanhuaibeia…