Measure name | Formula | Description | Scaled | Outlier Protection | Other forms | Penalize extreme deviation | Other Specification |
---|---|---|---|---|---|---|---|
Mean Absolute Error (MAE) | \( MAE=\frac {1}{T} \sum _{t=1}^{T} |e_{t}| \) | Demonstrates the magnitude of overall error | No | Not Good | GMAE | No | - |
Root Mean Squared Error (RMSE) | \( RMSE= \sqrt {\frac {\sum _{t=1}^{T} e_{t}^{2}}{T} } \) | Root square of average squared error | No | Not Good | MSE | Yes | - |
Mean Absolute Percentage Error (MAPE) | \( MAPE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{y_{t}}| \) | Measures the average of absolute percentage error | Yes | Not Good | MdAPE a, RMSPE b | No | - |
symmetric Mean Absolute Percentage Error (sMAPE) | \( sMAPE=\frac {2}{T} \sum _{t=1}^{T} |\frac {e_{t}}{y_{t}+x_{t}}| \) | Scale the error by dividing it by the average of y t and x t | Yes | Good | MdsAPE | No | Less possibility of division by zero rather than MAPE. |
Mean Absolute Relative Error (MARE) | \( MARE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{e_{RWt}}| \) | Measures the average ratio of absolute error to Random walk error | Yes | Fair | MdRAE, GMRAE | No | - |
Relative Measures: e.g. RelMAE (RMAE) | \(RMAE=\frac {MAE}{MAE_{RW}}= \frac {\sum _{t=1}^{T} |e_{t}|}{\sum _{t=1}^{T} |e_{RWt|} }\) | Ratio of accumulation of errors to cumulative error of Random Walk method | Yes | Not Good | No | - | |
Mean Absolute Scaled Error (MASE) | \( MASE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{\frac {1}{T-1}\times \sum _{i=2}^{T}|y_{i}-y_{i-1}|}| \) | Measures the average ratio of error to average error of one-step Random Walk method | Yes | Fair | RMSSE | No | - |
Percent Better (PB) | \( PB=\frac {1}{T} \sum _{t=1}^{T} [I\{e_{t},e_{RW_{t}}\}]\) | Demonstrates average number of times that method overcomes the Random Walk method | Yes | Good | - | No | Not good for calibration and close competitive methods. |
 | \( |e_{s,t}|\leq |e_{RW_{t}}| \leftrightarrow I\{e_{t},e_{RW_{t}}\}=1 \) |  |  |  |  |  |  |
Mean Arctangent Absolute Percentage Error (MAAPE) | \( MAAPE=\frac {1}{T} \sum _{t=1}^{T} arctan|\frac {e_{t}}{y_{t}}| \) | Calculates the average arctangent of absolute percentage error | Yes | Good | MdAAPE | No | Smooths large errors. Solve division by zero problem. |
Normalized Mean Squared Error (NMSE) | \( NMSE=\frac {MSE}{\sigma ^{2}} = \frac {1}{\sigma ^{2} T} \sum _{t=1}^{T} e_{t}^{2} \) | Normalized version of MSE: value of error is balanced | No | Not Good | NA | No | Balanced error by dividing by variance of real data. |