It estimates the mean absolute error using the naive-error approach for a continuous predicted-observed dataset.
Usage
MASE(
data = NULL,
obs,
pred,
time = NULL,
naive_step = 1,
oob_mae = NULL,
tidy = FALSE,
na.rm = TRUE
)Arguments
- data
(Required) argument to call an existing data frame containing the data.
- obs
Vector with observed values (numeric).
- pred
Vector with predicted values (numeric).
- time
(Optional) String with the "name" of the vector containing the time variable to sort observations. "Optional" to ensure an appropriate MASE estimation. Default: NULL, assumes observations are already sorted by time.
- naive_step
A positive number specifying how many observed values to recall back in time when computing the naive expectation. Default = 1
- oob_mae
A numeric value indicating the out-of-bag (out-of-sample) MAE. By default, an in-sample MAE is calculated using the naive forecast method. See Hyndman & Koehler (2006). Default : NULL.
- tidy
Logical operator (TRUE/FALSE) to decide the type of return. TRUE returns a data.frame, FALSE returns a list; Default : FALSE.
- na.rm
Logic argument to remove rows with missing values (NA). Default is na.rm = TRUE.
Value
an object of class numeric within a list (if tidy = FALSE) or within a
data frame (if tidy = TRUE).
Details
The MASE is especially well suited for time series predictions. It can be used to compare forecast methods on a single series and also to compare forecast accuracy between series.
This metric never gives an infinite or undefined values (unless all observations over time are exactly the same!).
By default, the MASE scales the error based on in-sample MAE from the naive forecast method (random walk). The in-sample MAE is used in the denominator because it is always available and it effectively scales the errors.Since it is based on absolute error, it is less sensitive to outliers compared to the classic MSE.
\(MASE = \frac{1}{n}(\frac{|O_i-P_i|}{ \frac{1}{T-1} \sum^T_{t=2}~|O_t - O_{t-1}| })\)
If available, users may use and out-of-bag error from an independent dataset, which
can be specified with the oob_mae arg. and will replace the denominator into the MASE
equation.
MASE measures total error (i.e. both lack of accuracy and precision.). The lower the MASE below 1, the better the prediction quality. MASE = indicates no difference between the model and the naive forecast (or oob predictions). MASE > 1 indicate poor performance.
For the formula and more details, see online-documentation
References
Hyndman, R.J., Koehler, A.B. (2006). Another look at measures of forecast accuracy. _ Int. J. Forecast_ doi:10.3354/cr030079
Examples
# \donttest{
set.seed(1)
# Create a fake dataset
X <- rnorm(n = 100, mean = 8, sd = 10)
Y <- rnorm(n = 100, mean = 8.2, sd = 15)
Z <- seq(1, 100, by = 1)
time_data <- as.data.frame(list("observed" = X, "predicted" = Y, "time" = Z))
MASE(data = time_data, obs = observed, pred = predicted, time = time)
#> $MASE
#> [1] 0
#>
# }