## 9.7 ARIMA modelling in R

### How does ARIMA() work?

The ARIMA() function in R uses a variation of the Hyndman-Khandakar algorithm , which combines unit root tests, minimisation of the AICc and MLE to obtain an ARIMA model. The arguments to ARIMA() provide for many variations on the algorithm. What is described here is the default behaviour.

Hyndman-Khandakar algorithm for automatic ARIMA modelling
1. The number of differences $$0 \le d\le 2$$ is determined using repeated KPSS tests.
1. The values of $$p$$ and $$q$$ are then chosen by minimising the AICc after differencing the data $$d$$ times. Rather than considering every possible combination of $$p$$ and $$q$$, the algorithm uses a stepwise search to traverse the model space.
1. Four initial models are fitted:
• ARIMA$$(0,d,0)$$,
• ARIMA$$(2,d,2)$$,
• ARIMA$$(1,d,0)$$,
• ARIMA$$(0,d,1)$$.
A constant is included unless $$d=2$$. If $$d \le 1$$, an additional model is also fitted:
• ARIMA$$(0,d,0)$$ without a constant.
1. The best model (with the smallest AICc value) fitted in step (a) is set to be the “current model.”
1. Variations on the current model are considered:
• vary $$p$$ and/or $$q$$ from the current model by $$\pm1$$;
• include/exclude $$c$$ from the current model.
The best model considered so far (either the current model or one of these variations) becomes the new current model.
1. Repeat Step 2(c) until no lower AICc can be found.

Figure 9.11 illustrates diagrammatically how the Hyndman-Khandakar algorithm traverses the space of the ARMA orders, through an example. The grid covers combinations of ARMA($$p,q$$) orders starting from the top-left corner with an ARMA($$0,0$$), with the AR order increasing down the vertical axis, and the MA order increasing across the horizontal axis. Figure 9.11: An illustrative example of the Hyndman-Khandakhar stepwise search process

The orange cells show the initial set of models considered by the algorithm. In this example, the ARMA(2,2) model has the lowest AICc value amongst these models. This is called the “current model” and is shown by the black circle. The algorithm then searches over neighbouring models as shown by the blue arrows. If a better model is found then this becomes the new “current model.” In this example, the new “current model” is the ARMA(3,3) model. The algorithm continues in this fashion until no better model can be found. In this example the model returned is an ARMA(4,2) model.

The default procedure uses some approximations to speed up the search. These approximations can be avoided with the argument approximation=FALSE. It is possible that the minimum AICc model will not be found due to these approximations, or because of the use of a stepwise procedure. A much larger set of models will be searched if the argument stepwise=FALSE is used. See the help file for a full description of the arguments.

If you want to choose the model yourself, use the ARIMA() function with a single value for input for pdq() and PDQ(). There is another function arima() in R which also fits an ARIMA model. it does not allow for the constant $$c$$ unless $$d=0$$, and it does not return everything required for other functions in the forecast package to work. Finally, it does not allow the estimated model to be applied to new data (which is useful for checking forecast accuracy). Consequently, it is recommended that ARIMA() be used instead.

### Modelling procedure

When fitting an ARIMA model to a set of (non-seasonal) time series data, the following procedure provides a useful general approach.

1. Plot the data and identify any unusual observations.
2. If necessary, transform the data (using a Box-Cox transformation) to stabilise the variance.
3. If the data are non-stationary, take first differences of the data until the data are stationary.
4. Examine the ACF/PACF: Is an ARIMA($$p,d,0$$) or ARIMA($$0,d,q$$) model appropriate?
5. Try your chosen model(s), and use the AICc to search for a better model.
6. Check the residuals from your chosen model by plotting the ACF of the residuals, and doing a portmanteau test of the residuals. If they do not look like white noise, try a modified model.
7. Once the residuals look like white noise, calculate forecasts.

The Hyndman-Khandakar algorithm only takes care of steps 3–5. So even if you use it, you will still need to take care of the other steps yourself.

The process is summarised in Figure 9.12. Figure 9.12: General process for forecasting using an ARIMA model.

### Example: Seasonally adjusted electrical equipment orders

We will apply this procedure to the seasonally adjusted electrical equipment orders data shown in Figure 9.13.

elec_equip <- as_tsibble(fpp2::elecequip)
elec_dcmp <- elec_equip %>%
model(STL(value ~ season(window="periodic"))) %>%
components() %>%
select(-.model) %>%
as_tsibble()
elec_dcmp %>%
autoplot(season_adjust) Figure 9.13: Seasonally adjusted electrical equipment orders index in the Euro area.

1. The time plot shows some sudden changes, particularly the big drop in 2008/2009. These changes are due to the global economic environment. Otherwise there is nothing unusual about the time plot and there appears to be no need to do any data adjustments.

2. There is no evidence of changing variance, so we will not do a Box-Cox transformation.

3. The data are clearly non-stationary, as the series wanders up and down for long periods. Consequently, we will take a first difference of the data. The differenced data are shown in Figure 9.14. These look stationary, and so we will not consider further differences.

elec_dcmp %>%
gg_tsdisplay(difference(season_adjust), plot_type='partial') Figure 9.14: Time plot and ACF and PACF plots for the differenced seasonally adjusted electrical equipment data.

4. The PACF shown in Figure 9.14 is suggestive of an AR(3) model. So an initial candidate model is an ARIMA(3,1,0). There are no other obvious candidate models.

5. We fit an ARIMA(3,1,0) model along with variations including ARIMA(4,1,0), ARIMA(2,1,0), ARIMA(3,1,1), etc. Of these, the ARIMA(3,1,1) has a slightly smaller AICc value.

fit <- elec_dcmp %>%
model(
arima = ARIMA(season_adjust ~ pdq(3,1,1) + PDQ(0,0,0))
)
report(fit)
#> Model: ARIMA(3,1,1)
#>
#> Coefficients:
#>          ar1     ar2     ar3      ma1
#>       0.0044  0.0916  0.3698  -0.3921
#> s.e.  0.2201  0.0984  0.0669   0.2426
#>
#> sigma^2 estimated as 9.577:  log likelihood=-492.7
#> AIC=995.4   AICc=995.7   BIC=1012
6. The ACF plot of the residuals from the ARIMA(3,1,1) model shows that all autocorrelations are within the threshold limits, indicating that the residuals are behaving like white noise.

fit %>% gg_tsresiduals() Figure 9.15: Residual plots for the ARIMA(3,1,1) model.

A portmanteau test returns a large p-value, also suggesting that the residuals are white noise.

augment(fit) %>%
features(.resid, ljung_box, lag = 24, dof = 4)
#> # A tibble: 1 x 3
#>   .model lb_stat lb_pvalue
#>   <chr>    <dbl>     <dbl>
#> 1 arima     24.0     0.241
7. Forecasts from the chosen model are shown in Figure 9.16.

fit %>% forecast() %>% autoplot(elec_dcmp) Figure 9.16: Forecasts for the seasonally adjusted electrical orders index.

### Understanding constants in R

A non-seasonal ARIMA model can be written as $\begin{equation} \tag{9.3} (1-\phi_1B - \cdots - \phi_p B^p)(1-B)^d y_t = c + (1 + \theta_1 B + \cdots + \theta_q B^q)\varepsilon_t, \end{equation}$ or equivalently as $\begin{equation} \tag{9.4} (1-\phi_1B - \cdots - \phi_p B^p)(1-B)^d (y_t - \mu t^d/d!) = (1 + \theta_1 B + \cdots + \theta_q B^q)\varepsilon_t, \end{equation}$ where $$c = \mu(1-\phi_1 - \cdots - \phi_p )$$ and $$\mu$$ is the mean of $$(1-B)^d y_t$$. The fable package uses the parameterisation of Equation (9.3) and other R implementations usually use Equation (9.4).

Thus, the inclusion of a constant in a non-stationary ARIMA model is equivalent to inducing a polynomial trend of order $$d$$ in the forecast function. (If the constant is omitted, the forecast function includes a polynomial trend of order $$d-1$$.) When $$d=0$$, we have the special case that $$\mu$$ is the mean of $$y_t$$.

By default, the ARIMA() function will automatically determine if a constant should be included. For $$d=0$$ or $$d=1$$, a constant will be included if it improves the AICc value. If $$d>1$$ the constant is always omitted as a quadratic or higher order trend is particularly dangerous when forecasting.

The constant can be specified by including 0 or 1 in the model formula (similarly to the intercept in lm()). For example, to automatically select an ARIMA model with a constant, you could use ARIMA(y ~ 1). Similarly, a constant can be excluded with ARIMA(y ~ 0).

### Plotting the characteristic roots

(This is a more advanced section and can be skipped if desired.)

We can re-write Equation (9.3) as $\phi(B) (1-B)^d y_t = c + \theta(B) \varepsilon_t$ where $$\phi(B)= (1-\phi_1B - \cdots - \phi_p B^p)$$ is a $$p$$th order polynomial in $$B$$ and $$\theta(B) = (1 + \theta_1 B + \cdots + \theta_q B^q)$$ is a $$q$$th order polynomial in $$B$$.

The stationarity conditions for the model are that the $$p$$ complex roots of $$\phi(B)$$ lie outside the unit circle, and the invertibility conditions are that the $$q$$ complex roots of $$\theta(B)$$ lie outside the unit circle. So we can see whether the model is close to invertibility or stationarity by a plot of the roots in relation to the complex unit circle.

It is easier to plot the inverse roots instead, as they should all lie within the unit circle. This is easily done in R. For the ARIMA(3,1,1) model fitted to the seasonally adjusted electrical equipment index, we obtain Figure 9.17.

gg_arma(fit) Figure 9.17: Inverse characteristic roots for the ARIMA(3,1,1) model fitted to the seasonally adjusted electrical equipment index.

The three red dots in the left hand plot correspond to the roots of the polynomials $$\phi(B)$$, while the red dot in the right hand plot corresponds to the root of $$\theta(B)$$. They are all inside the unit circle, as we would expect because R ensures the fitted model is both stationary and invertible. Any roots close to the unit circle may be numerically unstable, and the corresponding model will not be good for forecasting.

The ARIMA() function will never return a model with inverse roots outside the unit circle. Models automatically selected by the ARIMA() function will not select a model with roots close to the unit circle.