**ARIMA models for time series forecasting**

Notes
on nonseasonal ARIMA models (pdf file)

Slides on seasonal and
nonseasonal ARIMA models (pdf file)

Introduction
to ARIMA: nonseasonal models

Identifying the order of differencing in an ARIMA model

Identifying the numbers of AR or MA terms in an ARIMA
model

Estimation of ARIMA models

Seasonal differencing in ARIMA models

Seasonal random walk: ARIMA(0,0,0)x(0,1,0)

Seasonal random trend: ARIMA(0,1,0)x(0,1,0)

General seasonal models: ARIMA (0,1,1)x(0,1,1) etc.

Summary of rules for
identifying ARIMA models

ARIMA models with
regressors

The
mathematical structure of ARIMA models (pdf file)

**ARIMA models with
regressors**

An ARIMA
model can be considered as a special type of regression model--in which the
dependent variable has been stationarized and the independent variables are all
lags of the dependent variable and/or lags of the errors--so it is
straightforward in principle to extend an ARIMA model to incorporate
information provided by leading indicators and other exogenous variables: you
simply add one or more regressors to the forecasting equation.

Alternatively,
you can think of a hybrid ARIMA/regression model as a regression model which
includes a correction for autocorrelated errors. If you have fitted a multiple
regression model and find that its residual ACF and PACF plots display an
identifiable autoregressive or moving-average "signature" (e.g., some
significant pattern of autocorrelations and/or partial autocorrelations at the
first few lags and/or the seasonal lag), then you might wish to consider adding
ARIMA terms (lags of the dependent variable and/or the errors) to the
regression model to eliminate the autocorrelation and further reduce the mean
squared error. To do this, you would just re-fit the regression model as an
ARIMA model with regressors, and you would specify the appropriate AR and/or MA
terms to fit the pattern of autocorrelation you observed in the original
residuals.

Most
high-end forecasting software offers one or more options for combining the
features of ARIMA and multiple regression models. In the Forecasting procedure
in Statgraphics, you can do this by specifying "ARIMA" as the model
type and then hitting the "Regression" button to add regressors.
(Alas, you are limited to 5 additional regressors.) When you add a regressor to
an ARIMA model in Statgraphics, it literally just adds the regressor to the
right-hand-side of the ARIMA forecasting equation. To use a simple case,
suppose you first fit an ARIMA(1,0,1) model with no regressors. Then the
forecasting equation fitted by Statgraphics is:

Ŷ_{t} = μ + ϕ_{1}Y_{t-1} -
θ_{1}e_{t-1}

which can
be rewritten as:

Ŷ_{t} - ϕ_{1}Y_{t-1} = μ - θ_{1}e_{t}_{-1}

(Note:
this is a standard mathematical form which is often used for ARIMA models. All
terms involving the dependent variable--i.e., all the AR terms and
differences--are collected on the left-hand-side of the equation, while all
terms involving the erorrs--i.e., the MA terms--are collected on the right-hand
side.) Now, if you add a regressor X to the forecasting model, the equation
fitted by Statgraphics is:

Ŷ_{t} - ϕ_{1}Y_{t-1} = μ - θ_{1}e_{t-1 }+ β(X_{t }- ϕ_{1}X_{t-1})

Thus, the
AR part of the model (and also the differencing transformation, if any) is
applied to the X variable in exactly the same way as it is applied to the Y
variable *before* X is multiplied by the regression
coefficient. This effectively means that the ARIMA(1,0,1) model is
fitted to the errors of the regression of Y on X (i.e., the series "Y
minus beta X").

How can
you tell if it might be helpful to add a regressor to an ARIMA model? One
approach would be to save the RESIDUALS of the ARIMA model and then look at
their__ cross-correlations __with other potential explanatory variables. For
example, recall that we previously fitted a regression model model to
seasonally adjusted auto sales, in which the LEADIND variable (index of eleven
leading economic indicators) turned out to be slightly significant in addition
to lags of the stationarized sales variable. Perhaps LEADIND would also be
helpful as a regressor in the seasonal ARIMA model we
subsequently fitted to auto sales.

To test
this hypothesis, the RESIDUALS from the ARIMA(0,1,1)x(0,1,1) model fitted to
AUTOSALE were saved. Their cross-correlations with DIFF(LOG(LEADIND)), plotted
in the Descriptive Methods procedure, are as follows:

(A couple
of minor technical points to note here: we have logged and differenced LEADIND
to stationarize it because the RESIDUALS of the ARIMA model are also logged and
differenced--i.e., expressed in units of percentage change. Also, the
Descriptive Methods procedure, like the Forecasting procedure, does not like
variables which begin with too many missing values. Here the missing values at
the beginning of the RESIDUALS variables were replaced by *zeroes*--typed
in by hand--before running the Descriptive Methods procedure. Actually, the
Forecasting procedure is supposed to automatically draw cross-correlation plots
of the residuals versus other variables, but the graph which is labeled
"Residual Cross-Correlation Plot" merely shows the cross-correlations
of the *input variable* versus other variables.)

We see that the most significant
cross-correlation is at lag 0, but unfortunately we cannot use that for
forecasting one month ahead. Instead, we must try to exploit the smaller
cross-correlations at lags 1 and/or 2. As a quick test of whether lags of
DIFF(LOG(LEADIND)) are likely to add anything to our ARIMA model, we can use
the Multiple Regression procedure to regress RESIDUALS on lags of
DIFF(LOG(LEADIND)). Here is the result of regressing RESIDUALS on
LAG(DIFF(LOG(LEADIND)),1):

The R-squared value of only 3.66% suggests that not much
improvement is possible. (If two lags of DIFF(LOG(LEADIND)) are used, the
R-squared only increases to 4.06%.) If we return to the ARIMA procedure and add
LAG(DIFF(LOG(LEADIND)),1) as a regressor, we obtain the following model-fitting
results:

(Minor technical point here: we stored the values of
LAG(DIFF(LOG(LEADIND)),1) in a new column, filled in the two missing values at
the beginning with zeroes, and assigned the new column the name LGDFLGLEAD.) We
see that when a coefficient for the lag of DIFF(LOG(LEADIND)) is estimated *simultaneously
*with the other parameters of the model, it is even less significant than it
was in the regression model for RESIDUALS. The improvement in the
root-mean-squared error is just too small to be noticeable.

The negative result
we obtained here should not be taken to suggest that regressors will never be
helpful in ARIMA models or other time series models. For example, variables
which measure advertising or price levels or the occurrence of promotional
events are often helpful in augmenting ARIMA models (and exponential smoothing
models) for forecasting sales at the level of the firm or product. Remember
that the variable being analyzed here--nationwide sales at automotive
dealers--is a highly aggregated macroeconomic time series. We have learned by
now that the impact on a macroeconomic variable of events which occurred in *earlier
periods* (e.g., changes in various economic factors that make up the index
of leading indicators) is often most clearly represented in the prior history
of that variable itself. Hence, lagged values of other macroeconomic time
series may have little to add to a forecasting model which has already fully
exploited the history of the original time series. Leading economic indicators
are often more useful when applied as they are intended--namely as indicators
of turning points in business cycles that may have a bearing on the direction
of longer-term trend projections.