Introduction to
ARIMA: nonseasonal models

Identifying the order of differencing in an ARIMA model

Identifying the numbers of AR or MA terms in an ARIMA
model

Estimation of ARIMA models

Seasonal differencing in ARIMA models

Seasonal random walk: ARIMA(0,0,0)x(0,1,0)

Seasonal random trend: ARIMA(0,1,0)x(0,1,0)

General seasonal models: ARIMA (0,1,1)x(0,1,1) etc.

Summary of rules for identifying ARIMA models

ARIMA models with regressors

**Introduction to ARIMA:
nonseasonal models**

ARIMA(p,d,q)

ARIMA(0,1,0) = random walk

ARIMA(1,1,0) = differenced first-order autoregressive model

ARIMA(0,1,1) without constant = simple exponential smoothing

ARIMA(0,1,1) with constant = simple exponential smoothing
with growth

ARIMA(0,2,1) or (0,2,2) without constant = linear exponential
smoothing

A "mixed" model--ARIMA(1,1,1)

Spreadsheet implementation

**ARIMA(p,d,q):
**ARIMA
models are, in theory, the most general class of models for forecasting a time
series which can be stationarized by transformations such as differencing and
logging. In fact, the easiest way to think of ARIMA models is as fine-tuned
versions of random-walk and random-trend models: the fine-tuning consists of
adding *lags of the differenced series* and/or* lags of the forecast
errors *to the prediction equation, as needed to remove any last traces of
autocorrelation from the forecast errors.

The
acronym ARIMA stands for "Auto-Regressive Integrated Moving Average."
Lags of the differenced series appearing in the forecasting equation are called
"auto-regressive" terms, lags of the forecast errors are called
"moving average" terms, and a time series which needs to be
differenced to be made stationary is said to be an "integrated"
version of a stationary series. **Random-walk and random-trend models,
autoregressive models, and exponential smoothing models (i.e., exponential
weighted moving averages) are all special cases of ARIMA models.**

A
nonseasonal ARIMA model is classified as an "ARIMA(p,d,q)" model,
where:

**p**is the number of autoregressive terms,**d**is the number of nonseasonal differences, and**q**is the number of lagged forecast errors in the prediction equation.

To
identify the appropriate ARIMA model for a time series, you begin by
identifying the order(s) of differencing needing to stationarize the series and
remove the gross features of seasonality, perhaps in conjunction with a
variance-stabilizing transformation such as logging or deflating. If you stop
at this point and predict that the differenced series is constant, you have
merely fitted a random walk or random trend model. (Recall that the random walk
model predicts the first difference of the series to be constant, the seasonal
random walk model predicts the seasonal difference to be constant, and the
seasonal random trend model predicts the first difference of the seasonal
difference to be constant--usually zero.) However, the best random walk or random
trend model may still have autocorrelated errors, suggesting that additional
factors of some kind are needed in the prediction equation.

**ARIMA(0,1,0)
= random walk: **In
models we have studied previously, we have encountered two strategies for
eliminating autocorrelation in forecast errors. One approach, which we first used
in regression analysis, was the addition of lags of the stationarized series.
For example, suppose we initially fit the random-walk-with-growth model to the
time series Y. The prediction equation for this model can be written as:

...where
the constant term (here denoted by "mu") is the average difference in
Y. This can be considered as a degenerate regression model in which DIFF(Y) is the
dependent variable and there are no independent variables other than the
constant term. Since it includes (only) a nonseasonal difference and a constant
term, it is classified as an "ARIMA(0,1,0) model with constant." Of
course, the random walk without growth would be just an ARIMA(0,1,0) model *without*
constant

**ARIMA(1,1,0)
= differenced first-order autoregressive model: **If the errors of
the random walk model are autocorrelated, perhaps the problem can be fixed by
adding one lag of the dependent variable to the prediction equation--i.e., by
regressing DIFF(Y) on itself lagged by one period. This would yield the
following prediction equation:

which can
be rearranged to

This is a first-order
autoregressive, or "AR(1)", model with one order of nonseasonal
differencing and a constant term--i.e., an "ARIMA(1,1,0) model with
constant." Here, the constant term is denoted by "mu" and the
autoregressive coefficient is denoted by "phi", in keeping with the
terminology for ARIMA models popularized by Box and Jenkins. (In the output of
the Forecasting procedure in Statgraphics, this coefficient is simply denoted
as the AR(1) coefficient.)

**ARIMA(0,1,1)
without constant = simple exponential smoothing:** Another strategy
for correcting autocorrelated errors in a random walk model is suggested by the
simple exponential smoothing model. Recall that for some nonstationary time
series (e.g., one that exhibits noisy fluctuations around a slowly-varying
mean), the random walk model does not perform as well as a moving average of past
values. In other words, rather than taking the most recent observation as the
forecast of the next observation, it is better to use an *average *of the
last few observations in order to filter out the noise and more accurately
estimate the local mean. The simple exponential smoothing model uses an *exponentially
weighted moving average* of past values to achieve this effect. The
prediction equation for the simple exponential smoothing model can be written
in a number of mathematically equivalent ways, one of which is:

...where
e(t-1) denotes the error at period t-1. Note that this resembles the prediction
equation for the ARIMA(1,1,0) model, except that instead of a multiple of the
lagged difference it includes *a multiple of the lagged forecast error.*
(It also does not include a constant term--yet.) The coefficient of the lagged
forecast error is denoted by the Greek letter "theta" (again
following Box and Jenkins) and it is conventionally written with a *negative*
sign for reasons of mathematical symmetry. "Theta" in this equation
corresponds to the quantity "1-minus-alpha" in the exponential
smoothing formulas we studied earlier.

When a
lagged forecast error is included in the prediction equation as shown above, it
is referred to as a "moving average" (MA) term. The simple
exponential smoothing model is therefore a first-order moving average
("MA(1)") model with one order of nonseasonal differencing and no
constant term --i.e., an "ARIMA(0,1,1) model without constant." This
means that in Statgraphics (or any other statistical software that supports
ARIMA models) you can actually fit a simple exponential smoothing by specifying
it as an ARIMA(0,1,1) model without constant, and the estimated MA(1)
coefficient corresponds to "1-minus-alpha" in the SES formula.

**ARIMA(0,1,1)
with constant = simple exponential smoothing with growth:** By implementing
the SES model as an ARIMA model, you actually gain some flexibility. First of
all, the estimated MA(1) coefficient is allowed to be *negative*: this
corresponds to a smoothing factor larger than 1 in an SES model, which is
usually not allowed by the SES model-fitting procedure. Second, you have the
option of including a constant term in the ARIMA model if you wish, in order to
estimate an average non-zero trend. The ARIMA(0,1,1) model *with* constant
has the prediction equation:

The
one-period-ahead forecasts from this model are qualitatively similar to those
of the SES model, except that the trajectory of the long-term forecasts is
typically a sloping line (whose slope is equal to mu) rather than a horizontal
line.

**ARIMA(0,2,1)
or (0,2,2) without constant = linear exponential smoothing: **Linear exponential
smoothing models are ARIMA models which use two nonseasonal differences in
conjunction with MA terms. The second difference of a series Y is not simply
the difference between Y and itself lagged by two periods, but rather it is the
*first difference of the first difference*--i.e., the change-in-the-change
of Y at period t. Thus, **the second difference of Y at period t is equal to (Y(t)-Y(t-1))
- (Y(t-1)-Y(t-2)) = Y(t) - 2Y(t-1) + Y(t-2)**. A second difference of a
discrete function is analogous to a second derivative of a continuous function:
it measures the "acceleration" or "curvature" in the
function at a given point in time.

The
ARIMA(0,2,2) model without constant predicts that the second difference of the
series equals a linear function of the last two forecast errors:

which can
be rearranged as:

where
theta-1 and theta-2 are the MA(1) and MA(2) coefficients. This is essentially
the same as Brown's linear exponential smoothing model, with the MA(1)
coefficient corresponding to the quantity 2*(1-alpha) in the LES model. To see
this connection, recall that forecasting equation for the LES model is:

Upon
comparing terms, we see that the MA(1) coefficient corresponds to the quantity
2*(1-alpha) and the MA(2) coefficient corresponds to the quantity -(1-alpha)^2
(i.e., "minus (1-alpha) squared"). If alpha is larger than 0.7, the
corresponding MA(2) term would be less than 0.09, which might not be
significantly different from zero, in which case an ARIMA(0,2,1) model probably
would be identified.

**A "mixed" model--ARIMA(1,1,1):** The features of
autoregressive and moving average models can be "mixed" in the same
model. For example, an ARIMA(1,1,1) model with constant would have the
prediction equation:

Normally,
though, we will try to stick to "unmixed" models with either only-AR
or only-MA terms, because including both kinds of terms in the same model
sometimes leads to overfitting of the data and non-uniqueness of the
coefficients.

**Spreadsheet
implementation: **ARIMA
models such as those described above are easy to implement on a spreadsheet.
The prediction equation is simply a linear equation that refers to past values
of original time series and past values of the errors. Thus, you can set up an
ARIMA forecasting spreadsheet by storing the data in column A, the forecasting
formula in column B, and the errors (data minus forecasts) in column C. The
forecasting formula in a typical cell in column B would simply be a linear
expression referring to values in preceding rows of columns A and C, multiplied
by the appropriate AR or MA coefficients stored in cells elsewhere on the
spreadsheet.

Go to next topic: Identifying the order of differencing