ARIMA models for time series forecasting


Notes on nonseasonal ARIMA models (pdf file)

Slides on seasonal and nonseasonal ARIMA models (pdf file)

Introduction to ARIMA: nonseasonal models
Identifying the order of differencing in an ARIMA model
Identifying the numbers of AR or MA terms in an ARIMA model
Estimation of ARIMA models
Seasonal differencing in ARIMA models
Seasonal random walk: ARIMA(0,0,0)x(0,1,0)
Seasonal random trend: ARIMA(0,1,0)x(0,1,0)

General seasonal models: ARIMA (0,1,1)x(0,1,1) etc.
Summary of rules for identifying ARIMA models
ARIMA models with regressors
The mathematical structure of ARIMA models (pdf file)


Estimation of ARIMA models

Linear versus nonlinear least squares
Mean versus constant

Linear versus nonlinear least squares

ARIMA models which include only AR terms are special cases of linear regression models, hence they can be fitted by ordinary least squares.

ARIMA models which include MA terms are similar to regression models, but can't be fitted by ordinary least squares:

"Mean" versus "constant"

The "mean" and the "constant" in ARIMA model-fitting results are different numbers whenever the model includes AR terms. Suppose that you fit an ARIMA model to Y in which p is the number of autoregressive terms. (Assume for convenience that there are no MA terms.) Let y denote the differenced (stationarized) version of Y, e.g., yt = Yt - Yt-1 if one nonseasonal difference was used. Then the AR(p) forecasting equation for y is:

ŷt   =  μ  +  ϕ1 yt-1 + ϕ2yt-2 +… + ϕpyt--p

This is just an ordinary multiple regression model in which μ is the constant term, ϕ1 is the coefficient of the first lag of y, and so on.

Now, internally, the software converts this slope-intercept form of the regression equation to an equivalent form in terms of deviations from the mean. Let m denote the mean of the stationarized series y. Then the p-order autoregressive equation can be written in terms of deviations from the mean as:

ŷt   =  m  +  ϕ1 (yt-1 - m) + ϕ2(yt-2 - m) +… + ϕp(yt--p - m)

By collecting all the constant terms in this equation, we see it is equivalent to the original form of the equation if:

μ   =   m(1 - ϕ1 - ϕ2 - … - ϕp  )

or in words:

CONSTANT = MEAN x (1 - sum of AR coefficients)

The software actually estimates m (along with the other model parameters) and reports this as the MEAN in the model-fitting results, along with its standard error and t-statistic, etc. The CONSTANT (μ) is then calculated according to the formula above. If the model does not contain any AR terms, the MEAN and the CONSTANT are identical.

In a model with one order of nonseasonal differencing (only), the MEAN is the trend factor (average period-to-period change). In a model with one order of seasonal differencing (only), the MEAN is the annual trend factor (average year-to-year change).


Go to next topic: Seasonal differencing in ARIMA models