Data concepts


Principles and risks of forecasting (pdf)

Famous forecasting quotes
How to move data around
Get to know your data
Inflation adjustment (deflation)
Seasonal adjustment
Stationarity and differencing
The logarithm transformation


Get to know your data

Data sources and units: Before you even begin to analyze your data, you should ask:

Later, when you write up the results of your analysis, the variables in your data set should be clearly annotated to indicate their sources, units of measurement, and any problems or peculiarities you are aware of.

The bad news here is that assembling, cleaning, adjusting, and documenting the units of the data is often the most tedious step of forecasting, and failure to attend to these mundane details may lead to egregious errors of modeling. Fortunes may be lost and heads may roll. The good news is that you often learn a good deal in the process, gaining insight into the nature and logical relationships of the variables you wish to predict.

You may also find that the most important management benefit of your forecasting project is to identify ways in which your organization's data can be better collected, better organized, better integrated, and better summarized for purposes of decision-making. 

Draw the #!*$ picture: Before you crunch a single number, you should graph your data to get a feel for its qualitative properties. For example, suppose you are studying the history of economic growth in the U.S. over the period from 1992 to the present. Here's a time series plot of one very important measure, total retail sales:

Note that data are in millions of dollars, not adjusted for inflation and not seasonally adjusted ("NSA"). The chart and data were obtained from, a highly recommended source for economic data.

What qualitative features are evident on this graph? You might notice some of the following:

A forecasting model for this time series must accommodate all these qualitative features, and ideally it should shed light on their underlying causes. To study these features of the time series in more depth, and to help determine which kind of forecasting model is most appropriate, we should next plot some transformations of the original data.

Go on to next topic: Inflation adjustment (deflation)