Time

Series
Analysis

July 14

2011

These are Class Notes for certain topics of the Core Course STAT 569 to be
offered to the Students of M.Sc.(Ag) Statistics & Maths., specifically
prepared by Dr. B.S. Kulkarni

Course No.
STAT 569

I - Time Series Analysis

Time Series:
Data recorded in relation to time
Sequence of observations ordered in time
Collection of observations made (recorded) sequentially in time.
Univariate Time Series: Data recorded corresponding to a single variable in relation to time.
Multivariate Time Series: Data recorded corresponding to a vector of random variables in relation to time.
Types of Time Series Data:
Continuous: Observations that are recorded continuously in time.
Eg: Wind velocity recorded through equipment; Heart beats recorded through Cardiogram; Environmental
Temperature; Pollution Index, etc.
Discrete: Observations that are recorded only at specific times.
Eg: Annual Crop Production; Area under Crops during a season / year(s); monthly / weekly rainfall of
year(s)
Stationary: Data that fluctuate around a constant value.
Non-Stationary: A series having the parameters of the cyclic changes over time.
Deterministic Time Series: The data that can be predicted exactly, without any error.
Stochastic Time Series: The data are only partly determined by past values and future values have to be
described with probabilistic distribution.
Time Series Analysis- It refers to techniques that are applied to identify the characteristics of time series
data.
The basic assumption in time series analysis is that certain past pattern will continue to remain in the
future. The time series process is often assumed to be based on the past values of the main variable but
not on the explanatory variables, which influence the system. The limitation in these analyses is that it is
possible to predict the future without any emphasis on the causes (factors) that are responsible for the
forecast; i.e., what will happen is achieved instead of why it has happened!
The techniques of time series analysis are basically developed to deal with two major issues1. Characteristics of time series data and
2. Developing forecasting models, referred as Time Series Modeling.
Box and Jenkins are the pioneers in time series modeling. According to these scientists, a minimum of 50
years (observations) of data are required to perform the analysis.
Goals of Time Series Analysis:
1. Identification of the patterns (nature of the phenomenon) represented by the data
2. Forecasting the future values and
3. Intervention analysis
Intervention analysis involves identification of intervening factors (may be either events or factors) which
may have a significant impact on the data values (observations) of the time series. In the context of
agricultural time series data, introduction of High Yielding Varieties (HYVs) / Improved Technological
measures had a significant impact on the crop productivity and hence the crop production. However, the
role of these factors have to be identified by applying suitable methodologies, preferably after viewing the
line-graph of the data.
The fundamental requirement in these goals is identification of patterns in the data and its description /
quantification.
1. Identification of Patterns (Characteristics) in Time Series Data:
Time series data are affected by several factors. The combined effect of these factors fluctuate the data
over time. Some of these factors are identifiable and some others are un-identifiable, resulting out of
random causes. These factors, in general, are referred as Components of Time Series.
Measuring the relative contribution of these components (factors) helps in developing models for
forecasting.

Compiled by Dr. B.S. KULKARNI

Components of Time Series: It is assumed that an observation recorded at any moment of time t, i.e., Ot
is the result (outcome) of mainly three factor-effects, which are referred as Components of time series.
These are:
1. Trend (T);
Periodic Changes consisting of:
2. Seasonal (S) and
3. Cyclic (C) variations and
4. Irregular (I) or Random variations
Characteristics of the Components:
Trend:
It is the general tendency the data to increase / decrease during a long period of time.
It is the general smooth and long term average tendency. It is not necessary that the increase or
decrease should be in the same direction throughout a given period.
It is not necessary that every time series may exhibit an upward / downward trend. There may be a
series that fluctuate around a constant value (oscillatory type).
The trend may be linear or non-linear type. This can be identified only through a line-graph of the
series.
The term long term may depend on the type of data- In certain situations, a period as small as week
may be fairly long; while in some other situations, even a period of 2 years may be very small.
In the context of agricultural data on crop production, the period has to be sufficiently more than 10
years; while in bacterial count studies, the bacteria count recorded every after 5 minutes of a week
may exhibit a trend.
Periodic Changes: These are classified into Seasonal and Cyclic changes:
Seasonal Changes:
These are short-term changes, which operate in a regular and periodic manner over a span of less
than a year; i.e., the period is 12 months and the pattern is almost same during every year.
These changes may occur due to natural causes such as climatic /seasonal factors (Sale of ColdDrugs-Vicks, Rain-Coats during rainy season or sale of Air-Coolers / Air-Conditioners during
summer)
These changes may also occur due to man-made (human) causes- habits / customs / conventions of
people (Sale of Jewelry items during marriage season, sale of crackers during deepawali, etc.)
Cyclic Changes:
The oscillatory movements in time series with a period more than one year are termed as cyclic
variations.
One complete period is a Cycle
The period (Cycle) varies depending on the data (Weather Cycle- Draught every after 5 / 10 years;
business cycle- recession).
The cycle has up-swings and down-swings, commonly referred as peaks and troughs.
Irregular / Random Changes:
These variations do not belong to any of the above categories. The fluctuations are random
(erratic) or non-recurring type.
Eg: Political Instability, Natural Calamities (Earth Quakes, Floods, etc.)
Analysis of Time Series Data:
Objective: to identify the components affecting the time series and measure its extent /contribution.
The approach involved in measurement is isolation.
Models for Measuring the Contribution of Components:
1. Additive Model:
It is assumed that the components are independently affecting the observations of the time series.
2. Multiplicative (Product) Model:
It is assumed that the components are jointly affecting the observations of the time series. This
model is relatively preferred.
Compiled by Dr. B.S. KULKARNI

It is assumed that trend and periodic components are determined by separate forces acting
independently so that simple aggregation would constitute the series. However, it is possible that
the observation during a year may largely depend on its value in the previous year (lag effect). This
is a common phenomenon in economics data.
Measurement of Trend: There are no valid automatic techniques to identify the trend component in the
data. However, as long as the trend is monotonous- i.e., consistently increasing or decreasing the
identification and hence the measurement may not be difficult. Generally, the trend in the time series data
is masked by year-to-year fluctuations. Hence the fundamental step in the identification of trend is
smoothening.
Smoothening: It involves some form of local averaging of the data such that the non-systematic
components of the individual observations cancel out. The most commonly applied technique is the moving
average smoothening.
Patterns of Trend: The smoothed series may exhibit variety of patterns, which can be described with a
mathematical function. The trend may be linear or non-linear. The following are the most common
functional forms of the trend:

Mathematical Characteristics of the Trend Patterns:

The following methods are generally applied for measuring the trend:
1. Graphical Method
2. Method of Semi-Averages
3. Method of Moving-Averages
4. Method of Curve-fitting

Compiled by Dr. B.S. KULKARNI

1. Graphical Method:
This method involves obtaining a line-graph of the data and then joining it with a smooth line in the
either directions. The points on this smooth curve determine the trend.
The smoothened graph eliminates the variations due to periodic and irregular components.
It involves no computations. Simplicity in the approach is its greatest advantage.
It is applied for preliminary study of the trend.
2. Method of Semi-Averages:
This method involves dividing the data (series) into two equal groups and computing the semiaverages of these groups
These semi-averages are then plotted against their corresponding (centered) time values and are
joined with a line. This line is then extended either-ways for forecasting / estimation purpose.
The semi-averages smoothens the fluctuations.
Suitable only when the trend is of linear type.
Not suitable when the number of observations in the series are odd (for forming groups with equal
observations)
Not suitable when the number of observations in the series are even. This creates centering
(location) problem for the semi-averages
Not suitable when the fluctuations within the groups are heterogeneous.
3. Method of Moving Averages:
This method involves smoothing the fluctuations by computing the moving averages.
These moving averages are then plotted corresponding to their centered time axis
Moving Average refers to average of successive (time dependent) observations of a period. This
period has to be carefully defined on the basis of nature (period) of the year-to-year fluctuations.
Generally, the period for computing the moving average is odd number of years- i.e., 3 or 5 years.
The odd number is convenient for centering (locating) the year of the average.
If the period of fluctuations is not uniform, then the moving averages of the series may exhibit an
oscillatory series. This is the Slutsky-Yule Effect.
If there are outliers in the data, then instead of average, median would sufficiently smooth the
data.
If fluctuations are very large, then instead of moving averages, exponential smoothening techniques
can be applied. These are based on application of weighted least-squares.
The smoothening procedures filter out the irregular (i.e., noise) variations and transform the series
into a smoothed one, which is free from outliers.
4. Method of Curve-fitting
This method involves fitting of appropriate trend-curve (equation) to the time series data. The
identification of the trend-patterns can be carried out with a simple line-graph of the smoothed
series or through the mathematical characteristics of the trend curves, as listed above.
Curve-fitting is generally carried out by applying the Least-Squares approach. Least-Squares
method is the most scientific approach for fitting these curves.
The least-squares approach eliminates the element of subjectivity, as involved in the semiaverages method, particularly when there is even number of observations for computing the semiaverages.
It is the only scientific method for computing the growth rates.
The approach can be applied only for those curves that can be transformed to linearity. Hence it is
not suitable for non-linear curves such as Modified Exponential, Gompertz and Logistic.

The least squares approach is essentially fitting of a regression equation to the time series data.
The independent variable is the time t.
The linear trend equation can be fitted as a simple regression equation.
The Exponential trend equation can be also fitted as a simple regression equation, after
transforming to linearity through logarithmic transformation.
The Polynomial trend equations, however, can be fitted in the form of a multiple regression
equation by taking X1 (= t), X2 (= t2), X3 (=t3), etc., as the independent variables.
Compiled by Dr. B.S. KULKARNI

Fitting of Non-linear Trend Equations:

The non-linear trend equation, which cannot be transformed to linearity are:
and

These trend equations can be fitted by applying any of the following methods:
1. Method of Selected Points and
2. Method of Partial Totals (Sums)
The Method of Selected Points is based on selection of three equidistant points corresponding to
the time axis.
The rationale in selection of these points is not well defined and does not utilize complete data.
The Method of Partial Totals involves complete utilization of data that are to be divided into three
successive groups (sequentially ordered as per time axis) with equal number of observations.
This method is therefore preferable to that of the method of selected points due to the lack of
subjectivity in selection of data points.
Fitting of Modified Exponential Trend
1. Method of Selected Points:
Consider the trend equation,
Let Y1, Y2 and Y3 be three equidistant points corresponding to the time axis, such that
The equation (1) then implies that:

Dividing (3) by (2),

Substituting c in (2) and re-arranging the terms leads to:

Substitution of b and c in (I) leads to:

The parameters of the equation are then:

Compiled by Dr. B.S. KULKARNI

2. The Method of Partial Totals:

In this method, the time series data are divided into three equal parts that are ordered as per the time axis:
.
The totals of observations (sums) corresponding to these three groups is then computed:

Substitution of Yi (as defined in the trend equation) leads to:

By re-arrangement of (B) (A) and (C) (B) and division leads to:

On substitution of c and subsequent terms in (A) or (B) or (C) leads to the three parameters as:

Fitting of Gompertz Trend:

Applying the logarithmic transformation, the Gompertz curve reduces to the modified exponential:
i.e., the transformed equation is:
Hence, the Gompertz trend equation can be fitted by applying the method of partial totals (as described
above) to the transformed data.
Fitting of Logistic Trend:

The logistic equation can be transformed to Modified Exponential by applying the reciprocal transformation:

Hence the method of partial totals for fitting the Modified Exponential equation can be applied to the
transformed data (1/Y) for obtaining the Logistic Trend.
Measurement of Seasonal Variations: Seasonal variations are measured in the form of index. The
following methods are applied for measuring the indices:
1. Method of Simple Averages
2. Ratio-to-Trend Method
3. Ratio-to-Moving Average Method

Compiled by Dr. B.S. KULKARNI

1. Method of Simple Averages:

It is one of the most preliminary methods. Non-existence of any specific trend or cyclic variations in
the data is the basic assumption in this method.
The data are arranged in a two-way table with months / seasons and the corresponding years /
time as the factors.
Seasonal averages of the data over the years are obtained-

The effect of Irregular variations is eliminated on computing the seasonal averages over the years.
2. Ratio-to-Trend Method:
This method is based on the multiplicative model- O = T x S x C x I
It involves computing Trend (T) values by fitting suitable trend equation and then computing the
ratio:

The C and I components are eliminated by averaging the S values over the years.
The trend equation is fitted to the yearly data, which is averaged over the seasons. The Trend
values for the seasons are computed by expressing the incremental constant on per season basis.

This method assumes non-existence of Cyclic and Irregular variations. Hence, if these are present,
the averaging will not eliminate these components and thus the T values would be biased.
3. Ratio-to-Moving Average Method:
This method is also based on the multiplicative model- O = T x S x C x I
Since the period of seasonal variations is a year (12 months or 4 quarters), the Periodic variations
(Cyclic) and Trend variations can be obtained through computing the 12 monthly moving averages.
These averages are then centered to coincide with the corresponding month (of the year). These
centered averages give:
Finally I component is eliminated by averaging the percentages over the years.
This method is a refinement to the Ratio-to-Trend method and is relatively more stable
The only limitation is that it does not utilize the complete data due to computation of moving
averages- first 6 months and last 6 months cannot be included.
Measurement of Cyclic Variations:
Cyclic variations can be measured as percentages through elimination process, by measuring Trend and
Seasonal variations, as follows:

The approach is crude and very rarely applied.

The most scientific method is based on Harmonic Analysis approach.

Compiled by Dr. B.S. KULKARNI

Harmonic Analysis Approach:

The procedure is referred as Periodogram Analysis.
The approach involves fitting the famous Sine Curve, which oscillates around an axis:

The method involves assigning values for in the range of 0 to 9 (the choice is arbitrary).
The values of A and B are then computed corresponding to the each value of , which gives:
A plot of S () against is obtained. This is the Periodogram.
The most significant- i.e., relatively maximum value of S () and the corresponding is then identified from
the Periodogram.
This represents the period of oscillation, provided no is multiple of another .
The corresponding S () represents the amplitude.

Some of the techniques of identifying and measuring the contribution of time series
components are discussed in this section.
These techniques have their own importance in studying the characteristics of the
components. However, these are considered to be classical (old).
The techniques that are essential for developing the statistical models for forecasting are
presented in the next section, i.e., Time Series Modeling.

Compiled by Dr. B.S. KULKARNI

10

II - Time Series Modeling

Time Series models are developed for forecasting the values of the variable on the basis of the
past or lagged values. The approach of estimation is obviously the regression approach.
The fundamental assumption in application of regression approach is that the data are the
observations recorded from a random sample. In the context of time series data, the data are timedependent and hence non-random. Further, when the regression model involves a dependent variable and
an independent variable (instead of lagged variable of the dependent variable), it is possible that the time
series data on these variables may exhibit a trend in similar direction, that leads to a spurious regression,
though the fit of the model is significant! It is therefore essential to verify the assumption of randomness in
the data. Specifically, the assumption of Stationarity of the time series has to be verified.
The time series data can be thought of as being generated by a stochastic or random process.
Stationary Process: A stochastic process is said to be stationary, if its Mean and Variance are constant
over time and the value of the covariance between any two time periods depends only on the lag between
the two periods and not the actual time at which the covariance is computed.
Statistically, this definition implies that:
If Yt is the stochastic time series with the following properties:
i)
Mean: E(Y) =
ii)
Variance: E(Yt - )2 = 2 and
iii)
Covariance: E(Yt - )( Yt+k - )
Then, the shift of origin of Y from Yt to Yt+m i.e., m years apart, will not change the Mean, Variance and
Autocovariance computed by taking Yt+m data. Briefly, a time series is stationary if its Mean, Variance and
Autocovariance at various lags remain the same no matter at what period of timet these are measured.
Tests for Stationarity of Time Series:
The tests for verifying the assumption of Stationarity of time series are based on the concept of
Autocorrelation Coefficient.
Autocorrelation:
It is defined as correlation between the members of the series of observations ordered in time or
space (cross section data)
Autocorrelation is also defined as lag-correlation of a given series with itself, lagged by a number of
time units (Tinter)
Serial Correlation is defined as correlation between the members of two different series (Tinter)
Autocorrelation is exhibited by a time series due to several reasons. The most common pattern in
any economic time series is Inertia or Sluggishness. The production/employment time series
often exhibits business cycles. i.e., starting from the recession (trough), whenever economic
recovery starts, these series move in upward direction. In this up-swing, the value of the series at
one point of time is greater than its previous value. This trend continues until some intervention
occurs.
Autocorrelation may be positive or negative. Positively Auto Correlated series are referred as
persistent. The characteristics of this type of series are that positive departures from the general
trend (mean) are followed by positive departures from the mean. Negatively Auto correlated series
exhibit positive departures to be followed by negative departures from the mean.
Autocorrelation implies that a time series is predictable, probabilistically, as future values are
correlated with the current and past values.
Autocorrelation complicates the application of statistical tests by reducing the effective sample size.
The three tools for identifying the autocorrelation are:
i.
The Time Series Plot
ii.
The Lagged Scatter Plot and
iii.
The Auto Correlation Function (ACF)
The time series plot and lagged scatter plot approaches are based on visual inspection of the graph and
may be subjective. The approach of Auto Correlation Function (ACF) is therefore more appropriate as it
measures the degree of correlation between the observations that are separated by a specific lag (period).

Compiled by Dr. B.S. KULKARNI

11

Auto Correlation Function (ACF):

ACF at lag k, denoted by k is defined as:
=
k represents the population ACF; its sample estimate is:

Where,
and

k has no units. It lies between -1 and +1

A plot of k against k is known as Correlogram

The function

Autocorrelation is also defined as a regression coefficient in the relationship Yt = Y(t-1) + et -1 < < -1.
The Autocorrelation coefficient in this relation is the first order autocorrelation or autocorrelation of lag 1.

is known as Auto Covariance Function

Partial Auto Correlation Function (PACF):

It is the counterpart of ACF. ACF involves measurement of direct correlation between Yt and Y(t-k), which is
of k-period apart.
Partial Auto Correlation resembles the conventional Partial Correlation between the variables after
eliminating (controlling) the common correlation between the other intervening variables.
Partial Auto Correlation kk measures the correlation between time series variables that are k-period
apart after controlling the correlations at intermediate lags- i.e., lags < k.
It imply Correlation between Yt and Y(t-k) after removing the effect of intermediate Ys
It can be also considered as correlation between two sets of residuals obtained from regressing Yt and

Y(t-k), on the set of intervening variables Y1, Y2, , Y(t-k+1).

Statistically, kk can be computed as follows:

Where,

In particular, if k = 2,

If PACF plot of
against k exhibits significant- tall spikes, i.e., bars at p-th lag, then it indicates
that the autocorrelation of lag p, i.e., effectively explains all the higher order autocorrelations
A plot of
against k is referred as the Periodogram
ACF and PACF helps in identifying the Stationarity in the time series data.

Autocorrelation and Stationarity:

Bartlett has shown that if a time series is stationary/ purely random, i.e., it exhibits a white noise, the
sample autocorrelations s are approximately distributed with Mean = Zero and Variance = 1/N,
where, N is the number of observations (years) in the time series data.
This result implies that for any rk, the Standard Error (S.E.) of the estimate is [1/N]. Further,
applying the properties of Standard Normal Distribution, the 95% Confidence Interval for rk is:
1.96 [1/N].
Hence if an estimated rk falls inside this confidence interval, then the Null Hypothesis rk = 0 has to
be accepted. However, if rk falls outside these limits, then the Null Hypothesis has to be rejected.
This verification based on Bartletts result is applicable to testing of any specified rk.

Compiled by Dr. B.S. KULKARNI

12

Testing of Joint Hypothesis

The Null Hypothesis is: Whether all the rk coefficients are simultaneously Zero.
1. Box and Pierce Q-test:
It is based on Chi-Square Distribution.
The test criterion is:
N = Sample size (No. of Years); m = lag length
Q ~ Chi-Square with df = m
Inference : If Q (cal) Q(tab) then the Null Hypothesis is rejected, implying that Not All the rks are Zero;
At least some of them must be Non-Zero
2. Ljung-Box (LB) Test:
This test is modified form of Q-test.
The test Criterion is:
LB ~ Chi-Square with df = m
Inference: Same as in the Q-test.
Other Approaches to Stationarity:
The concept of Stationarity of a time series and hence its identification can be also viewed through the
characteristics of the most common non-stationary time series. A Non-Stationary series have time-varying
Mean and / or time-varying Variance or both. The classical example of a non-stationary series in the
Random Walk Model (RWM):
Random Walk Model (RWM)
The term random walk is often compared with a drunkards walk. The distance covered in walking by a
drunkard is often away from the normal path and is random; however, it depends on the time/duration of the
walk!
Random Walk Model Time Series is Non-Stationary:
Suppose {Yt} is a time series with N-observations. Suppose Ut is a white noise (random error) error term ~
(0, 2). Then the series {Yt} is said to be a random walk if:
Yt = Yt-1 + Ut (1)
Eq.(1) can be written as:
Y1 = Y0 + U1
Y2 = Y1 + U2 = Y0 + U1 + U2
Y3 = Y2 + U3 = Y0 + U1 + U2 + U3
In general, Yt = Y0 + Ut (2)
E(Yt) = E[Y0 + Ut] = Y0
Similarly, V(Yt) = V[Y0 + Ut] = t 2 (3)
Eqs. (2) and (3) imply that E(Yt) = Initial Value of the series = Y0, which is constant and V(Yt) = t 2 which is
not constant but increases with time t.
These results clearly violate the conditions of Stationarity of Constant Mean and Constant Variance. Thus a
RWM is non-stationary
The concept of Random Walk Model helps in transforming a non-stationary series to stationary series.
This is clear from the following result:
First Differenced Random Walk Model Series is Stationary:
Proof: Consider the RWM series with the usual assumptions of whit noise error term Ut,
Yt = Yt-1 + Ut (A)
Subtracting Yt-1 on both sides, (A)
Yt - Yt-1 = Yt-1 + Ut - Yt-1
i.e.,
(Yt - Yt-1) = Ut,
i.e.,
Yt = Ut (B); where, Yt represents the first order difference (Yt - Yt-1).
(B) implies that on differencing Yt, the differenced series Yt becomes stationary.
Unit Root Stochastic Process:
The Random Walk Model can be written in a general way as:
Yt = Yt-1 + Ut; -1 +1 (C)
Thus if = 1, (C) becomes the Random Walk Model. (C) Implies that for the regression model (C), testing of
= 1 or Unit root may decide whether a series is stationary.
Compiled by Dr. B.S. KULKARNI

13

The terms- Unit Root, Random Walk and Non-Stationarity are thus synonymous.
The concept of Unit Root has led to the following statistical test for verifying the Stationarity of the series:.
Dickey Fuller (DF) Test
The D-F test is based on the concept of unit root.
Let {Yt} be a time series with N-observations and let Ut be a white noise (random error) error term. Then Yt
can be modeled as:
Yt = Yt-1 + Ut; -1 +1 (C)
Subtracting Yt-1 on both sides, (C)
Yt - Yt-1 = Yt-1 - Yt-1 + Ut or,
Yt = ( 1) Yt-1 + Ut (D)
i.e., Yt = Yt-1 + Ut (D); where, = ( 1).
If {Yt} is non-stationary, i.e., Random Walk Model series, then we know that Y t is stationary. Thus testing
the Stationarity implies testing the Null Hypothesis H0: = 0 1 = 0 or = 1.
To test the null hypothesis, we can fit the regression model (D), i.e., Yt = Yt-1 + Ut to the time series data
with N-observations and verify H0: = 0, by computing the test criterion

Normally, in regression analysis, the distribution of the above criterion is Students t with (N-2) df. In the
context of time dependent series, the distribution of the criterion is nott but . Dickey and Fuller have
derived the distribution of and tabulated the critical values of .
Inference: When

(Cal) (Tab), then reject Null Hypothesis, which implies that the series is Stationary.

Similarly, when
(Cal) < (Tab), then accept Null Hypothesis, which implies that the series is NonStationary.
D-F test can be applied to the following forms of regression model:
i) Yt = Yt-1 + Ut
-- --- --- --- Without Intercept
ii) Yt = 0 + Yt-1 + Ut
-- --- --- --- With Intercept
iii) Yt = 0 + 1 t + Yt-1 + Ut
-- --- --- --- With Intercept and Trend
In the above models, the Inference about stationarity is however drawn only on the basis of testing
the Null Hypothesis about .
Augmented Dickey Fuller (ADF) Test
It is likely that in the application of D-F test to the model (D) and its other forms, the error term Ut may be
auto-correlated, i.e., not independent. Under this situation, Dickey Fuller proposed the following
modification to the Model (iii):
(iv) Yt = 0 + 1 t + Yt-1 + Yt - i + Ut,
Where, Yt - i = (Yt I - Yt i-1); I = 1, , m.
Specifically,
Yt 1 = (Yt 1 - Yt 2); Yt 2 = (Yt 2 - Yt 3), etc.
The number of differenced variables included in (iv) has to be determined empirically- stepwise. i.e.,
starting with Yt 1, the variables Yt i are added during each step of fitting the model till Ut becomes uncorrelated.
The D-F / ADF test can be also considered as a tool for transforming the non-stationary series to
stationarity, by applying the differencing. (It determines the order of differencing required to transform
the series to Stationarity):
The model to be fitted for this purpose is:
(v) Dt = 0 + Dt - 1,
where: Dt = (Dt - Dt - 1) and Dt - 1 = Yt - 1 = (Yt - 1 - Yt - 2).
Suppose {Yt} is verified to be Non-Stationary. Then we can fit the Model (v) and test the Null Hypothesis
H0: = 0
If the H0 is rejected, implying that the differenced series is Stationary, then transform Yt to Yt for
developing the time series models.
If H0 is accepted, implying that the series is Non-Stationary, then we can explore whether the second
order differenced series is stationary, by taking Yt = (Yt - Yt 1) instead of Yt (i.e., Dt) in the relation
(v).
Model (v) involves differenced variables on both sides of the equation, unlike that of Model (i).
Compiled by Dr. B.S. KULKARNI

14

Trend Stationary Process (TSP): and Difference Stationary Process

Stationarity of a time series is the basic assumption for developing the time series forecasting models. The
series, in general, is non-stationary. It has to be therefore transformed to Stationarity. The two approaches
that are applied for this purpose are:
1) The De-Trending Approach and 2) the Differencing Approach.
The De-Trending Approach: Trend is one of the important components of time series that affects the
Stationarity of the series. Thus when the series exhibits a trend, the de-trended series, i.e., the residual
obtained after eliminating the trend values, may lead to a stationary series. The trend values corresponding
to Yt are generally estimated by fitting the trend equation:
Yt = 0 + 1 t + Ut
The time series data- agricultural data in particular- generally exhibits trend that is not of uniform type over
the years. For example the production data may exhibit varying trends in the level of production due to the
yield-improving technologies released time to time. Due to these complexities, the above trend equation may
not provide the correct estimate of trend values, as the trend equation assumes uniformity in the year-toyear variation in the data.
Theoretically, we can de-trend a series if the trend is deterministic, instead of stochastic, as assumed in the
above equation.
Thus for verifying the nature of the trend- i.e., whether is it deterministic or stochastic, the D-F test can be
applied to the Model (iii) that involves the trend variable:
Yt = 0 + 1 t + Yt-1 + Ut
Thus if the inference of the test is that the series is non-stationary, then it implies that the trend-t in Yt is
stochastic and it is not appropriate to apply the de-trend approach for transforming the series to Stationarity.
If the series exhibits a deterministic trend, then only this de-trending approach is appropriate.
A Deterministic trend can be considered as the one which can be obtained exactly from the trend equation:
Yt = 0 + 1 t . i.e., there is no error in the estimated trend values. This leads to the following definition:
Def: Trend Stationary Process (TSP)- The Trend Model Yt = 0 + 1 t + Ut represents a Trend Stationary
Process, when (i) Ut is stationary, (ii) E(Ut) = 0 and (iii) V(Ut)2 = 2.
We can show that de-trending the deterministic time series leads to a stationary series:
Consider the Trend Stationary Process, as defined above: Yt = 0 + 1 t + Ut. Let Yt be the trend value of
Y corresponding to t. Then,
(Yt - Yt) = (0 + 1 t + Ut - 0 - 1 t) = Ut.
{(Yt - Yt)} is Stationary.
The Differencing Approach: This approach involves transforming the time series with suitable order of
differencing by applying the D-F / ADF tests. The approach is most practical one. It do not involve any strong
assumptions like that of the De-trending approach.
Def: Difference Stationary Process (DSP)- If Yt is generated as: Yt = (Yt - Yt - 1) = + Ut, where (i) Ut is
stationary, (ii) E(Ut) = 0 and (iii) V(Ut)2 = 2, then the Model represents a Difference Stationary Process
(DSP). i.e., the first order differences of Yt leads to a Stationary series.
Role of Trend Stationary Process (TSP) and Difference Stationary Process in Time Series Analysis
Forecasts made from TSP are more reliable for long-term forecasting than those obtained from DSP.
Forecasts obtained from DSP are un-reliable and sometimes hazardous and may be reliable for shortterm forecasting.
With a deterministic trend as assumed in TSP, variables can be made stationary by including a time
trend in the regression model and then taking the deviations from the trend values for analysis.
However, with a stochastic trend, tests for Cointegration and Non-Stationarity are necessarily to be
applied.
Cointegration is a concept that implies that a linear combination of two or several time-dependent
variables may be stationary even though individually these are non-stationary. This approach implies
that if Yt is non-stationary, it is possible to include one or several X j ts (which are also non-stationary) in
the model so that the Ut may be stationary. However, this involves testing for identification of X j ts to be
included in the model.

Compiled by Dr. B.S. KULKARNI

15

Time Series Models for Forecasting

Time Series models for forecasting the future values are based on the past or lagged values of the
dependent variable Y. These models are developed under the assumption that the given time series is
stationary.
In the context of economics and agricultural data, there are five approaches for forecasting:
1. Exponential Smoothing Methods
2. Single Equation Regression Models
3. Simultaneous Equations Regression Models
4. Auto Regressive Integrated Moving Average (ARIMA) Models and
5. Vector Auto Regression (VAR) Models
Exponential Smoothing Methods essentially involve fitting of a suitable curve to a historic time series
data. Though these methods have their own advantages, these are now regarded as supplementary
methods.
Approaches (2), (3) and (5) involve use of exogenous (independent) variables in modeling. These also
involve certain risk in identifying a spurious relationship.
ARIMA Modeling approach involves analyzing the probabilistic or stochastic properties of time series on
their own under the philosophy of let the data speak for themselves. These models do not consider any
exogenous (independent) variables but these are the past values of the dependent variable itself. These
models were made popular due to the methodological efforts of Box and Jenkins, which is also referred
as B-J Methodology.
Models under ARIMA Frame-Work:
1. Auto Regressive (AR) Model (Process): Let {Yt}; t = 1, , N represent a time series data of N years.
Suppose is the Mean of Y. Then Yt follows a first order Auto Regressive or AR (1) Stochastic process if
Yt is modeled as:
(Yt - ) = 1 (Yt - 1 - ) + Ut - - - - - - (1)
Where, Ut is un-correlated random error term with Mean = 0 and Variance = 2, i.e., white noise.
Model (1) implies that Yt, (i.e., deviation from mean) at any specified timet depends on its previous years
value (i.e., deviation from mean) and random shock Ut.
Similarly, AR (2) Process can be defined as:
(Yt - ) = 1 (Yt - 1 - ) + 2 (Yt - 2 - ) + Ut - - - - - - (2)
In general, the AR (p) Process is:
(Yt - ) = 1 (Yt - 1 - ) + 2 (Yt - 2 - ) + + p (Yt - p - ) + Ut - - - - - - (3)
2. Moving Average (MA) Model (Process): Let {Yt}; t = 1, , N represent a time series data of N years.
Then Yt follows a first order Moving Average or MA (1) Stochastic process if Yt is modeled as:
Yt = + 0 Ut + 1 Ut - 1 - - - - - - (4), where is constant
In general the MA (q) Process is:
Yt = + 0 Ut + 1 Ut-1 + + p Ut - q - - - - - - (5)
The MA process depends only on the error term Ut, which is the Moving Average of the current and the
past error terms.
3. Auto Regressive Moving Average (ARMA) Model (Process): If Yt has the characteristics of both AR
and MA then the process is termed as ARMA. In particular, the Yt follows ARMA (1, 1) process if it is
modeled as:
Yt = + 1 Yt - 1 + 0 Ut + 1 Ut-1 - - - - - - (6)
This model involves a single AR and a single MA terms (Yt 1 and Ut 1)
4. Auto Regressive Integrated Moving Average (ARIMA) Model (Process): This process is an
amalgamation to ARMA process when the time series {Yt} is Non-Stationary or Integrated.
It is obvious that to develop the ARMA model in this situation, the series has to be differenced to
make it stationary and this differenced series, which is now stationary has to be subjected to fitting
of ARMA model.
This process is referred as ARIMA (p, d, q), where p and q refer to the number of AR and MA terms
and d refers to the order of differencing required for making the series a Stationary.
The characteristics of the time series models, i.e., the parameters (p, d, q) and thereafter the
estimation of the relevant model can be carried out in a planned approach outlined by Box and
Jenkins.
Compiled by Dr. B.S. KULKARNI

16

Box and Jenkins (B - J) Methodology

The methodology involves the following four steps:

1. Identification of the characteristics (p, d, q) for the Model
2. Estimation
3. Diagnostics Checking
4. Forecasting
1. Identification: The main tools for identification of AR and MA parameters are the ACF and PACF that
leads to Correlogram and Periodogram, respectively.
The given time series, which is stationary or made stationary by differencing, is subjected to computation of
Autocorrelation Coefficients and Partial Autocorrelation Coefficients with varying lags k. The number of these
computations depends on N-the number of observations (years) of data in the series. Approximately we can
compute N/3 coefficients with varying lags k.
and
are then plotted against k to give respectively the Correlogram and Periodogram.
Box and Jenkins suggested the following guidelines for identification of the model on the basis of the patterns
exhibited by ACF and PACF. (These patterns are theoretical and one can consider them that approximately
fit in the practical application)
Theoretical Patterns of ACF and PACF for Identification of ARIMA Models
Model

Pattern of ACF

Pattern of PACF

AR (p)

Decays Exponentially or with damped Sine Significant Spikes through lags k

Wave Pattern or both

MA (q)

Significant Spikes through lags q

Declines Exponentially

ARMA (p, q)

Exponential Decay

Exponential Decay
Compiled by Dr. B.S. KULKARNI

17

Some of the typical patterns of ACF and PACF are as follows:

Here is a series which is differenced with first order- i.e., the parameter in ARIMA (p, d, q) Model, d =
1. The ACF plot shows four significant correlations corresponding to lags 1, 2, 3 and 4; while PACF
plot shows two significant spikes, which are positive, corresponding to lag 1 and 2. Hence the AR
parameters are 2, i.e., p = 2. The residuals (error) of this model ARIMA (2, 1, 0) has also confirmed
randomness from the ACF and PACF plots:
A: ACF Plot of First Differenced Series.

B: PACF Plot of First Differenced Series

C: ACF Plot of Residual ARIMA (2, 1, 0)

In this example, the series is differenced with order 2 for making it to Stationarity, i.e., the ARIMA
parameter d = 2. It can be observed from the ACF plot that there is a single negative significant spike
corresponding to lag 1; while the PACF plot is decaying exponentially. Hence it is appropriate to
model the series with MA (1), i.e., the parameter q = 1. Thus the model is ARIMA (0, 2, 1):
A: ACF Plot of Second Order Differenced Series
B: PACF Plot of Second Order Differenced Series

Similarly, the characteristics of ARIMA (1, 1, 1) can be interpreted as a first order differenced series,
which has the ACF plot with significant positive spike at lag 1 and the PACF plot exhibits both positive
and negative spikes. However there is a significant positive spike at lag 1 and thereafter the plot is
decaying exponentially.
In general, the actual data may not coincide with the patterns listed in the Table. The practical
approach would be to draw 95% Confidence Limits and select the parameters on the basis of either
or
that fall outside these limits.
Compiled by Dr. B.S. KULKARNI

18

If the time series data is not stationary, transform it to stationarity by applying first / second order
differences, depending on the outcome of the D-F test.
2. Estimation of the Model: On the basis of identification of the parameters (p, d, q) the series is subjected
to fitting of the appropriate ARIMA (p, d, q) model.
The procedure for fitting the model involves transforming the series through appropriate differencing, in case
it is non-stationary, and then subjecting the differenced series to fitting.
Choice of parameters on the basis of significant
or
is solely on empirical (trial and error) basis. For
example, for a AR process, if the Auto Correlations are significant up to lag 4, then one can model AR
systematically in a stepwise manner the models AR (1), AR (2), AR (3) and finally AR (4). If during any step
of the fitting, say, AR (2), the model satisfies the diagnostics checks, then the series can be modeled as AR
(2). The choice of AR (4), as is done in the straight forward approach may not be then relevant.
One can even include only those AR variables, for which the corresponding Auto Correlation coefficients are
statistically significant, in case the ACFs are not significant in a systematic order like for lag 1, 3 and 4, etc.
The estimation procedure for AR Models is straight forward either by ML or Least-Squares approach, as is
done in regression analysis. However, when the MA terms are involved in the model, the estimation
procedure is complex and requires software.
3. Diagnostics Checking: The models that are estimated are acceptable only when the residuals are
random. For this purpose, several alternative models that may be appropriate are to be fitted. The ACF and
PACF of the residuals of these models are then estimated. If the plot of these ACF and PACF exhibit a nonsignificant pattern, then the corresponding model is considered as valid and can be considered for
forecasting.
4. Forecasting: The model that satisfies all the diagnostic checks is considered for forecasting. If the model
is based on differencing / de-trending transformations, then the model has to be represented with relevant
expressions of original series. Then only, the forecasts can be made.
Estimation of ARIMA models is computationally not simple. Software / Packages are essential for its fitting.
The popular packages are SAS, SYSTATand SPSS
Practical Illustration
Developing Forecasting Model for A.P. Rice Production:
Data: 50 Years- 1955-56 to 2004-05
Step-1: Testing the Stationarity of Data
i) ACF Plot of Production Data- No. of : N/3 = 17
PACF Plot of Production Data:

Partial Autocorrelation Plot

Autocorrelation Plot
1.0

0.5

Correlation

Correlation

1.0

0.0
-0.5
-1.0

0.5
0.0
-0.5
-1.0

10
Lag

15

20

10
Lag

15

20

ii) Application of Dickey-Fuller Test:

Table Value of = - 2.93 (5%); - 3.58 (1%); df = 50-2 = 48; D(t) = (Yt Yt 1)
D(t) = 1034.92 - 0.13 Y(t-1)
Parameter
Intercept
Y(t-1)

Coeff

SE

t Stat

1034.92

505.72

2.05

-0.13

0.07

-1.91

Comments:
ACF Plot indicates significant Auto Correlations up to lag 4; PACF has significant spike at lag 1.
This implies that the production data is non-stationary
Further verification by applying Dickey-Fuller Test reveal that (Cal) = -1.91 < (Tab), implying the
series to be non-stationary.

Compiled by Dr. B.S. KULKARNI

19

Step-2: Transforming the data to Stationarity

Application of Dickey-Fuller Test
The series is to be transformed to first-order differencing, i.e., D(t) = (Yt Yt 1); D(t) = (Dt Dt 1);
Model:

D'(t) = 234.36 - 1.37 D(t)

Parameter
Intercept
D(t)
13000

Coeff

SE

t- Stat

234.36

174.72

1.34

-1.37

0.13

-10.27
4000

A.P. Rice Production

1955-2004

11000

First Differenced Series

2000

9000
7000

5000

6 11 16 21 26 31 36 41 46

-2000

3000
1 6 11 16 21 26 31 36 41 46

-4000

Comments:
Dickey-Fuller Test reveal that (Cal) = -10.27 > (Tab), implying that the series is Stationary.
Step-3 Identification of ARIMA Model:
Since the production data was non-stationary, the first order differencing has led it to stationarity. The
differenced series, i.e., D(t) is then subjected to fitting of ARIMA Model.
The ACF and PACF plots of D(t) helps in identifying the AR and MA parameters:
ACF Plot of D(t)
PACF Plot of D(t)
Partial Autocorrelation Plot

1.0

1.0

0.5

0.5

Correlation

Correlation

Autocorrelation Plot

0.0
-0.5
-1.0

0.0
-0.5
-1.0

10
Lag

15

20

10
Lag

15

20

The ACF plot indicates that the auto correlation corresponding to lag 1 is significant; while the PACF plot
indicate that the partial auto correlations corresponding to lag 1 and 4 significant (falling above the
confidence limits).
On the basis of PACF plot, we can infer that the forecasting model may not have any Moving Average (MA)
component and the series can be modeled with only Auto Regressive (AR) Component.
The significant partial correlations in PACF plot indicate that there could be three options to formulate the
ARIMA model1. ARIMA with only single AR term to account for significant spike at lag-1
2. ARIMA with two AR terms to account for significant spikes at lag-1 and lag-4
3. ARIMA with all the four AR terms up to lag-4.
Step-4 Estimation:
Choice-1: ARIMA(1, 1, 0):
Model: D(t) = 179.23 -0.41 D(t-1)
Parameter

Coeff.

S.E.

t Stat

P-value

Intercept

179.23

174.67

1.03

0.31

D(t-1)

-0.41

0.13

-3.01

0.00

Compiled by Dr. B.S. KULKARNI

20
ACF Plot of Residuals

PACF Plot of Residuals

Partial Autocorrelation Plot

1.0

1.0

0.5

0.5

Correlation

Correlation

Autocorrelation Plot

0.0
-0.5
-1.0

0.0
-0.5
-1.0

10
Lag

15

20

10
Lag

15

20

Choice-2: ARIMA(2, 1, 0):

Model: D(t) = 250.10 - 0.39 D(t-1) - 0.37 D(t-4)
Parameter

Coeff.

SE

t Stat

P-value

250.10

179.85

1.39

0.17

D(t-1)

-0.39

0.13

-2.96

0.01

D(t-4)

-0.37

0.16

-2.38

0.02

Intercept

ACF Plot of Residuals

PACF Plot of Residuals

Partial Autocorrelation Plot

1.0

1.0

0.5

0.5

Correlation

Correlation

Autocorrelation Plot

0.0
-0.5
-1.0

10
Lag

15

0.0
-0.5
-1.0

20

10
Lag

15

20

Choice-3: ARIMA(4, 1, 0)

Parameter

Coeff.

SE

t Stat

P-value

Intercept

401.92

185.06

2.17

0.04

D(t-1)

-0.51

0.14

-3.61

0.00

D(t-2)

-0.33

0.16

-2.06

0.05

D(t-3)

-0.39

0.20

-2.00

0.05

D(t-4)

-0.56

0.18

-3.07

0.00

ACF Plot of Residuals

PACF Plot of Residuals

Autocorrelation Plot

Partial Autocorrelation Plot

1.0

1.0

0.5

0.5

Correlation

Correlation

Model: D(t) = 401.92 - 0.51 D(t-1) - 0.33 D(t-2) - 0.39 D(t-3) - 0.56 D(t-4)

0.0
-0.5
-1.0
0

10
Lag

15

20

0.0
-0.5
-1.0
0

10
Lag

15

20

The 3 Choices clearly indicate that ARIMA (4, 1, 0) is the appropriate choice, as its residuals are uncorrelated.
Compiled by Dr. B.S. KULKARNI

21

Step-4: Forecasting
The ARIMA (4, 1, 0) Model can be applied for forecasting. Since the Model involves D(t), the differenced
series, it has to be transformed back to original series with Y t. The transformed model and the forecasts are
as follows:
Transformed Model:
Y(t) = Y(t-1)+ 401.92 - 0.51 D(t-1) - 0.33 D(t-2) - 0.39 D(t-3) - 0.56 D(t-4)
Year

Observed

Forecast

Bias (%)

2002-03

7327

2003-04

8953

10143.21

-13.29

2004-05

9601

9266.10

3.49

2005-06

11704

11319.47

3.29

2006-07

11872

12437.88

-4.77

2007-08

10335.35
**** ******** ***

Compiled by Dr. B.S. KULKARNI