CHAPTER THREE

THE CLASSICAL REGRESSION ANALYSIS

[The Multiple Linear Regression Model]

3.1 Introduction

In simple regression we study the relationship between a dependent variable and a

single explanatory (independent variable). But it is rarely the case that economic
relationships involve just two variables. Rather a dependent variable Y can
depend on a whole series of explanatory variables or regressors. For instance, in
demand studies we study the relationship between quantity demanded of a good
and price of the good, price of substitute goods and the consumer’s income. The
model we assume is:
Y i =β 0 + β 1 P1 + β 2 P2 + β3 X i +ui -------------------- (3.1)

Where Y i = quantity demanded, P1 is price of the good, P2 is price of substitute

good, Xi is consumer’s income, and β ' s are unknown parameters and ui is the
disturbance.
Equation (3.1) is a multiple regression with three explanatory variables. In general
for K-explanatory variable we can write the model as follows:
Y i =β 0 +β 1 X 1 i +β 2 X 2 i +β 3 X 3 i +.. . .. .. . .+ β k X ki +ui ------- (3.2)
X k =( i=1 , 2 ,3 , .. . .. .. , K )
Where i are explanatory variables, Yi is the dependent

variable and β j ( j=0 ,1 ,2 ,. .. .(k+1 )) are unknown parameters and ui is the

disturbance term. The disturbance term is of similar nature to that in simple
regression, reflecting:
- the basic random nature of human responses
- errors of aggregation
- errors of measurement
- errors in specification of the mathematical form of the model

Multiple Linear Regression Analysis 1

and any other (minor) factors, other than x i that might influence Y.
In this chapter we will first start our discussion with the assumptions of the
multiple regressions and we will proceed our analysis with the case of two
explanatory variables and then we will generalize the multiple regression model in
the case of k-explanatory variables using matrix algebra.

3.2 Assumptions of Multiple Regression Model

In order to specify our multiple linear regression models and proceed our analysis
with regard to this model, some assumptions are compulsory. But these
assumptions are the same as in the single explanatory variable model developed
earlier except the assumption of no perfect multicollinearity. These assumptions
are:
1. Randomness of the error term: The variable u is a real random variable.

2. Zero mean of the error term: E(u i )=0

3. Homoscedasticity: The variance of each ui is the same for all the x i values.
E( u 2 )=σ 2
i.e. i u (constant)

4. Normality of u: The values of each ui are normally distributed.

2
i.e. U i ~ N (0 ,σ )

5. No auto or serial correlation: The values of ui (corresponding to Xi ) are

independent from the values of any other ui (corresponding to Xj ) for i j.

i.e. E(u i u j )=0 for x i≠ j

6. Independence of ui and Xi : Every disturbance term ui is independent of

the explanatory variables. i.e. E(u i X 1i )=E(u i X 2i )=0

This condition is automatically fulfilled if we assume that the values of the
X’s are a set of fixed numbers in all (hypothetical) samples.

Multiple Linear Regression Analysis 2

7. No perfect multicollinearity: The explanatory variables are not perfectly
linearly correlated.

We can’t exclusively list all the assumptions but the above assumptions are some
of the basic assumptions that enable us to proceed our analysis.

3.3 A Model with Two Explanatory Variables

In order to understand the nature of multiple regression model easily, we start our
analysis with the case of two explanatory variables, then extend this to the case of
k-explanatory variables.

3.3.1 Estimation of parameters of two-explanatory variables model

The model: Y = β0 + β 1 X 1 + β 2 X 2 +U i ……………………………………(3.3)

is multiple regression with two explanatory variables. The expected value of the
above model is called population regression equation i.e.
E(Y )=β 0 + β1 X 1 + β 2 X 2 , Since E(U i )=0 . …………………................(3.4)

where β i is the population parameters. β 0 is referred to as the intercept and β 1

and β 2 are also sometimes known as regression slopes of the regression. Note that,
β 2 for example measures the effect on E(Y ) of a unit change in X 2 when X 1 is

held constant.

Since the population regression equation is unknown to any investigator, it has to

be estimated from sample data. Let us suppose that the sample data has been used
to estimate the population regression equation. We leave the method of estimation
unspecified for the present and merely assume that equation (3.4) has been
estimated by sample regression equation, which we write as:
Y^ = β^ 0 + β^ 1 X 1 + β^ 2 X 2 ……………………………………………….(3.5)

Multiple Linear Regression Analysis 3

^
Where β j are estimates of the β j and Y^ is known as the predicted value of Y.

Now it is time to state how (3.3) is estimated. Given sample observation on

Y , X 1 ∧X 2 , we estimate (3.3) using the method of least square (OLS).

Y^ = β^ 0 + β^ 1 X 1i + β^ 2 X 2i + ei ……………………………………. (3.6)

is sample relation between Y , X 1 ∧X 2 .

e i=Y i−Y^ =Y i− β^ 0 − β^ 1 X 1 − β^ 2 X 2 …………………………………..(3.7)

To obtain expressions for the least square estimators, we partially differentiate

∑ e2i with respect to β^ 0 , β^ 1 and β^ 2 and set the partial derivatives equal to zero.

∂ [ ∑ e2i ]
=−2 ∑ (Y i − β^ 0 − β^ 1 X 1i − β^ 2 X 2 i )=0
∂ β^0 ………………………. (3.8)
∂ [ ∑ e2i ]
=−2 ∑ X 1 i ( Y i − β^ 0− β^ 1 X 1i− β^ 2 X 2i )=0
∂ β^
1 ……………………. (3.9)
∂ [ ∑ e2i ]
=−2 ∑ X 2 i ( Y i − β^ 0− β^ 1 X 1i − β^ 2 X 2i )=0
∂ β^
2 ………… ………..(3.10)
Summing from 1 to n, the multiple regression equation produces three Normal
Equations:

∑ Y =n β^ 0+ β^ 1 ΣX 1 i + β^ 2 ΣX 2 i …………………………………….(3.11)
∑ X 1 Y i = β^ 0 ΣX1 i + β^ 1 ΣX 21i + β^ 2 ΣX 2i X 1i …………………………(3.12)
∑ X 2i Y i = β^ 0 ΣX2 i + β^ 1 ΣX 1 i X 2i + β^ 2 ΣX 22i ………………………...(3.13)
^
From (3.11) we obtain β 0
β^ 0 =Ȳ − β^ 1 X̄ 1 − β^ 2 X̄ 2 ------------------------------------------------- (3.14)

Substituting (3.14) in (3.12), we get:

∑ X 1i Y i =( Ȳ − β^ 1 X̄ 1− β^ 2 X̄ 2 ) ΣX 1 i + β^ 1 ΣX 1 i2 + β^ 2 ΣX 2i

Multiple Linear Regression Analysis 4

⇒ ∑ X 1i Y i −Ȳ ΣX 1i = β^ 1 ( ΣX 1 i2 − X̄ 1 ΣX 2i )+ β^ 2 ( ΣX 1 i X 2i − X̄ 2 ΣX 2i )
X 1i Y i −n Ȳ X̄ 1i = β^ 1 ( ΣX )+ β^ 2 ( ΣX 1 i X 2 −n X̄ 1 X̄ 2 )
⇒∑
−n X̄
1i2 1 i2 ------- (3.15)
We know that

∑ ( X i −Y i )2=(ΣX i Y i−n X̄ i Y i )=Σxi y i

∑ ( X i − X̄ i ) 2=( ΣX i2 −n X̄ i2 )=Σxi2
Substituting the above equations in equation (3.14), the normal equation (3.12) can
be written in deviation form as follows:
∑ x 1 y = β^ 1 Σx12 + β^ 2 Σx1 x 2 …………………………………………(3.16)
Using the above procedure if we substitute (3.14) in (3.13), we get
∑ x 2 y = β^ 1 Σx 1 x 2 + β^ 2 Σx 22 ………………………………………..(3.17)
Let’s bring (3.16) and (3.17) together
∑ x 1 y = β^ 1 Σx12 + β^ 2 Σx1 x 2 ……………………………………….(3.18)
∑ x 2 y = β^ 1 Σx 1 x 2 + β^ 2 Σx 22 ……………………………………….(3.19)
β^ 1 and β^ 2 can easily be solved using matrix

We can rewrite the above two equations in matrix form as follows.

∑ x 12 ∑ x1 x2 β^ 1 = ∑ x1 y ………….(3.20)
∑ x1 x2 ∑ x 22 β^ 2 ∑ x2 y
If we use Cramer’s rule to solve the above matrix we obtain
Σx1 y . Σx 2−Σx1 x 2 . Σx2 y
β^ 1=
2

Σx 2 . Σx 2 −Σ( x 1 x2 )2
1 2 …………………………..…………….. (3.21)
Σx2 y . Σx 2−Σx1 x 2 . Σx1 y
β^ 2 =
1

Σx 2 . Σx 2 −Σ( x 1 x 2 )2
1 2 ………………….……………………… (3.22)
^ ^
We can also express β 1 and β 2 in terms of covariance and variances of
Y , X 1 and X 2

Multiple Linear Regression Analysis 5

Cov ( X 1 , Y ) . Var ( X 1 )−Cov( X 1 , X 2 ) . Cov ( X 2 , Y )
β^ 1= −−−−−−−−−(3. 23 )
Var ( X 1 ). Var ( X 2 )−[cov ( X 1 , X 2 )]2
Cov ( X 2 , Y ) . Var ( X 1 )−Cov( X 1 , X 2 ) . Cov ( X 1 , Y )
β^ 2 = −−−−−−−−−(3. 24 )
Var ( X 1 ). Var ( X 2 )−[ Cov( X 1 , X 2 )] 2

3.3.2 The coefficient of determination (R2): two explanatory variables case

In the simple regression model, we introduced R2 as a measure of the proportion of
variation in the dependent variable that is explained by variation in the explanatory
variable. In multiple regression model the same measure is relevant, and the same
formulas are valid but now we talk of the proportion of variation in the dependent
variable explained by all explanatory variables included in the model. The
coefficient of determination is:
Σe 2
2 ESS RSS i
R = =1− =1−
TSS TSS Σy 2
i ------------------------------------- (3.25)
In the present model of two explanatory variables:
Σe2i =Σ ( y i− β^ 1 x 1i − β^ 2 x 2 i )2

=Σe i ( y i− β^ 1 x 1i − β^ 2 x 2i )

=Σe i y − β^ 1 Σx 1i e i− β^ 2 Σei x 2i
=Σe i y i since Σei x 1i=Σei x 2i =0

=Σy i ( y i − β^ 1 x 1 i− β^ 2 x 2i )

i. e Σe2i =Σy2 − β^ 1 Σx 1i y i − β^ 2 Σx 2i y i

otalsumof ¿ var iation) ¿¿=β^ 1 Σx1i yi+ β^ 2 Σx2i yi underbracealignl ⏟

⇒Σy2underbracealignlT⏟ Explained sumof ¿ var iation) ¿¿¿+Σe 2 underbracealignl⏟
Residualsumof squares ¿ ¿¿¿¿
i
square (Total ¿ square (Explained ¿ (unexplained var iation)¿ ----------------- (3.26)

ESS β^ 1 Σx 1i y i + β^ 2 Σx2 i y i
2
∴ R= =
TSS Σy2 ----------------------------------(3.27)
As in simple regression, R2 is also viewed as a measure of the prediction ability of
the model over the sample period, or as a measure of how well the estimated
regression fits the data. The value of R2 is also equal to the squared sample

Multiple Linear Regression Analysis 6

^
correlation coefficient between Y ∧Y t . Since the sample correlation coefficient
measures the linear association between two variables, if R 2 is high, that means

there is a close association between the values of Y t and the values of predicted by
^
the model, Y t . In this case, the model is said to “fit” the data well. If R 2 is low,

there is no association between the values of Y t and the values predicted by the
^
model, Y t and the model does not fit the data well.

2
3.3.3 Adjusted Coefficient of Determination ( R̄ )
2
One difficulty with R is that it can be made large by adding more and more
variables, even if the variables added have no economic justification.
Algebraically, it is the fact that as the variables is added the sum of squared errors
2
(RSS) goes down or it can remain unchanged, but this is rare, and thus R goes up.
2
If the model contains n-1 variables then R =1. The manipulation of model just to
2
obtain a high R is not wise. An alternative measure of goodness of fit, called the
2 2
adjusted R and often symbolized as R̄ , is usually reported by regression
programs. It is computed as:
Σe2i / n−k
2
R̄ =1− 2
Σy / n−1
=1−( 1−R 2 ) ( n−k
n−1
)--------------------------------(3.28)
This measure does not always goes up when a variable is added because of the
degree of freedom term n-k is the numerator. As the number of variables k
2
increases, RSS goes down, but so does n-k. The effect on R̄ depends on the
2
amount by which R falls. While solving one problem, this corrected measure of
2
goodness of fit unfortunately introduces another one. It loses its interpretation; R̄
2
is no longer the percent of variation explained. This modified R̄ is sometimes
used and misused as a device for selecting the appropriate set of explanatory
variables.

Multiple Linear Regression Analysis 7

3.4. General Linear Regression Model and Matrix Approach

So far we have discussed the regression models containing one or two explanatory
variables. Let us now generalize the model assuming that it contains k variables.
It will be of the form:
Y = β0 + β 1 X 1 + β 2 X 2 +. .. .. .+ β k X k +U
There are k parameters to be estimated. The system of normal equations consist of

k+1 equations, in which the unknowns are the parameters β 0 , β 1 , β2 . .. . .. . βk and the
known terms will be the sums of squares and the sums of products of all variables
in the structural equations.
Least square estimators of the unknown parameters are obtained by minimizing
the sum of the squared residuals.
Σe2i =Σ ( y i− β^ 0 − β^ 1 X 1 − β^ 2 X 2 −.. . .. .− β^ k X k )2

With respect to β j ( j=0,1,2,....(k+1))

The partial derivations are equated to zero to obtain normal equations.
∂ Σe2i
=−2 Σ( Y i − β^ 0− β^ 1 X 1− β^ 2 X 2−. .. . ..− β^ k X k )=0
∂β ^
0

∂ Σe2i
=−2 Σ(Y i − β^ 0− β^ 1 X 1− β^ 2 X 2−. .. . ..− β^ k X k )( x i )=0
∂β ^
1

……………………………………………………..
∂ Σe2i
=−2 Σ(Y i − β^ 0− β^ 1 X 1− β^ 2 X 2−. .. . ..− β^ k X k )( x ki )=0
∂ β^ k

The general form of the above equations (except first ) may be written as:
∂ Σe2i
=−2 Σ(Y i − β^ 0− β^ 1 X 1i−−−−−− β^ k X ki )=0
∂ β^ j ; where ( j=1,2,....k )
The normal equations of the general linear regression model are
ΣY i =n β^ 0 + β^ 1 ΣX 1i + β^ 2 ΣX 2i +. .. .. . .. .. . .. .. . .. .. . .. .. . .. .. .+ β^ k ΣX ki
ΣY i X 1i = β^ 0 ΣX 1 i + β^ 1 ΣX +.. . .. .. . .. .. . .. .. . .. .. . .. .. .. . .. .+ β^ k ΣX 1 i X ki
1i 2

Multiple Linear Regression Analysis 8

ΣY i X 2i = β^ 0 ΣX 2 i + β^ 1 ΣX 1i X 2i + β^ 2 ΣX + .. .. . .. .. .+ β^ k ΣX 2i X ki
2i2

: : : : :
: : : : :
ΣY i X ki= β^ 0 ΣX ki + β^ 1 ΣX 1 X ki +∑ X 2 i X ki .. .. . .. .. . .. .. . .. .+ β^ k ΣX
i ki 2

Solving the above normal equations will result in algebraic complexity. But we
can solve this easily using matrix. Hence in the next section we will discuss the
matrix approach to linear regression model.

3.4.1 Matrix Approach to Linear Regression Model

The general linear regression model with k explanatory variables is written in the

form: Y i =β 0 +β 1 X 1 i +β 2 X 2 i +.. .. . .. .. . .. .+β k X ki +U

i

where (i=1,2,3,........n) and β 0 = the intercept, β 1 to β k = partial slope coefficients

U= stochastic disturbance term and i=ith observation, ‘n’ being the size of the
observation. Since i represents the ith observation, we shall have ‘n’ number of
equations with ‘n’ number of observations on each variable.
Y 1 =β 0 +β 1 X 11 + β 2 X 21+ β3 X 31 . .. . .. .. . .. ..+β k X k 1 +U 1
Y 2 =β 0 +β 1 X 12 +β 2 X 22 +β 3 X 32 .. .. . .. .. . .. .+β k X k 2 +U 2
Y 3 =β 0 +β 1 X 13+β 2 X 23+β 3 X 33 . .. . .. .. . .. ..+β k X k 3 +U 3
…………………………………………………...
Y n =β 0 + β1 X 1n + β2 X 2n +β 3 X 3 n . .. .. . .. .. . ..+β k X kn +U n

These equations are put in matrix form as:

Multiple Linear Regression Analysis 9

[] [ ][ ] [ ]
Y1 1 X 11 X 21 . . .. .. . Xk 1 β0 U1
Y2 1 X 12 X 22 . . .. .. . Xk 2 β1 U2
Y3 = 1 X 13 X 23 . . .. .. . Xk 3 β2 + U3
. . . . . . .. .. . . . .
Yn 1 X 1n X2 n . . .. .. . X kn βn Un
Y = X . β + U

In short Y = Xβ+U ……………………………………………………(3.29)

The order of matrix and vectors involved are:
Y =(n×1), X= {(n×(k +1 ) } , β={ (k +1)×1 } and U=(n×1 )

To derive the OLS estimators of β , under the usual (classical) assumptions

^
mentioned earlier, we define two vectors β and ‘e’ as:

[] []
β^ 0 e1
^
β e2
1
^
β= and e= .
.
. .
^
β en
k

^ ^
Thus we can write: Y = X β+e and e=Y −X β
We have to minimize:
n
∑ e2i =e 21+ e22+ e23 +.. . .. .. . .+ e2n
i=1

[]
e1
e2
=[ e1 , e 2 .. . .. . en ] . = e'e
.
en

=∑ e2i =e ' e

e' e=(Y −X β^ )' (Y − X β^ )

^ X ' Y −Y ' X β+
=YY '− β' ^ β^ ' X ' X β^
………………….…(3.30)
^
Since β' X ' Y ' is scalar (1x1), it is equal to its transpose;

Multiple Linear Regression Analysis 10

β^ ' X ' Y =Y ' X β^
e' e=Y ' Y −2 β' ^ X ' Y + β^ ' X ' X β^
-------------------------------------(3.31)
^
Minimizing e’e with respect to the elements in β
∂ Σe2i ∂(e ' e )
= =−2 X ' Y +2 X ' X β^
∂ β^ ∂ β^
∂( X ' AX )
=2 AX
Since ∂ β^ and also too 2X’A
Equating the expression to null vector 0, we obtain:
^
−2 X ' Y +2 X ' X β=0 ^
 X ' X β= X ' Y
^ X ' X )−1 X ' Y
β=( ………………………………. ………. (3.32)
^ β^ , β^ , β^ ,. .. . .. .. β^
Hence β is the vector of required least square estimators 0 1 2 k.

3.4.2. Statistical Properties of the Parameters (Matrix) Approach

^
We have seen, in simple linear regression that the OLS estimators ( α^ ∧ β ) satisfy
the small sample property of an estimator i.e. BLUE property. In multiple
regression, the OLS estimators also satisfy the BLUE property. Now we proceed
to examine the desired properties of the estimators in matrix notations:
1. Linearity
^ X ' X )−1 X ' Y
β=(
We know that:
'−1 '
Let C=( X X ) X

⇒ β^ =CY …………………………………………….(3.33)

Since C is a matrix of fixed variables, equation (3.33) indicates us β^ is linear in Y.

2. Unbiased ness
^ X ' X )−1 X ' Y
β=(
^ X ' X )−1 X ' ( Xβ+U )
β=(

Multiple Linear Regression Analysis 11

^ +( X ' X )−1 X ' U
β=β …….……………………………... (3.34)

Since [ ( X ' X ) X ' X=I ]

−1

Ε( β^ )=Ε { β +( X ' X )−1 X ' U }

=Ε( β )+ Ε [ ( X ' X )−1 X ' U ]

=β+ Ε( X ' X )−1 X ' Ε(U )

=β , Since Ε(U )=0
Thus, least square estimators are unbiased.

3. Minimum variance
Before showing all the OLS estimators are best (possess the minimum variance
property), it is important to derive their variance.
We know that, var( β )=Ε [ ( β−β ) ] =Ε [ ( β−β )( β−β )' ]
^ ^ 2 ^ ^

Ε [( β−
^ β )( β−β
^ )' ]=

[ ]
Ε ( β^ 1−β 1 )2 Ε [ ( β^ 1−β 1 )( β^ 2−β 2 ) ] .. .. . .. Ε [( β^ 1 −β 1 )( β^ k −β k ) ]
Ε [ ( β^ 2 −β 2 )( β^ 1 −β 1 ) ] Ε( β^ −β ) Ε [( β^ −β )( β^ −β ) ]
2
2 2 .. .. . .. 2 2 k k
: : :
: : :
Ε [( β^ k −β k )( β^ 1−β 1 ) ] Ε [ ( β^ k−β k )( β^ 2 −β 2 ) ] .. .. . .. . Ε ( β^ k−β k )2

[ ]
var ( β^ 1 ) cov ( β^ 1 , β^ 2 ) . . .. .. . cov ( β^ 1 , β^ k )
cov ( β^ 2 , β^ 1 ) var ( β^ 2 ) . . .. .. . cov ( β^ , β^ )
2 k
=: : :
: : :
cov ( β^ k , β^ 1 ) cov ( β^ k , β^ 2 ) . . .. .. . var ( β^ k )

The above matrix is a symmetric matrix containing variances along its main
diagonal and covariance of the estimators everywhere else. This matrix is,
therefore, called the Variance-covariance matrix of least squares estimators of the
regression slopes. Thus,
var ( β^ )=Ε [ ( β−β
^ ^
)( β−β )' ] ……………………………………………(3.35)

Multiple Linear Regression Analysis 12

^ −1
From (3.15) β=β +( X ' X ) X ' U
⇒ β^ −β=( X X )−1 X U ………………………………………………(3.36)
' '

Substituting (3.17) in (3.16)

var( β^ )=Ε [ {( X ' X )−1 X ' U }{( X ' X )−1 X ' U } ' ]

var( β^ )=Ε [ ( X ' X )−1 X ' UU ' X ( X ' X )−1 ]

=( X ' X )−1 X ' Ε(UU ' ) X ( X ' X )−1

=( X ' X )−1 X ' σ 2u I n X ( X ' X )−1

=σ 2u ( X ' X )−1 X ' X ( X ' X )−1

^ 2 −1
var( β ) =σ u ( X ' X ) ………………………………………….……..(3.37)

2
Note: (σ u being a scalar can be moved in front or behind of a matrix while

identity matrix I n can be suppressed).

^ 2 −1
Thus we obtain, var( β )=σ u ( X ' X )

[ ]
n ΣX 1n . .. .. . . ΣX kn
2
1n
ΣX 1 n ΣX . .. .. . . ΣX 1 n X kn
: : :
: : :
2
−1 kn
ΣX kn ΣX 1n X kn . .. .. . . ΣX
Where, ( X ' X ) =
^
We can, therefore, obtain the variance of any estimator say β 1 by taking the ith term
−1 2
from the principal diagonal of ( X ' X ) and then multiplying it by σ u .
Where, the X’s are in their absolute form. When the x’s are in deviation form we
can write the multiple regression in matrix form as ;
^ x' x )−1 x ' y
β=(

Multiple Linear Regression Analysis 13

[ ]
β^ 1 2
∑x1 Σx 1 x 2 .. . .. .. Σx 1 x k
β^ 2 Σx 2 x 1 Σx 2 .. . .. .. Σx 2 x k
2
: : : :
: : : :
^ ^ Σx n x1 Σx n x2 .. . .. .. Σx
where β = β k
' 2
and ( x x )= k

^ ^
The above column matrix β doesn’t include the constant term β 0 .Under such
conditions the variances of slope parameters in deviation form can be written as:
var( β^ )=σ 2u (x ' x )−1 …………………………………………………….(2.38)

(the proof is the same as (3.37) above). In general we can illustrate the variance of
the parameters by taking two explanatory variables.
The multiple regressions when written in deviation form that has two explanatory
variables is;
y 1 = β^ 1 x 1 + β^ 2 x 2

var( β^ )=Ε [ ( β−β

^ ^
)( β−β )' ]

^
( β−β)=¿ [( β^ 1−β1 ) ¿] ¿ ¿¿
In this model; ¿
^
( β−β ) ' =[( β^ 1 −β 1 )( β^ 2 −β 2 ) ]

)'=¿ [ ( β1 −β 1 ) ¿ ] ¿ ¿ ¿
^ ^ ^
∴ ( β−β )( β−β
¿
]= Εalignl [( β^ 1−β1 ) ( β^ 1−β 1)( β^ 2−β 2 ) ¿ ] ¿ ¿¿
2
Ε [( β−
^ β)( β−β)'
^
and ¿

=
[ var ( β^ 1 ) cov ( β^ 1 , β^ 2 )
cov ( β^ 1 , β^ 2 ) var( β^ 2 ) ]
In case of two explanatory variables, x in the deviation form shall be:

Multiple Linear Regression Analysis 14

[ ]
x 11 x 21
x= x 12
:
x1 n
x 22
:
x 2n
and x '=
[ x 11
x 12
x 12 .. .. . .. x 1n
x 22 . . .. .. . x 2n ]

[ ]
−1
−1 Σx21 Σx1 x 2
∴ σ 2u ( x ' x) =σ 2u
Σx1 x 2 Σx22

σ 2u ( x ' −1
x) =
σ 2u
[ Σx 22
−Σx 1 x 2
−Σx 1 x 2
Σx 21 ]
Σx 21 Σx 1 x 2
| |
Or Σx 1 x 2 Σx 22

σ 2u Σx22
var ( β^ 1 )=
i.e., Σx 12 Σx 22−( Σx 1 Σx 2 )2 ……………………………………(3.39)

σ 2u Σx 12
var ( β^ 2 )=
and, Σx 21 Σx 22−( Σx 1 Σx 2 )2 ………………. …….…….(3.40)

(−) σ 2u Σx 1 x 2
cov ( β^ 1 , β^ 2 )=
Σx 21 Σx 22−( Σx 1 Σx 2 )2 …………………………………….

(3.41)
2
The only unknown part in variances and covariance of the estimators is σ u .

As we have seen in simple regression modelσ^ = n−2 . For k-parameters

2 { }Σe2i

(including the constant parameter)σ^ = n−k .

2 { } Σe 2i

In the above model we have three parameters including the constant term and

σ^ 2= n−3
{ } Σe i2

∑ ei 2=∑ y i 2−β 1 ∑ x 1 y−β 2 ∑ x 2 y .. . .. .. . .+ β K ∑ x K y ………………………(3.42)

this is for k explanatory variables. For two explanatory variables

Multiple Linear Regression Analysis 15

∑ ei 2=∑ y i 2−β 1 ∑ x 1 y−β 2 ∑ x 2 y ………………………………………...(3.43)
This is all about the variance covariance of the parameters. Now it is time to see
the minimum variance property.

^
Minimum variance of β
^
To show that all the β i ' s in the β vector are Best Estimators, we have also to
prove that the variances obtained in (3.37) are the smallest amongst all other
possible linear unbiased estimators. We follow the same procedure as followed in
case of single explanatory variable model where, we first assumed an alternative
linear unbiased estimator and then it was established that its variance is greater
than the estimator of the regression model.

^ ^
Assume that β is an alternative unbiased and linear estimator of β . Suppose that
^^
β= [ ( X ' X )−1 X ' +B ] Y
Where B is (k x n) matrix of known constants.
^^
∴ β= [( X ' X )−1 X '+B ] [ Xβ+U ]
^^
β=( X ' X )−1 X ' ( Xβ+U )+B( Xβ +U )
^
Ε( β^ )=Ε [( X ' X )−1 X ' ( Xβ+U )+B ( Xβ+U ) ]
=Ε [ ( X ' X )−1 X ' Xβ+( X ' X )−1 X ' U +BX β +BU ]
=β + BX β , [since E(U) = 0].……………………………….(3.44)

^ ^
Since our assumption regarding an alternative β is that it is to be an unbiased
^ ^
estimator of β , therefore, Ε( β ) should be equal to β ; in other words ( β XB) should
be a null matrix.
^
Thus we say, BX should be =0 if ( β )= [( X ' X ) X '+B ] Y is to be an unbiased
^ −1

estimator. Let us now find variance of this alternative estimator.

Multiple Linear Regression Analysis 16

^^
var( β)=Ε [
^^
( β−β
^^
)( β−β)' ]
=Ε [ {[ ( X ' X )
−1
X ' + B ] Y −β }{ [( X ' X )−1 X ' + B ] Y −β } ' ]
=Ε [ {[ ( X ' X )−1
X ' + B ] ( Xβ +U )−β }{[ ( X ' X )−1 X ' + B ] ( Xβ +U )− β } ' ]
=Ε [ {( X ' X )−1 X ' Xβ +( X ' X )−1 X ' U + BX β + BU −β }
{( X ' X )−1 X ' Xβ+( X ' X )−1 X ' U + BX β + BU −β } ' ¿
¿

=Ε [ {( X ' X )−1 X ' U +BU }{( X ' X )−1 X ' U +BU } ' ¿ ¿
( ∵ BX=0)
=Ε [ {( X ' X )−1 X ' U + BU }{U ' X ( X ' X )−1 +U ' B ' }]

=Ε [ {( X ' X )−1 B } UU ' { X ( X ' X )−1 +U ' B ' } ]

=[ ( X ' X )−1 X '+B ] Ε(UU ' ) [ X ( X ' X )−1 +B ' ]

=σ 2u I n [( X ' X )−1 X '+B ][ X ( X ' X )−1 +B' ]

=σ 2u [ ( X ' X )−1 X ' X ( X ' X )−1 + BX ( X ' X )−1 +( X ' X )−1 ]

=σ 2u [ ( X ' X )−1 X ' X ( X ' X )−1 +BX ( X ' X )−1 +( X ' X )−1 X ' B' +BB' ]

=σ 2u [ ( X ' X )−1 +BB ' ] ( ∵ BX=0 )

^
var( β^ ) =σ 2u ( X ' X )−1 + σ 2u BB ' ……………………………………….(3.45)

^ ^ ^ 2
Or, in other words, var( β ) is greater than var( β ) by an expression σ u BB ' and it
^
proves that β is the best estimator.

3.4.3. Coefficient of Determination in Matrix Form

2
The coefficient of determination( R ) can be derived in matrix form as follows.
2 ^ ^ ^ ^
We know that Σei =e ' e=Y ' Y −2 β ' X ' Y + β ' X ' X β since ( X ' X ) β= X ' Y and
∑ Y i2=Y ' Y
∴ e' e=Y ' Y −2 β^ ' X ' Y + β'
^ X'Y

e' e=Y ' Y − β^ ' X ' Y ……………………………………...……..(3.46)

Multiple Linear Regression Analysis 17

^ X ' Y = e ' e−Y ' Y
β' ……………………………………………….(3.47)

We know, y i =Y i−Ȳ
1
∴ Σy 2i =ΣY 2i − ( ΣY i )2
n
In matrix notation
1
Σy2i =Y ' Y − ( ΣY i )2
n ………………………………………………(3.48)
Equation (3.48) gives the total sum of squares variations in the model.
2 2
Explained sum of squares=Σy i −Σe i
1
=Y ' Y − ( Σy )2 −e ' e
n
1
= β^ ' X ' Y − ( ΣY i )2
n ……………………….(3.49)
Explained sum of squares
R2 =
Since Total sum of squares
1
β^ ' X ' Y − (ΣY i )2 ^
n β ' X ' Y −n Ȳ
∴ R 2= =
1 Y ' Y −n Ȳ 2
Y ' Y − ( ΣY i )2
n ……………………(3.50)

Dear Students! We hope that from the discussion made so far on multiple
regression model, in general, you may make the following summary of results.

(i) Model: Y = Xβ+U

^ X ' X )−1 X ' Y
β=(
(ii) Estimators:
(iii) Statistical properties: BLUE

(iv) Variance-covariance: var( β^ )=σ 2u ( X ' X )−1

(v) Estimation of (e’e): e' e=Y ' Y − β^ ' X ' Y

Multiple Linear Regression Analysis 18

1
β^ ' X ' Y − ( ΣY i )2
n
R2 =
1 β^ ' X ' Y −n Ȳ
Y ' Y − ( ΣY i )2 =
(vi) Coeff. of determination: n Y ' Y −n Ȳ

3.5. Hypothesis Testing in Multiple Regression Model

In multiple regression models we will undertake two tests of significance. One is

significance of individual parameters of the model. This test of significance is the
same as the tests discussed in simple regression model. The second test is overall
significance of the model.
3.5.1. Tests of individual significance
2
If we invoke the assumption that U i ~. N (0 , σ ) , then we can use either the t-test or
standard error test to test a hypothesis about any individual partial regression
coefficient. To illustrate consider the following example.

^ ^ ^
Let Y = β0 + β 1 X 1 + β 2 X 2 +e i ………………………………… (3.51)

A. H 0 : β 1 =0
H 1 : β 1 ≠0

B. H 0 : β 2 =0
H 1 : β 2 ≠0

The null hypothesis (A) states that, holding X2 constant X1 has no (linear)
influence on Y. Similarly hypothesis (B) states that holding X1 constant, X2 has no
influence on the dependent variable Yi.To test these null hypothesis we will use
the following tests:
i- Standard error test: under this and the following testing methods we
^ ^
test only for β 1 .The test for β 2 will be done in the same way.

Multiple Linear Regression Analysis 19

SE( β^ 1 )=√ var( β^ 1 )=

^ 1 ^
√ ∑ ∑
x12 i
σ^ 2 ∑ x 22i
x 22i −( ∑ x1 x 2)
2
; where
σ^ 2=
Σe 2i
n−3

 If SE( β 1 )> 2 β1 , we accept the null hypothesis that is, we can

conclude that the estimate β i is not statistically significant.
^ 1 ^
 If SE( β 1 < 2 β 1 , we reject the null hypothesis that is, we can

conclude that the estimate β i is statistically significant.

Note: The smaller the standard errors, the stronger the evidence that the estimates
are statistically reliable.
^
ii. The student’s t-test: We compute the t-ratio for each β i
β^ i−β
t∗¿ ~ t n-k
SE ( β^ )
i , where n is number of observation and k is number of
parameters. If we have 3 parameters, the degree of freedom will be n-3. So;
β^ 2 −β2
t∗¿
SE ( β^ )
2 ; with n-3 degree of freedom

In our null hypothesis β 2 =0 , the t* becomes:

β^ 2
t∗¿
SE ( β^ 2 )

 If t*<t (tabulated), we accept the null hypothesis, i.e. we can conclude

^
that β 2 is not significant and hence the regressor does not appear to
contribute to the explanation of the variations in Y.
 If t*>t (tabulated), we reject the null hypothesis and we accept the
^
alternative one; β 2 is statistically significant. Thus, the greater the value

of t* the stronger the evidence that β i is statistically significant.

3.5.2 Test of Overall Significance

Multiple Linear Regression Analysis 20

Throughout the previous section we were concerned with testing the significance
of the estimated partial regression coefficients individually, i.e. under the separate
hypothesis that each of the true population partial regression coefficients was zero.

In this section we extend this idea to joint test of the relevance of all the included
explanatory variables. Now consider the following:
Y = β0 +β 1 X 1 +β 2 X 2 +. .. .. . .. .+β k X k +U i
H 0 : β 1 =β 2=β 3 =. .. .. . .. .. . .=β k =0
H 1 : at least one of the β k is non-zero

This null hypothesis is a joint hypothesis that β 1 , β 2 ,........ β k are jointly or

simultaneously equal to zero. A test of such a hypothesis is called a test of overall
significance of the observed or estimated regression line, that is, whether Y is

linearly related to X 1 , X 2 ,. ..... .. X k .

Can the joint hypothesis be tested by testing the significance of individual
^
significance of β i ’s as the above? The answer is no, and the reasoning is as
follows.

In testing the individual significance of an observed partial regression coefficient,

we assumed implicitly that each test of significance was based on different (i.e.
^
independent) sample. Thus, in testing the significance of β 2 under the hypothesis

that β 2 =0 , it was assumed tacitly that the testing was based on different sample
^
from the one used in testing the significance of β 3 under the null hypothesis that
β 3 =0 . But to test the joint hypothesis of the above, we shall be violating the

assumption underlying the test procedure.

Multiple Linear Regression Analysis 21

“…..testing a series of single (individual) hypothesis is not equivalent to testing
those same hypothesis. The institutive reason for this is that in a joint test of
several hypotheses any single hypothesis is affected by the information in the
other hypothesis.”1

The test procedure for any set of hypothesis can be based on a comparison of the
sum of squared errors from the original, the unrestricted multiple regression
model to the sum of squared errors from a regression model in which the null
hypothesis is assumed to be true. When a null hypothesis is assumed to be true,
we in effect place conditions or constraints, on the values that the parameters can
take, and the sum of squared errors increases. The idea of the test is that if these
sum of squared errors are substantially different, then the assumption that the joint
null hypothesis is true has significantly reduced the ability of the model to fit the
data, and the data do not support the null hypothesis.

If the null hypothesis is true, we expect that the data are compliable with the
conditions placed on the parameters. Thus, there would be little change in the sum
of squared errors when the null hypothesis is assumed to be true.

Let the Restricted Residual Sum of Square (RRSS) be the sum of squared errors
in the model obtained by assuming that the null hypothesis is true and URSS be
the sum of the squared error of the original unrestricted model i.e. unrestricted
residual sum of square (URSS). It is always true that RRSS - URSS¿ 0.

^ ^ ^ ^ ^
Consider Y = β0 + β 1 X 1 + β 2 X 2 +. .. .. . .. .+ β k X k +e i .
This model is called unrestricted. The test of joint hypothesis is that:
H 0 : β 1 =β 2=β 3 =. .. .. . .. .. . .=β k =0

1
Gujurati, 3rd ed.pp

Multiple Linear Regression Analysis 22

H 1 : at least one of the β k is different from zero.

^ ^ ^ ^ ^
We know that: Y = β0 + β 1 X 1i + β 2 X 2i +.. .. .. . ..+ β k X ki
Y i =Y^ +e
e i=Y i−Y^ i

Σe2i =Σ (Y i−Y^ i )2

This sum of squared error is called unrestricted residual sum of square (URSS).
This is the case when the null hypothesis is not true. If the null hypothesis is
assumed to be true, i.e. when all the slope coefficients are zero.
Y = β^ 0 +e i

β^ 0 =
∑ Y i =Ȳ →
n (applying OLS)…………………………….(3.52)
e=Y − β^ 0 ^
but β 0 =Ȳ
e=Y − β^
Σe2i =Σ (Y i−Y^ i )2 =Σy2 =TSS
The sum of squared error when the null hypothesis is assumed to be true is called
Restricted Residual Sum of Square (RRSS) and this is equal to the total sum of
square (TSS).
RRSS−URSS / K−1
~ F (k −1, n−k )
The ratio: URSS /n−K ……………………… (3.53);
(has an F-ditribution with k-1 and n-k degrees of freedom for the numerator and denominator respectively)
RRSS=TSS
URSS=Σe2i =Σy2 − β^ 1 Σ yx 1 − β^ 2 Σ yx 2 +.. .. . .. .. . β^ k Σ yx k =RSS
(TSS−RSS)/ k−1
F=
RSS /n−k
ESS/ k −1
F=
RSS / n−k ………………………………………………. (3.54)

Multiple Linear Regression Analysis 23

2
If we divide the above numerator and denominator by Σy =TSS then:
ESS
/ k −1
TSS
F=
RSS
/ k −n
TSS
R 2 / k −1
F=
1−R2 / n−k …………………………………………..(3.55)

This implies the computed value of F can be calculated either as a ratio of ESS &
TSS or R2 & 1-R2. If the null hypothesis is not true, then the difference between
RRSS and URSS (TSS & RSS) becomes large, implying that the constraints
placed on the model by the null hypothesis have large effect on the ability of the
model to fit the data, and the value of F tends to be large. Thus, we reject the null
hypothesis if the F test static becomes too large. This value is compared with the
critical value of F which leaves the probability of α in the upper tail of the F-
distribution with k-1 and n-k degree of freedom.

If the computed value of F is greater than the critical value of F (k-1, n-k), then the
parameters of the model are jointly significant or the dependent variable Y is
linearly related to the independent variables included in the model.

Application of Multiple Regression.

In order to help you understand the working of matrix algebra in the estimation of
the regression coefficient, variance of the coefficients and testing of the
parameters and the model, consider the following numerical example.
Example 1. Consider the data given in Table 2.1 below to fit a linear function:

Y=α + β 1 X 1 + β 2 X 2 + β 3 X 3 +U

Multiple Linear Regression Analysis 24

Table: 2.1. Numerical example for the computation of the OLS estimators.
n Y X1 X2 X3 yi x1 x2 x3 y 2i x1 x2 x2 x3 x1 x3 x 21 x 22 x 23 x1 yi x2 yi x3 yi
1 49 35 53 200 -3 -7 -9 0 9 63 0 0 49 81 0 21 27 0
2 40 35 53 212 -12 -7 -9 12 144 63 -108 -84 49 81 144 84 108 -144
3 41 38 50 211 -11 -4 -12 11 121 48 -132 -44 16 144 121 44 132 -121
4 46 40 64 212 -6 -2 2 12 36 -4 24 -24 4 4 144 12 -12 -72
5 52 40 70 203 0 -2 8 3 0 -16 24 -6 4 64 9 0 0 0
6 59 42 68 194 7 0 6 -6 49 0 -36 0 0 36 36 0 42 -42
7 53 44 59 194 1 2 -3 -6 1 -6 18 -12 4 9 36 2 -3 -06
8 61 46 73 188 9 4 11 -12 81 44 -132 -48 16 121 144 36 99 -108
9 55 50 59 196 3 8 -3 -4 9 -24 12 -32 64 9 16 24 -9 -12
1 64 50 71 190 12 8 9 -10 144 72 -90 -80 64 81 100 96 108 -120
0
520 420 620 2000
Σyi=0

Σx1=0

Σx2=0

Σx3=0

Σyi2=594

Σx1x2=240

Σx12=270

Σx22=630

Σx32=750

Σx3yi=319

Σx2yi=492
Σx2x3=-420

Σx1x3=-330

Σx3yi=-625
From the table, the means of the variables are computed and given below:

Ȳ =52 ; X̄ 1 =42 ; X̄ 2 =62; X̄ 3 =200

Multiple Linear Regression Analysis 25

Based on the above table and model answer the following question.
i. Estimate the parameter estimators using the matrix approach
ii. Compute the variance of the parameters.
iii. Compute the coefficient of determination (R2)
iv. Report the regression result.
Solution:
^ −1
In the matrix notation: β=( x' x) x ' y ; (when we use the data in deviation form),

[] [ ]
β^ 1 x 11 x21 x 31
^ β^ , x= x 12
β=
x22 x 32
2
^β : : :
3 x1 n x 2n x3 n
Where, ; so that

[ ] [ ]
Σx21 Σx 1 x2 Σx1 x 3 Σx1 y
2
( x ' x )= Σx1 x 2 Σx 1 Σx2 x 3 and x ' y= Σx2 y
2 Σx 3 y
Σx1 x 3 Σx 2 x3 Σx3

(i) Substituting the relevant quantities from table 2.1 we have;

[ ] [ ]
270 240 −330 319
( x' x )= 240 630 −420 and x ' y= 492
−330 −420 750 −625
Note: the calculations may be made easier by taking 30 as common factor from
all the elements of matrix (x’x). This will not affect the final results.

[ ]
270 240 −330
|x' x|= 240 630 −420 =4716000
−330 −420 750

[ ]
0.0085 −0.0012 0.0031
−1
( x' x ) = −0.0012 0.0027 0.0009
0.0031 0.0009 0.0032

Multiple Linear Regression Analysis 26

[] [ ][ ] [ ]
β^ 1 0 .0085 −0 .0012 0.0031 319 0 .2063
^ β^ =( x ' x )−1 x ' y=
β= 2 −0 .0012 0. 0027 0.0009 492 = 0 .3309
^β 0 .0031 0. 0009 0.0032 −625 −0 .5572
3

And
α=Ȳ − β^ 1 X̄ 1− β^ 2 X̄ 2− β^ 3 X̄ 3
=52−(0 . 2063)( 42)−(0 .3309 )(62)−(−0 .5572 )(200)
=52−8 .6633−20 .5139+111. 4562=134 .2789

−1 2
(ii) The elements in the principal diagonal of ( x' x ) when multiplied σ u
give the variances of the regression parameters, i.e.,
2
Σe 17.11
var( β^ 1 )=σ2u (0.0085) ¿ } var( β^ 2 )=σ2u(0.0027) ¿ }¿ ¿ σ^ 2u= i = =2.851¿
n−k 6
var( β^ 1 )=0 .0243 , SE( β^ 1 )=0 .1560
var( β^ 2 )=0 . 0077 , SE ( β^ 2 )=0 .0877
var( β^ )=0 . 0093 , SE ( β^ )=0 .0962
3 3

1
β^ ' X ' Y − ( ΣY i )2 ^
2 n β 1 Σx1 y + β^ 2 Σx2 y + β^ 3 Σx3 y
R= =
1 2 Σy 2i 575 . 98
Y ' Y − ( ΣY i ) = =0 . 97
(iii) n 594
(iv) The estimated relation may be put in the following form:
Y^ =134 .28+0 .2063 X 1 +0 .3309 X 2−0 . 5572 X 3
SE( β^ i ) (0 .1560 ) (0 . 0877 ) (0 . 0962) R 2=0. 97
t∗ (1 .3221 ) (3 .7719 ) (5 .7949 )

The variables X 1 , X 2 and X 3 explain 97 percent of total variations.

We can test the significance of individual parameters using the student’s t-test.
¿ ^
The computed value of ‘t’ is given above as t .these values indicates us only β 1
is insignificant.

Multiple Linear Regression Analysis 27

Example 2. The following matrix gives the variances and covariance of the three
variables:
y x1 x2

[ ]
y 7 .59 3. 12 26 . 99
x 1 − 29. 16 30 . 80
x2 − − 133 . 00

The first raw and the first column of the above matrix shows ∑ y 2 and the first
raw and the second column shows ∑ y x 1i and so on.
Consider the following model
β β vi
Y 1 =AY 2 1 Y 2 2 e

Where; Y1 is food consumption per capita

Y2 is food price
Y3 is disposable income per capita

And Y =ln Y 1 , X 1 =ln Y 2 and X 2 =ln Y 3

y=Y −Ȳ , x1 =X − X̄ , and x 2 =X − X̄

Using the values in above matrix answer the following questions.

a. Estimate β 1 and β 2
^ ^
b. Compute variance of β 1 and β 2
c. Compute coefficient of determination
d. Report the regression result.
Solution: It is difficult to estimate the above model as it is, to estimate the above
model easily let’s take the natural log of the above model;
ln Y 1 =ln A+β 1 lnY 2 +β 2 ln Y 3 +V i

And let: β 0 =ln A , Y =ln Y 1 , X 1 =lnY 2 and X 2 =lnY 3 the above model

becomes: Y = β0 + βX 1 + βX 2 +V i
The above matrix is based on the transformed model. Using values in the matrix
we can now estimate the parameters of the original model.

Multiple Linear Regression Analysis 28

^ −1
We know that β=( x' x) x ' y
In the present question:

^β=¿ [ β^ 1 ¿ ] ¿ ¿ ¿
¿
∴ x ' x=
[ Σx 12
Σx 1 x 2
Σx 1 x 2
Σx 22 ] and x ' y=
[ ]
Σx 1 y
Σx 2 y

Substituting the relevant quantities from the given variance-covariance matrix, we

obtain:

x ' x=
[2930 .16. 80 30 .80
133 .00 ]
and x ' y=
3. 12
26 . 99 [ ]
29.16 30.80
|x' x|=| | =2929.64
30.80 133.00

∴ ( x' x )−1=
1
=
[
133 . 00 −30 . 80
2929 . 64 −30 . 80 29 .16
≈
0 .0454 −0. 0105
−0. 0105 0.0099 ] [ ]
x)−1 x ' y=¿ [ β 1 ¿ ] ¿ ¿¿
^ ^
β=(x'
(a) ¿

[ 0. 0454 −0 .0105 3 .12 −0. 1421

−0 .0105 0 .0099 26 . 99
≈
0 . 2358][ ] [ ]
−1 2
(b). The element in the principal diagonal of ( x' x ) when multiplied by σ u give
^
the variances of the α^ and β .

2 α^ Σx2 y + β^ Σx 3 y
R =
(c). Σy2i

−(0. 1421)(3 . 12)+( 0. 2358 )(26 .99 )

=
7 . 59
∴ R 2 =0 .78 ; Σe2i =(1−R 2 )( Σy 2i )≈1 .6680
1. 6680
∴ σ^ 2u = =0 . 0981
17

Multiple Linear Regression Analysis 29

var( α^ )=(0.0981 )(0.0454 )≈0.0045 , ∴ ( α^ )SE=0.0667
var( β^ )=(0 .0981 )(0 . 0099)≈0 .0009, ∴ ( β^ )SE=0 . 0312
(d). The results may be put in the following form:
Y^ 1 =AY (−0 . 1421) Y ( 0 . 2358)
1 3
SE ( 0 . 0667)( 0 . 0312) R2 =0 . 78
t∗ (−2 .13 ) ( 7 . 55)
The (constant) food price elasticity is negative but income elasticity is positive.
Also income elasticity if highly significant. About 78 percent of the variations in
the consumption of food are explained by its price and income of the consumer.

Example 3:

Consider the model: Y =α + β 1 X 1 i + β 2 X 2 i +U i

On the basis of the information given below answer the following question
ΣX 12=3200 ΣX 1 X 2=4300 ΣX 2 =400
ΣX 22=7300 ΣX 1 Y =8400 ΣX 2 Y =13500
ΣY =800 ΣX 1 =250 n=25
ΣY i2=28 , 000
a. Find the OLS estimate of the slope coefficient
^
b. Compute variance of β 2

c. Test the significant of β 2 slope parameter at 5% level of significant

2 2
d. Compute R and R̄ and interpret the result
e. Test the overall significance of the model
Solution:
a. Since the above model is a two explanatory variable model, we can estimate
β^ 1 and β^ 2 using the formula in equation (3.21) and (3.22) i.e.

Σx2 yΣx 22−Σx 2 yΣx1 x 2

β^ 1=
Σx21 Σx22 −( Σx 1 x 2 )2

Multiple Linear Regression Analysis 30

Σx2 yΣx 21 −Σx 1 yΣx1 x 2
β^ 2 =
Σx21 Σx22 −( Σx 1 x 2 )2

Since the x’s and y’s in the above formula are in deviation form we have to find
the corresponding deviation forms of the above given values.
We know that:
Σx1 x 2 =ΣX 1 X 2−n X̄ 1 X̄ 2

=4300−(25)(10)(16)
=300
Σx1 y=ΣX 1 Y −n X̄ 1 Ȳ
=8400−25(10 )(32)
=400
Σx2 y =ΣX 2 Y −n X̄ 2 Ȳ
=13500−25(16 )(32)
=700
Σx21 =ΣX 21 −n X̄ 21

=3200−25(10 )2
¿ 700
Σx22 =ΣX 22 −n X̄ 22

=7300−25(16 )2
¿ 900
Now we can compute the parameters.
Σx2 yΣx 22−Σx 2 yΣx1 x 2
β^ 1=
Σx21 Σx22 −( Σx 1 x 2 )2

(400 )(900 )−(700 )(300)

=
(900 )(700)−(300)2
¿0 . 278
Σx2 yΣx 21 −Σx 1 yΣx1 x 2
β^ 2 =
Σx21 Σx22 −( Σx 1 x 2 )2

Multiple Linear Regression Analysis 31

(700)(700 )−(400 )(300)
=
(900 )(700)−(300)2
¿0 . 685
The intercept parameter can be computed using the following formula.
α^ =Ȳ − β^ 1 X̄ 1− β^ 2 X̄ 2

=32−(0.278)(10)−(0.685)(16)
=18.26
σ^ 2 Σx 22
var ( β^ 1 )=
b. Σx 12 Σx 22−( Σx 1 x 2 )2

Σe 2i
2
^ =
⇒σ
n−k Where k is the number of parameter

In our case k=3

Σe 2i
^ 2=
⇒σ
n−3
Σe21 =Σy 2 − β^ 1 Σx 1 y− β^ 2 Σx2 y

=2400−0 . 278(400 )−(0 .685 )(700)

=1809 . 3

2 Σe 2i
σ^ =
n−3
1809 . 3
=
25−3
=82. 24
(82 . 24 )(900)
⇒ var( β^ 1 )= =0 . 137
540 , 000

SE( β^ )= √ var( β^ 1 )=√ 0 .137=0 .370

σ^ 2 Σx 21
var ( β^ 2 )=
Σx 21 Σx 21−( Σx 1 x 2 )2

( 82. 24 )(700)
= =0 . 1067
540 , 000

SE( β^ )= √ var( β^ 2 )=√ 0 .1067=0 . 327

Multiple Linear Regression Analysis 32

^
c. β 1 can be tested using students t-test
This is done by comparing the computed value of t and critical value of t which is
α
obtained from the table at 2 level of significance and n-k degree of freedom.
t∗¿ 0. 278
= =0 . 751 ¿
SE ( ^ ) 0. 370
β
Hence; 1

α
=0 . 05 2 =0 . 025
The critical value of t from the t-table at 2 level of significance and
22 degree of freedom is 2.074.
t c=2. 074
t∗¿ 0 . 755
⇒ t∗¿ t c

The decision rule if t∗¿ t c is to reject the alternative hypothesis that says β is

different from zero and to accept the null hypothesis that says β is equal to zero.
^
The conclusion is β 1 is statistically insignificant or the sample we use to estimate
β^ 1 is drawn from the population of Y & X in which there is no relationship
1

between Y and X1(i.e. β 1=0 ).

2
d. R can be easily using the following equation
ESS RSS
R2 = =
TSS 1- TSS
2
We know that RSS=Σei
^2 ^ ^ ^
and TSS=Σy and ESS=Σ y = β 1 Σx1 y + β 2 Σx2 y +.. . .. .+ β k Σxk y
2

For two explanatory variable model:

RSS 10809 .3
=1−
R = 1- TSS
2
2400
=0. 24

Multiple Linear Regression Analysis 33

⇒ 24% of the total variation in Y is explained by the regression line
Y^ =18 . 26+0 . 278 X 1 +0. 685 X 2 ) or by the explanatory variables (X and X ).
1 2

2 Σe2i / n−k (1−R2 )(n−1)

R =1− =1−
Adjusted Σy2 / n−1 n−k

(1−0. 24 )(24 )
=1−
22
=0. 178
e. Let’s set first the joint hypothesis as
H 0 : β 1 =β 2=0

against H 1 : at least one of the slope parameter is different from zero.

The joint test hypothesis is testing using the F-test given below.
ESS /k−1
F ∗[( k−1) ,( n−k )] ¿
RSS/n−k
R2 / k−1
=
1−R 2 / n−k

From (d) R2 =0 . 24 and k =3

F ∗( 2 ,22 ) ¿ 3 . 4736
this is the computed value of F. Let’s compare this
with the critical value F at 5% level of significance and (3,.23) numerator and
denominator respectively. F (2,22) at 5%level of significance = 3.44.

F*(2,22) = 3.47
Fc(2,22)=3.44

⇒ F*>Fc, the decision rule is to reject H 0 and accept H1. We can say that
the model is significant i.e. the dependent variable is, at least, linearly
related to one of the explanatory variables.

Multiple Linear Regression Analysis 34

Multiple Linear Regression Analysis 35