Lecture No.

Probability & Statistics

IN THE LAST LECTURE,
YOU LEARNT

 Sampling Distribution of p̂
 Sampling Distribution of
X1  X2
TOPICS FOR TODAY

Sampling Distribution of X1  X 2
(continued)
Sampling Distribution of p̂1  p̂ 2
Point Estimation
Desirable Qualities of a Good Point
Estimator
–Unbiasedness
–Consistency
EXAMPLE: Car batteries produced by
company A have a mean life of 4.3 years with
a standard deviation of 0.6 years.
A similar battery produced by company B
has a mean life of 4.0 years and a standard
deviation of 0.4 years.
What is the probability that a random
sample of 49 batteries from company A will
have a mean life of at least 0.5 years more
than the mean life of a sample of 36 batteries
from company B?
SOLUTION: We are given the following data:
Population A:
1 = 4.3 years, 1 = 0.6 years, n1 = 49
Population B:
2 = 4.0 years, 2 = 0.4 years, n2 = 36
Both sample sizes (n1 = 49, n2 = 36) are large
enough to assume that the sampling distribution
of the differences is approximately a normal
such that:
Mean:
 x1  x 2 1   2 4.3  4.0 0.3 years

and standard deviation:

2 2
1 2 0.36 0.16
 x1  x 2    
n1 n2 49 36
0.1086 years.
Thus the variable
Z
X1  X 2  1   2 
2 2
1 2

n1 n2


X1  X 2  0.3
0.1086
is approximately N (0, 1)
We are required to find the probability that the
mean life of 49 batteries produced by company
A will have a mean life of at least 0.5 years
longer than the mean life of 36 batteries
produced by company B, i.e. We are required
to find: PX1  X 2 0.5.
Transforming X1  X 2 0.5 to z-value, we find:
0.5  0.3
z 1.84
0.1086
0.3 0.5 X1  X 2

0 1.84 Z
PX1  X 2 0.5PZ 1.84
0.5  P0  Z  1.84
0.5  0.4671
0.0329
In other words, (given that the real
difference between the mean lifetimes of
batteries of company A and batteries of
company B is 4.3 - 4.0 = 0.3 years), the
probability that a sample of 49 batteries
produced by company A will have
a mean life of at least 0.5 years longer than the
mean life of a sample of 36 batteries produced
by company B, is only 3.3%.
Next, we consider the Sampling Distribution
of the Differences between Proportions.
Suppose there are two binomial
populations with proportions of successes p1
and p2 respectively.
Let independent random samples of sizes n 1
and n2 be drawn from the respective
populations, and the differences p̂1  p̂ 2
between the proportions of all possible
pairs of samples be computed.
T hen, a probability distribution of the
differences p̂ 1  p̂ 2 can be obtained.
Such a probability distribution is
called the sam pling distribution
of the differences betw een the
proportions p̂ 1  p̂ 2 .
We illustrate the sampling
distribution of p̂1  p̂ 2
with the help of the following example:
EXAMPLE: It is claimed that 30% of the
households in Community A and 20% of the
households in Community B have at least one
teenager.
A simple random sample of 100
households from each community yields the
following results:
p̂ A 0.34, p̂ B 0.13.
What is the probability of observing a
difference this large or larger if the claims
are true?
SOLUTION: We assume that if the
claims are true, the sampling distribution of
p̂ A  p̂ B is approximately normally
distributed.
Since we are reasonably confident
that our sampling distribution is
approximately normally distributed, hence
we will be finding any required probability
by computing the relevant areas under our
normal curve, and, in order to do so, we will
first need to convert our variable p̂ A  p̂ B to
Z.
In order to convert p̂ A  p̂ B to Z, we need
the values of  P̂  P̂ as well as  P̂  P̂ .
A B A B

It can be mathematically proved that:

PROPERTIES OF THE SAMPLING
DISTRIBUTION OF p̂1  p̂ 2 :
Property No. 1:
The mean of the sampling distribution
of p̂1  p̂ 2 , denoted by  P̂  P̂ , is equal to the
1 2
difference between the population proportions,
that is  p̂1  p̂2 p1  p 2 .
Property No. 2:
The standard deviation of the sampling
distribution of p̂1  p̂ 2 , (i.e. the standard error
of p̂1  p̂ 2 ) denoted by  p̂  p̂ is given by
1 2

p1q1 p 2 q 2
 p̂1  p̂ 2   ,
n1 n2
where q = 1 – p.
Hence, in this example, we have:
 p̂A  p̂ B 0.30  0.20 0.10 and
2
 p̂A  p̂B 
0.300.70 0.200.80
 0.0037
100 100
The observed difference in sample proportions is
p̂ A  p̂ B 0.34  0.13 0.21
The probability that we wish to determine is
represented by the area to the right of 0.21 in the
sampling distribution of p̂ A  p̂ B .
To find this area, we compute
0.21  0.10 0.11
z  1.83
0.0037 0.06
0.4664 0.0336
p̂ A  p̂ B
0.10 0.21
Z
0 1.83
The area between z = 0 and z = 1.83 is 0.4664.
Hence, the area to the right of z = 1.83 is
0.0336 i.e. 3.36%, the probability of observing
a difference as larger as or larger than the
actually observed.
The students are encouraged to try to
interpret this result with reference to the
situation at hand, as, in attempting to solve a
statistical problem, it is very important not just
to apply various formulae and obtain numerical
results, but to interpret the results with
reference to the problem under consideration.
Does the result indicate that at least one
of the two claims is untrue, or does it imply
something else?
1) We have discussed various sampling
distributions with reference to the simplest
technique of random sampling, i.e. simple
random sampling.
And, with reference to simple random
sampling, it should be kept in mind that this
technique of sampling is appropriate in that
situation when the population is
homogeneous.
2) Let us consider the reason why the
standard deviation of the sampling distribution
of any statistic is known as its standard error:
Consider the fact that any statistic, considered
as an estimate of the corresponding population
parameter, should be as close in magnitude to
the parameter as possible.
The difference between the value of the statistic
and the value of the parameter can be regarded
as an error and is called ‘sampling error’.
Geometrically, each one of these errors
can be represented by horizontal line
segment below the X-axis, as shown below:

x6 x5 x 4 x1 x 2 x 3
x

The above diagram clearly indicates that
there are various magnitudes of this error,
depending on how far or how close the values
of our statistic are in different samples.
The standard deviation of X gives us a
‘standard’ value of this error, and hence the
term ‘Standard Error’.
Having presented the basic ideas
regarding sampling distributions, we now
begin the discussion regarding POINT
ESTIMATION:
POINT ESTIMATION
Point estimation of a population parameter
provides an estimate a single value calculated
from the sample that is likely to be close in
magnitude to the unknown parameter.
The difference between ‘Estimate’ and
‘Estimator’:
An estimate is a numerical value of the
unknown parameter obtained by applying a rule
or a formula, called an estimator, to a sample
X1, X2, …, Xn of size n, taken from a
population.
In other words, an estimator stands for the rule
or method that is used to estimate a parameter
whereas an estimate stands for the numerical
value obtained by substituting the sample
observations in the rule or the formula.
If X1, X2, …, Xn is a random sample of
size n from a population with mean , then
1 n
X   X i is an estimator of , and x,
n i 1
the numerical value of X, is an estimate
of  (i.e. a point estimate of ).
In general, the (the Greek letter ) is
customarily used to denote an unknown
parameter that could be a mean, median,
proportion or standard deviation, while an
estimator of  is commonly denoted by
̂ , or sometimes by T.
It is important to note that an estimator is
always a statistic which is a function of the
sample observations and hence is a random
variable as the sample observations are likely
to vary from sample to sample.
In other words:
In repeated sampling, an estimator is a
random variable, and has a probability
distribution, which is known as its sampling
distribution.
Having presented the basic definition of a
point estimator, we now consider some
desirable qualities of a good point estimator:
In this regard, the point to be understood
is that a point estimator is considered a good
estimator if it satisfies various criteria.

DESIRABLE QUALITIES OF A GOOD

POINT ESTIMATOR
• unbiasedness
• consistency
• efficiency
U N B IA S E D N E S S
A n e s tim a to r is d e fin e d to b e u n b ia s e d if
th e s ta tis tic u s e d a s a n e s tim a to r h a s its
e x p e c te d v a lu e e q u a l to th e tr u e v a lu e o f th e
p o p u la tio n p a r a m e te r b e in g e s tim a te d .
I n o t h e r w o r d s , l e t ̂ b e a n e s t i m a t o r o f a
p a r a m e te r  . T h e n ̂ w i l l b e c a l l e d a n
u n b ia s e d e s tim a to r if E ˆ
 
 .
If E 
ˆ
   , th e s ta tis tic is s a id to b e a
b ia s e d e s tim a to r .
EXAMPLE
Let us consider the sample mean X
as an estimator of the population mean .
Then we have  =
ˆ 1 n
and X  Xi.

n i1
Now, we knowthat EX 
i.e. 
E ˆ
 .
Hence, X is an unbiasedestimator of .
Let us illustrate the concept of
unbiasedness by considering the example of the
annual Ministry of Transport test that was
presented in the last lecture:
EXAMPLE: Let us examine the case of an
annual Ministry of Transport test to which all
cars, irrespective of age, have to be submitted.
The test looks for faulty breaks, steering,
lights and suspension, and it is discovered after
the first year that approximately the same
number of cars have 0, 1, 2, 3, or 4 faults.
If we let X denote the number of faults
in a car, then X can take the values 0, 1, 2, 3,
and 4, and the probability of each of these X
values is 1/5.
No. of Probability
Faulty Items f(x)
(X)
0 1/5
1 1/5
2 1/5
3 1/5
4 1/5
Total 1
50
 x  x f x   2 
25
i.e. the mean of the sampling distribution ofX
is equal to the population mean.
By virtue of this property, we say that the
sample mean is an UNBIASED estimate of the
population mean.
Un
bi
asedne
ssisa p r
ope
rt
y
t
hat r equ
ir
es t hat t h
e
pro
bab
ili
tydis
tri
b u
ti
on of̂
ben e
ces
sari
lycen t
ereda t
t
h epar
am e
ter,i
rresp
ec
ti
ve
ofth
eval
u eofn
.
Visual Representation of the
Concept of Unbiasedness:

X

E X   implies that the distribution
of X is centered at .
What this means is that, although many of
the individual sample means are either under-
estimates or over-estimates of the true
population mean, in the long run, the over-
estimates balance the under-estimates so that
the mean value of the sample means comes
out to be equal to the population mean.
Let us now consider
some other estimators
which possess the
desirable property of
being unbiased:
The sample median is also
an unbiased estimator of 
when the population is
normally distributed
(i.e.
If X is normally distributed,
then

~
E X . )
Also, as far as p, the
proportion of successes
in the sample is concerned,
we have:
Considering the binomial
random variable X (which denotes
the number of successes in n
trials), we have:
 X 1
Ep̂  E   EX 
n n
np
 p
n
Hence, the sample proportion is
an unbiased estimator of the
population parameter p.
But
As far as the sample
variance S2 is concerned, it
can be mathematically proved
that
E(S )   .
2 2

Hence, the sample

variance S2 is a biased
estimator of  .
2
F o r a n y p o p u la tio n p a r a m e te r  a n d

i t s e s t i m a t o r ̂ , t h e q u a n t i t y E ̂   i s
k n o w n a s th e a m o u n t o f b ia s .

T h is q u a n tity is p o sitiv e if E ˆ
  , 
a n d is n e g a tiv e if E ˆ
 
 ,

and, hence, th e e s tim a to r is s a id to b e

p o s itiv e ly b ia s e d w hen 
E ̂   and

n e g a t i v e l y b i a s e d w h e n E ̂   .
Since unbiasedness is a
desirable quality, we would
like the sample variance to be
an unbiased estimator of  .
2

In order to achieve this

end, the formula of the sample
variance is modified as
follows:
Modified formula for the
sample variance:

2
s 
 x  x 2

n 1
Since E(s ) =  , hence s is
2 2 2

an unbiased estimator of 2.

Why is unbiasedness
consider a desirable
property of an estimator?
In order to obtain an
answer to this question,
consider the following:
With reference to the
estimation of the population
mean , we note that, in an
actual study, the probability is
very high that the mean of our
sample i.e.X will either be less
than  or more than .
Hence, in an actual study, we
can never guarantee that our X
will coincide with .
Unbiasedness implies that,
although in an actual study,
we cannot guarantee that
our sample mean will coincide
with , our estimation
procedure (i.e. formula) is
such that, in repeated
sampling, the average value of
our statistic will be equal to .
The next desirable
quality of a good point
estimator is consistency:
C O N S IS T E N C Y

A n e s tim a to r ˆ is s a id to b e
a c o n s is te n t e s tim a to r o f th e
p a r a m e te r  if, fo r a n y
a r b itr a r ily s m a ll p o s itiv e
q u a n tity e ,
Lim
n  
P  ˆ  
  e  1.
I
n otherw o
r d
s,an est
imatr̂
o
i
sc al
led aconsi
stentest
imat
or
of i
fthe pro
bab i
li
tythat̂i
s
ver
yc l
oset o ,a ppr
oach
es
uni
tyw ith an i
ncreasein t
he
s
a mplesi
ze.
It should be noted
that consistency is a
large sample property.
Another point to be
noted is that a consistent
estimator may or may not
be unbiased.
T h e sam p le m ean
1 n
X   X i , w h ich is an
n i 1
u n b iased estim ator of  , is a
con sisten t estim ator of th e
m ean  .
The sample proportion p̂
is also a consistent
estimator of the parameter
p of a population that has a
binomial distribution.
The median is not a
consistent estimator of 
when the population has
a skewed distribution.
The sample variance
1 n
S   X i  X  ,
2 2
n i1
though a biased estimator, is
a consistent estimator of the
population variance  .
2
Generally speaking, it can
be proved that a statistic
whose STANDARD ERROR
decreases with an increase in
the sample size, will be
consistent.
IN TODAY’S LECTURE,
YOU LEARNT
Sampling Distribution of X1  X 2
(continued)
Sampling Distribution of p̂1  p̂ 2
Point Estimation
Desirable Qualities of a Good Point
Estimator
–Unbiasedness
–Consistency
IN THE NEXT LECTURE,
YOU WILL LEARN

Desirable Qualities of a Good Point Estimator:

Efficiency
•Methods of Point Estimation:
The Method of Moments
The Method of Least Squares
The Method of Maximum Likelihood
•Interval Estimation:
•Confidence Interval for 