52

6. Week 6

The next result is stated without proof.

Theorem 6.1. Let X be a continuous RV with p.d.f. fX and support SX . Suppose {x ∈ R :

fX (x) > 0} = ∪ki=1 (ai , bi ) and fX is continuous on each (ai , bi ). We assume that the intervals
(ai , bi ) are pairwise disjoint.
Let h : R → R be a function such that on each (ai , bi ), h : (ai , bi ) → R is strictly monotone and
continuously differentiable with inverse function h−1
i for i = 1, . . . , k.
Then Y = h(X) is a continuous RV with support SY = ∪ki=1 [ci , di ], where ci = min{h(ai ), h(bi )}
and di = max{h(ai ), h(bi )}. The p.d.f. is given by
k d −1
fX h−1
X
fY (y) = i (y) h (y) 1(ci ,di ) (y), y ∈ R
i=1 dy i

where 1(ci ,di ) (y) = 1 if y ∈ (ci , di ) and 0 otherwise.

Note 6.2. In Theorem 6.1, the function h may be strictly monotone increasing in some (ai , bi )
and strictly monotone decreasing in other intervals. Moreover, this monotonicity may be verified
by looking at the sign of h′ . If h′ (x) > 0, ∀x ∈ (ai , bi ), then h is strictly monotone increasing on
(ai , bi ). If h′ (x) < 0, ∀x ∈ (ai , bi ), then h is strictly monotone decreasing on (ai , bi ).

Example 6.3. Let X be a continuous RV with p.d.f.


 e−x , if x > 0
fX (x) =
 0, otherwise

and consider Y = X 2 . Here, SX = [0, ∞) and the function h : R → R defined by h(x) := x2 , ∀x ∈ R

is continuous differentiable on (0, ∞). Moreover, h′ (x) = 2x > 0, ∀x ∈ (0, ∞) and hence h is strictly
√
monotone increasing on (0, ∞). The inverse function is given by h−1 (y) = y, ∀y ∈ (0, ∞).
The p.d.f. fY is given by
√

 e−√ y ,

if y > 0
2 y
fY (y) = 
 0, otherwise.
The DF FY can now be computed from the p.d.f. fY by standard techniques.
53

Example 6.4. Let X be a continuous RV with p.d.f.


|x|
 , if − 1 < x < 1
 2


fX (x) = x
 3
, if 1 ≤ x < 2



0, otherwise

and consider Y = X 2 .
Observe that {x ∈ R : fX (x) > 0} = (−1, 0) ∪ (0, 2). Now, h(x) = x2 is strictly decreasing on
√
(−1, 0) with inverse function h−1 2
1 (t) = − t; and h(x) = x is strictly increasing on (0, 2) with
√
inverse function h−1
2 (t) = t. Note that h ((−1, 0)) = (0, 1) and h ((0, 2)) = (0, 4). Then, Y = X 2
has p.d.f. given by
√ d √ √ d √
fY (y) = fX (− y) (− y) 1(0,1) (y) + fX ( y) ( y) 1(0,4) (y)
dy dy

1
 , if 0 < y < 1
 2


= 1
 6
, if 1 < y < 4



0, otherwise.

We can compute the DF of Y and verify that this matches with our earlier computation in Exam-
ple 5.19.

Let X be a discrete (or continuous) RV defined on a probability space (Ω, F, P) with DF FX ,

support SX and p.m.f. (or p.d.f.) fX .

Definition 6.5 (Expectation/Expected value/Mean of the RV X). The Expectation/Expected

value/Mean of the RV X, denoted by EX, is defined as the quantity

|x|fX (x) < ∞ for discrete X,
P P
xfX (x), if


x∈SX x∈SX

E[X] := R
∞ R∞

 −∞ xfX (x) dx, if −∞ |x|fX (x) dx < ∞ for continuous X.

Remark 6.6. If the sum or the integral above converges absolutely, we say that the expectation
EX exists or equivalently, EX is finite. Otherwise, we shall say that the expectation EX does not
exist.
54

Note 6.7. Note that it is possible to define the expectation EX through the law/distribution PX
of X. However, this is beyond the scope of this course.

Example 6.8. Fix c ∈ R. Let X be a discrete RV with p.m.f.


1,
if x = c


fX (x) = P(X = x) = 
0, otherwise.


Such RVs are called constant/degenerate RVs. Here, the support is a singleton set SX = {c} and
|x|fX (x) = |c| < ∞ and hence EX =
P P
x∈SX x∈SX xfX (x) = c.

Example 6.9. Let X be a discrete RV with p.m.f.


 1 , ∀x ∈ {1, 2, 3, 4, 5, 6}


6
fX (x) :=
0, otherwise.



Here, the support is SX = {1, 2, 3, 4, 5, 6}, a finite set with all elements positive and hence
|x|fX (x) =
P P
x∈SX x∈SX xfX (x) is finite and
X 1 7
EX = xfX (x) = (1 + 2 + 3 + 4 + 5 + 6) = .
x∈SX 6 2

Example 6.10. Let X be a discrete RV with p.m.f.


1 , ∀x ∈ {1, 2, 3, · · · }


2x
fX (x) :=
0, otherwise.



Here, the support is SX = {1, 2, 3, · · · }, the set of natural numbers. To check the existence of EX,
P∞
|x|fX (x) = x 21x . Now, the x-th term is
P
we need to check the convergence of the series x∈SX x=1
x
2x
and
x+1
2x+1 1
lim = < 1.
x→∞ x 2
2x
55

By ratio test, we have the required convergence and the existence of EX follows.
Observe that
∞ ∞ ∞ ∞
X 1 1 X 1 1 X 1 1 1X 1 1 1
EX = x x
= + x x
= + (x + 1) x+1
= + x x + = 1 + EX,
x=1 2 2 x=2 2 2 x=1 2 2 2 x=1 2 2 2
which gives EX = 2.
P∞ 1 π2
Note 6.11. It is fact that n=1 n2 = 6
.

Example 6.12. Let X be a discrete RV with p.m.f.


 3 , ∀x ∈ {±1, ±2, ±3, · · · }


π 2 x2
fX (x) :=
0, otherwise.



Here, the support is SX = {±1, ±2, ±3, · · · }. To check the existence of EX, we need to check
P∞ P∞
|x|fX (x) = 2 n π23n2 = 6 1
P
the convergence of the series x∈SX n=1 π2 n=1 n . However, this series
diverges and hence EX does not exist.

Example 6.13. Let X be a continuous RV with the p.d.f.


1, if 0 < x < 1,


fX (x) =
0, otherwise.



R∞
To check the existence of EX, we need to check the existence of −∞ |x|fX (x) dx. Now,
Z ∞ Z 1
1
|x|fX (x) dx = x dx =
−∞ 0 2
and hence EX = 12 .

Example 6.14. Let X be a continuous RV with the p.d.f.

1
fX (x) = e−|x| , ∀x ∈ R.
2
R∞
To check the existence of EX, we need to check the existence of −∞ |x|fX (x) dx. Now,
Z ∞ Z ∞ Z ∞
1 −|x|
|x|fX (x) dx = |x| e dx = xe−x dx = 1 < ∞
−∞ −∞ 2 0
56

and hence EX exists and

Z ∞ Z ∞
1
EX = xfX (x) dx = x e−|x| dx = 0.
−∞ −∞ 2

Example 6.15. Let X be a continuous RV with the p.d.f.

1 1
fX (x) = , ∀x ∈ R.
π 1 + x2
R∞
To check the existence of EX, we need to check the existence of −∞ |x|fX (x) dx. Now,
Z ∞ Z ∞
1 1 2Z∞ x
|x|fX (x) dx = |x| dx = dx = ∞
−∞ −∞ π 1 + x2 π 0 1 + x2
and hence EX does not exist.

Proposition 6.16. Let X be a discrete or continuous RV such that EX exists. Then,

Z ∞ Z 0
EX = P(X > x) dx − P(X < x) dx.
0 −∞

Proof. We prove the result when X is continuous. The case for discrete X can be proved in a
similar manner. Observe that
Z ∞ Z 0 Z ∞
EX = xfX (x) dx = xfX (x) dx + xfX (x) dx
−∞ −∞ 0
Z 0 Z 0 Z ∞ Z x
=− fX (x) dydx + fX (x) dydx
x=−∞ y=x x=0 y=0
Z 0 Z y Z ∞ Z ∞
=− fX (x) dxdy + fX (x) dxdy
y=−∞ x=−∞ y=0 x=y
Z ∞ Z 0
= P(X > y) dy − P(X < y) dy.
0 −∞

This completes the proof. □

Remark 6.17. (a) Suppose X is discrete or continuous with P(X ≥ 0) = 1. Then P(X ≤ x) =
R∞
0, ∀x < 0 and hence EX = 0 P(X > x) dx.
57

(b) Suppose that X is discrete with P(X ∈ {0, 1, 2, · · · }) = 1. Then P(X > x) = P(X ≥
n + 1), ∀x ∈ [n, n + 1), n ∈ {0, 1, 2, · · · } and hence by part (a),
Z ∞ ∞
X ∞
X
EX = P(X > x) dx = P(X ≥ n + 1) = P(X ≥ n).
0 n=0 n=1

(c) For any RV X, recall that P(X = x) ̸= 0 for at most countably many points x and
consequently, P(X < x) = P(X ≤ x) except those points x. Thus,
Z ∞ Z 0
EX = P(X > x) dx − P(X < x) dx
0 −∞
Z ∞ Z 0
= P(X > x) dx − P(X ≤ x) dx
0 −∞
Z ∞ Z 0
= (1 − FX (x)) dx − FX (x) dx.
0 −∞

Note 6.18 (Expectation of functions of RVs). Given a function h : R → R and an RV X, we have

already discussed about the distribution of Y = h(X). If the p.m.f./p.d.f. fY is known, we can
then consider the existence of EY through fY , as per Definition 6.5. However, to do this, we first
need to compute fY from X and then check the relevant existence. In what follows, we discuss
the computation of EY = Eh(X) directly from X, using the p.m.f./p.d.f. fX .

Proposition 6.19. (a) Let X be a discrete RV with p.m.f. fX and support SX and let h :
R → R be a function. Consider the discrete RV Y := h(X). Then EY exists provided
|h(x)|fX (x) < ∞ and in this case,
P
x∈SX

X
EY = Eh(X) = h(x)fX (x).
x∈SX

(b) Let X be a continuous RV with p.d.f. fX and support SX and let h : R → R be a function.
R∞
Consider the RV Y := h(X). Then EY exists provided −∞ |h(x)|fX (x) dx < ∞ and in this
case, Z ∞
EY = Eh(X) = h(x)fX (x) dx.
−∞

Proof. We consider the proof for the case when X is discrete. The other case can be proved by
similar arguments.
58

By Theorem 5.21, Y = h(X) is discrete with support SY = h(SX ). Now,

X X X X X X
|y|fY (y) = |y| fX (x) = |h(x)|fX (x) = |h(x)|fX (x).
y∈SY y∈SY {x∈SX :h(x)=y} y∈SY {x∈SX :h(x)=y} x∈SX

|h(x)|fX (x) < ∞ and in this case,

P
Therefore, EY exists provided x∈SX

X X X X
EY = yfY (y) = y fX (x) = h(x)fX (x).
y∈SY y∈SY {x∈SX :h(x)=y} x∈SX

This completes the proof. □

Note 6.20. If X is discrete with p.m.f. fX such that EX exists, then E|X| = |x|fX (x) < ∞.
P
x∈SX
R∞
Similarly, if X is continuous with p.d.f. fX such that EX exists, then E|X| = −∞ |x|fX (x) dx < ∞.
Therefore EX exists if and only if E|X| < ∞. In other words, EX is finite if and only if E|X| is
finite.

Note 6.21. Fix a, b ∈ R with a ̸= 0. Let X be a discrete/continuous RV with p.m.f./p.d.f. fX

such that EX exists. Then Y = aX + b is also a discrete/continuous RV. If X is discrete, then
X X X
|ax + b|fX (x) ≤ |a| |x|fX (x) + |b| fX (x) = |a|E|X| + |b| < ∞
x∈SX x∈SX x∈SX

and hence E(aX + b) exists and equals

X X X
E(aX + b) = (ax + b)fX (x) = a xfX (x) + b fX (x) = a EX + b.
x∈SX x∈SX x∈SX

If X is continuous, a similar argument shows E(aX + b) = a EX + b.

Using arguments similar to the above observations, we obtain the next result. We skip the
details for brevity.

Proposition 6.22. Let X be a discrete/continuous RV with p.m.f./p.d.f. fX .

(a) Let hi : R → R be functions and let ai ∈ R for i = 1, 2, · · · , n. Then
n n
!
X X
E ai hi (X) = ai Ehi (X),
i=1 i=1

provided all the expectations above exist.

59

(b) Let h1 , h2 : R → R be functions such that h1 (x) ≤ h2 (x), ∀x ∈ SX , where SX denotes the
support of X. Then,
Eh1 (X) ≤ Eh2 (X),

provided all the expectations above exist.

(c) Take h1 (x) := −|x|, h2 (x) := x, h3 (x) := |x|, ∀x ∈ R. If EX exists, then

−E|X| ≤ EX ≤ E|X|,

i.e. |EX| ≤ E|X|.

(d) If P(a ≤ X ≤ b) = 1 for some a, b ∈ R, then EX exists and a ≤ EX ≤ b.

Note 6.23. Given an RV X, by choosing different functions h : R → R, we obtain several

quantities of interest of the form Eh(X).

Definition 6.24 (Moments). The quantity µ′r := E[X r ], if it exists, is called the r-th moment of
RV X for r > 0.

Definition 6.25 (Absolute Moments). The quantity E[|X|r ], if it exists, is called the r-th absolute
moment of RV X for r > 0.

Definition 6.26 (Moments about a point). Let c ∈ R. The quantity E[(X − c)r ], if it exists, is
called the r-th moment of RV X about c for r > 0.

Definition 6.27 (Absolute Moments about a point). Let c ∈ R. The quantity E[|X − c|r ], if it
exists, is called the r-th absolute moment of RV X about c for r > 0.

Note 6.28. It is clear from the definitions above that the usual moments and absolute moments
are moments and absolute moments about origin, respectively.

Proposition 6.29. Let X be a discrete/continuous RV such that E|X|r < ∞ for some r > 0.
Then E|X|s < ∞ for all 0 < s < r.

Proof. Observe that for all x ∈ R, we have |x|s ≤ max{|x|r , 1} ≤ |x|r + 1 and hence

E|X|s ≤ E|X|r + 1 < ∞.

60

Remark 6.30. Suppose that the m-th moment EX m of X exists for some positive integer m. Then
we have E|X|m < ∞ (see Note 6.20). By Proposition 6.29, we have E|X|n < ∞ for all positive
integers n ≤ m and hence the n-th moment EX n exists for X. In particular, the existence of the
second moment EX 2 implies the existence of the first moment EX, which is the expectation of X.

Definition 6.31 (Central Moments). Let X be an RV such that µ′1 = EX exists. The quantity
µr := E[(X − µ′1 )r ], if it exists, is called the r-th moment of RV X about the mean or r-th central
moment of X for r > 0.

Remark 6.32. (a) If EX exists, then µ1 := E[X − µ′1 ] = 0.

(b) If EX 2 exists, then so does EX and hence for any c ∈ R,

E(X − c)2 = EX 2 − 2cEX + c2

also exists. A similar argument shows that E(X − c)2 exists if and only if E(X − d)2 exists,
for any c, d ∈ R.
(c) If EX 2 exists, then for any c ∈ R,

E(X −c)2 = E(X −µ′1 +µ′1 −c)2 = E(X −µ′1 )2 +(µ′1 −c)2 −2(µ′1 −c)E(X −µ′1 ) = E(X −µ′1 )2 +(µ′1 −c)2

and hence,
E(X − µ′1 )2 = inf E(X − c)2
c∈R

Definition 6.33 (Variance). The second central moment µ2 of an RV X, if it exists, is called the
variance of X and denoted by V ar(X). Note that V ar(X) = µ2 = E [(X − µ′1 )2 ].

Remark 6.34. The following are some simple observations about the variance of an RV X.
(a) We have
h i
V ar(X) = E (X − µ′1 )2 = E[X 2 +(µ′1 )2 −2µ′1 X] = µ′2 −2(µ′1 )2 +(µ′1 )2 = µ′2 −(µ′1 )2 = EX 2 −(EX)2 .

(b) Since the RV (X − µ′1 )2 takes non-negative values, we have V ar(X) = E(X − µ′1 )2 ≥ 0.
61

(c) We have (EX)2 ≤ EX 2 .

(d) V ar(X) = 0 if and only if P(X = µ′1 ) = 1. (see problem set 6).
(e) For any a, b ∈ R, we have V ar(aX + b) = a2 V ar(X).
(f) Let V ar(X) > 0. Then Y := √X−EX has the property that EY = 0 and V ar(Y ) = 1.
V ar(X)

q
Definition 6.35 (Standard Deviation). The quantity σ(X) = V ar(X) is defined to be the
standard deviation of X.

Example 6.36. Let X be a discrete RV with p.m.f.


 1 , ∀x ∈ {1, 2, 3, 4, 5, 6}


6
fX (x) :=
0, otherwise.



Here, existence of µ′1 = EX and µ′2 = EX 2 can be established by standard calculations. Moreover,
X 1 7
EX = xfX (x) = (1 + 2 + 3 + 4 + 5 + 6) =
x∈SX 6 2

and
1 91
EX 2 = x2 fX (x) = (12 + 22 + 32 + 42 + 52 + 62 ) = .
X

x∈SX 6 6
Variance can now be computed using the relation V ar(X) = EX 2 − (EX)2 .

Example 6.37. In Example 6.13, we had shown EX = 21 , where X is a continuous RV with the
p.d.f. 
1, if 0 < x < 1,


fX (x) =
0, otherwise.


R∞ R1 2
Now, EX 2 = −∞ x2 fX (x) dx = 0 x dx = 13 . Then V ar(X) = EX 2 − (EX)2 = 1
3
− 1
4
= 1
12
.

Note 6.38. We are familiar with the Laplace transform of a given real-valued function defined on
R. We also know that under certain conditions, the Laplace transform of a function determines
the function almost uniquely. In probability theory, the Laplace transform of a p.m.f./p.d.f. of a
random variable X plays an important role.
62

Let X be a discrete/continuous RV defined on a probability space (Ω, F, P) with DF FX ,

p.m.f./p.d.f. fX and support SX .

Definition 6.39 (Moment Generating Function (MGF)). We say that the moment generating
function (MGF) of X exists, denoted by MX and equals MX (t) := EetX , provided EetX exists for
all t ∈ (−h, h), for some h > 0.

Note 6.40. Observe that ex > 0, ∀x ∈ R.

Note 6.41. If X is discrete/continuous with p.m.f./p.d.f. fX , then following the definition of an

expectation of an RV, we write

etx fX (x), etx fX (x) < ∞ for discrete X, ∀t ∈ (−h, h) for some h > 0
X X
if




tX
MX (t) = Ee = x∈SX x∈SX

 ∞ tx
R∞
etx fX (x) dx < ∞ for continuous X, t ∈ (−h, h) for some h > 0.
 R

−∞ e fX (x) dx, if −∞

In this case, we shall say that the MGF MX exists on (−h, h).
n
Remark 6.42. (a) MX (0) = 1 and hence A := t ∈ R : E[etX ] is finite} =
̸ ∅.
(b) MX (t) > 0 ∀t ∈ A, with A as above.
(c) For c ∈ R, consider the constant/degenerate RV X given by the p.m.f. (see Example 6.8)

1, if x = c


fX (x) =
0, otherwise.



Here, the support is SX = {c} and MX (t) = EetX = etx fX (x) = etc exists for all
P
x∈SX

t ∈ R.
(d) Suppose the MGF MX exists on (−h, h). Take constants c, d ∈ R with c ̸= 0. Then,
the RV Y = cX + d is discrete/continuous, according to X being discrete/continuous and
moreover,
MY (t) = Eet(cX+d) = etd MX (ct)
h h
exists for all t ∈ (− |c| , |c| ).
63

Note 6.43. The MGF can be used to compute the moments of an RV and this is the motivation
behind the term ‘Moment Generating Function’. This result is stated below. We skip the proof
for brevity.

Theorem 6.44. Let X be an RV with MGF MX which exists on (−h, h) for some h > 0. Then,
we have the following results.
(a) µ′r = E[X r ] is finite for each r ∈ {1, 2, . . .}.
dr
" #
′ r (r) (r)
(b) µr = E[X ] = MX (0), where MX (0) = MX (t) is the r-th derivative of MX (t) at
dtr t=0
the point 0 for each r ∈ {1, 2, . . .}.
(c) MX has the following Maclaurin’s series expansion around t = 0 of the following form
∞ r
X t ′
MX (t) = µr with t ∈ (−h, h).
r=0 r!

Proposition 6.45. Continue with the notations and assumptions of Theorem 6.44 and define
ψX : (−h, h) → R by ψX (t) := ln MX (t), t ∈ (−h, h). Then
(1) (2)
µ′1 = E[X] = ψX (0) and µ2 = V ar(X) = ψX (0),
(r)
where ψX denotes the r-th (r ∈ {1, 2}) derivative of ψX .

Proof. We have, for t ∈ (−h, h)

2
(1) (2) (1)
(1) M (t) (2)
MX (t)MX (t) − MX (t)
ψX (t) = X and ψX (t) = .
MX (t) (MX (t))2
Evaluating the above equalities at t = 0 give the required results. □

Example 6.46. Let X be a discrete RV with p.m.f.

 −λ x
 e λ , if x ∈ {0, 1, 2, . . .}
x!
fX (x) = 
0, otherwise,

where λ > 0. We have

∞ −λ x ∞ x
tx e λ (λet )
= e−λ eλe = eλ(e −1) ∀ t ∈ R
t
h i t
tX −λ
X X
MX (t) = E e = e =e
x=0 x! x=0 x!
64
n o
since A = t ∈ R : E etX < ∞ = R. Now,

MX (t) = λet eλ(e −1) MX (t) = λet eλ(e −1) 1 + λet

t t

(1) (2)
and ∀ t ∈ R.

Then,
(1) (2)
µ′1 = E(X) = MX (0) = λ, µ′2 = E(X 2 ) = MX (0) = λ(1 + λ), V ar(X) = µ2 = µ′2 − (µ′1 )2 = λ.
(1) (2)
Again, for t ∈ R, ψX (t) = ln (MX (t)) = λ (et − 1), which yields ψX (t) = ψX (t) = λet , ∀t ∈ R.
Then, µ′1 = E(X) = λ, µ2 = V ar(X) = λ. Higher order moments can be calculated by looking at
higher order derivatives of MX .

Example 6.47. Let X be a continuous RV with p.d.f.


 e−x , if x > 0
fX (x) =
 0, otherwise.

We have
Z ∞ Z ∞
tx −x
MX (t) = E e tX
= e e dx = e−(1−t)x dx = (1 − t)−1 < ∞, if t < 1.
0 0
n o
In particular, MX exists on (−1, 1) and A = t ∈ R : E etX < ∞ = (−∞, 1) ⊃ (−1, 1). Now,
(1) (2)
MX (t) = (1 − t)−2 and MX (t) = 2(1 − t)−3 , t < 1.

Then,
(1) (2)
µ′1 = E(X) = MX (0) = 1, µ′2 = E(X 2 ) = MX (0) = 2, V ar(X) = µ2 = µ′2 − (µ′1 )2 = 1.
(1) 1 (2) 1
Again, for t < 1, ψX (t) = ln (MX (t)) = − ln(1 − t), which yields ψX (t) = 1−t
, ψX (t) = (1−t)2
, ∀t <
1. Then, µ′1 = E(X) = 1, µ2 = V ar(X) = 1.
Now, consider the Maclaurin’s series expansion for MX around t = 0. We have
∞
MX (t) = (1 − t)−1 = tr , ∀t ∈ (−1, 1)
X

r=0

tr
and hence µ′r = r!, which is the coefficient of r!
in the above power series.
65

Example 6.48. Let X be a continuous RV with p.d.f.

1 1
fX (x) = · , ∀x ∈ R.
π 1 + x2
As observed earlier in Example 6.15, EX does not exist. Since the existence of moments is a
necessary condition for the existence of MGF, we conclude that the MGF does not exist for this
RV X.