Introduction to probability theory

Lecturer: Dr. Sunanda Saha

February, 2019

1 Introduction
Sample Space and Events
There are lots of phenomena in nature, like tossing a coin or tossing a die, whose outcomes cannot
be predicted with certainty in advance, but the set of all the possible outcomes is known.These
are what we call random phenomena or random experiments. Probability theory is concerned with
such random phenomena or random experiments.
Consider a random experiment. The set of all the possible outcomes is called the sample space of
the experiment and is usually denoted by S. Any subset E of the sample space S is called an event.
Here are some examples.

EXAMPLE: 1.1. Tossing a coin. The sample space is S = {H, T }E = {H} is an event.

EXAMPLE: 1.2. Tossing a coin twice. The sample space is S = {HH, HT, T H, T T }.
E = {HH, HT } is an event, which can be described in words as the ”first toss results in a
Heads”.

EXAMPLE: 1.3. Tossing a coin twice. The sample space isS={HH, HT, T H, T
T}.E={HH, HT} is an event, which can be described in words as ”the first toss results in a
Heads

Now we recall some properties of set theoretic operations:-

Axioms of Probability
Consider an experiment with sample spaceS. A real-valued function P on the space of all events of
the experiment is called a probability measure if
(i) for all events E, 0 ≤ P (E) ≤ 1
(ii)P (S) = 1;
(iii) For any sequence of mutually exclusive events E1 , E2 , ...
∞
∪∞
X
P i=1 Ei = P (Ei )
i=1

1
EXAMPLE: 1.4. Tossing a fair die. In this case, the probability measure is given by
P (1) = P (2) = · · · = P (6) = 1/6. If the die is not fair, the probability measure will be
different.

Exercise

1. An urn contains 2 white and 2 black balls and a second urn contains 2 white and 4
black balls. If one ball is drawn at random from each urn what is the probability that
they are of the same color?

2. A bag contains 4 white and 2 black balls. Another bag contains 3 white and 5 black
balls. If a ball is drawn from each, find the probability that (i) both are white (ii) both
are black (iii) one is black and another is white.

3. The probability that a person A solves the problem is 1/3, that of B is 1/2 and that of C
is 3/5. If the problem is simultaneously assigned to all of them what is the probability
that the problem is solved?

4. From 6 positive and 8 negative numbers, 4 numbers are chosen at random and multi-
plied. What is the probability that the product is a positive number?

5. A bag contains 10 white and 3 red balls while another bag contains 3 white and 5 red
balls. 2 balls are drawn at random from the first bag and put in the second bag. Then
a ball is drawn at random from the second bag. What is the probability that it is a
white ball?

6. Two dice are thrown. Find the probability of (a) getting an odd number on the one
and a multiple of 3 on the other (b) one of the dice showed 3 and the sum on the two
dice is 9 (c) sum on the two dice is 9 (d) sum on the two dice is 13.

7. A shooter can hit a target in 3 out of 5 shots and another shooter can hit the target
in 2 out of 4 shots. Find the probability that the target is being hit (a) when both of
them try (b) by only one shooter.

8. Three students A, B, C write an entrance examination. Their chances of passing are

1/2, 1/3, and 1/4 respectively. Find the probability that (a) atleast one of them passes
(b) all of them passes (c) atleast two of them passes.

9. There are 10 students of which three are graduates. If a committee of five is to be

formed, what is the probability that there are (i) only 2 graduates (ii) atleast 2 grad-
uates.

2
Self-test problems

1. Two dice are thrown. Let E be the event that the sum of the dice is odd. Let F be the
event that at least one of the dice lands on 1, and let G be the event that the sum is
5. Find the probability of the events EF, E ∪ F, F G, EF c , and EFG.

2. A system is composed of 5 components, each of which is either working or failed.

Consider an experiment that consists of observing the status of each component, and
let the outcome of the experiment be given by the vector (x1 , x2 , x3 , x4 , x5 ) where xi
is equal to 1 if component i is working and is equal to 0 if component i is failed.
a) How many outcomes are in the sample space of this experiment?
b) Suppose that the system will work if components 1 and 2 are both working, or if
components 3 and 4 are both working, or if components 1, 3 and 5 are all working. Let
W be the event that the system will work. Specify all the outcomes.
c) Let A be the event that components 4 and 5 are both failed. How many outcomes
are contained in the event A?
d) Write all the outcomes in the event AW.

3. Suppose that A and B are mutually exclusive events for which P (A) = .3 and P (B) =
.5. What is the probability that
a) either A or B occurs?
b) A occur but B does not?
c) both A and B occur?

4. A retail establishment accepts either the American Express or the VISA credit card. A
total of 24 percent of its customers carry an American Express card, 61 percent carry
a VISA card, and 11 percent carry both cards. What percentage of its customers carry
a credit card that the establishment will accept?

5. Sixty percent of the students at a certain school wear neither a ring nor a necklace.
Twenty percent wear a ring and 30 percent wear a necklace. If one of the students is
chosen randomly, what is the probability that this student is wearing a) a ring or a
necklace? b) a ring and a necklace?

6. A total of 28 percent of American males smoke cigarettes, 7 percent smoke cigars, .05
smoke cigars and cigarettes. How many smoke cigars but not cigarettes?

7. A pair of fair dice is rolled. What is the probability that the second die lands on a
higher value than does the first die?

8. Seven balls are randomly withdrawn from an urn that contains 12 red, 16 blue, and 18
green balls. Find the probability that (a) 3 red, 2 blue, and 2 green balls are withdrawn.
(b) At least 2 red balls are withdrawn. (c) All withdrawn balls are the same colour.
(d) Either exactly 3 red balls or exactly 3 blue balls are withdrawn.

9. An instructor gives her class a set of 1010 problems with the information that the final
exam will consist of a random selection of 55 of them. If a student has figured out how
to do 77 of the problems, what is the probability that he or she will answer correctly
(a)All 55 problems? (b) At least 44 of the problems?

3
2 Conditional probability

Given two events A and B from the sigma-field of a probability space with P (B) > 0, the condi-
tional probability of A given B is defined as the quotient of the probability of the joint of events A
and B, and the probability of B:

P (A ∩ B)
P (A|B) =
P (B)

As an axiom of probability(Multiplication Rule)

P (A ∩ B) = P (A|B)P (B)

EXAMPLE: 2.1. The probability that it is Friday and that a student is absent is 0.03.
Since there are 5 school days in a week, the probability that it is Friday is 0.2. What is the
probability that a student is absent given that today is Friday?

P (F riday and Absent) 0.03

SOLUTION: 2.5. P(Absent/Friday)= = = 0.15
P (F riday) 0.2
EXAMPLE: 2.2. A jar contains black and white marbles. Two marbles are chosen without
replacement. The probability of selecting a black marble and then a white marble is 0.34, and
the probability of selecting a black marble on the first draw is 0.47. What is the probability
of selecting a white marble on the second draw, given that the first marble drawn was black?
P (white and black) 0.34
SOLUTION: 2.6. P(white/black)= = = 0.72
P (black) 0.47
EXAMPLE: 2.3. A math teacher gave her class two tests. 25% of the class passed both
tests and 42% of the class passed the first test. What percent of those who passed the first
test also passed the second test?
P (F irst and second) 0.25
SOLUTION: 2.7. P(Secon/First)= = = 0.60
F irst 0.42

Exercise

1. Joe is 80% certain that his missing key is in one of the two pockets of his hanging
jacket, being 40% certain it is in the left-hand pocket and 40% certain it is in the right
- hand pocket . If a search of the left hand pocket does not find the key . What is the
conditional probability that it is in the other pocket? Ans:- 2/3

4
2. Celine is undecided as to whether to take a French course or a chemistry course. She
estimates that her probability of receiving an A grade would be 1/2 in a French course
and 2/3 in a Chemistry course. If Celine decides to base her decision on the flip of a
fair coin, what is the probability that she gets an A in chemistry?

3. A coin is flipped twice . Assuming that all four points in the sample space are equally
likely , What is the conditional probability that both flips land on head , given that
(a) the first flip lands on heads? (b) at least one flip lands on heads?

4. Suppose that an urn contain 8 red balls and 4 white balls . We drawn 2 balls from
the urns without replacement (a) If we assume that each draw , each ball in the urn
is equally likely to be chosen , what is the probability that both balls drawn are red
? (b) Now suppose that the balls have different weights, with red ball having weight r
and each white ball having weight w . suppose that the probability that a given ball
in the urn is the next one selected is its weights divided by the sum of the weights of
all balls currently in the urn . Now what is the probability that both balls are red?
8r 7r
Ans- (a)14/33 (b)
8r + 4w 7r + 4w
5. At Kennedy Middle School, the probability that a student takes Technology and Span-
ish is 0.087. The probability that a student takes Technology is 0.68. What is the
probability that a student takes Spanish given that the student is taking Technology?
Ans:- P(Spanish/Technology) = 0.13

6. The probability of raining on Sunday is 0.07. If today is Sunday than find the proba-
bility of rain today.
Ans:- 0.49

2.1 Bayes’s Formula

In probability theory and statistics, Bayes’s theorem (alternatively Bayes’s law or Bayes’s
rule) describes the probability of an event, based on prior knowledge of conditions that might
be related to the event. For example, if cancer is related to age, then, using Bayes’s theorem,
a person’s age can be used to more accurately assess the probability that they have cancer,
compared to the assessment of the probability of cancer made without knowledge of the
person’s age.

EXAMPLE: 2.4. A bin contains 3 different types of disposable flashlights. The probability
that a type 1 flashlight will give over 100 hours of use is 0.7, with the corresponding proba-
bilities for type 2 and type 3 flashlights being .4 and .3, respectively. Suppose that 20 percent
of the flashlights in the bin are type 1, 30 percent are type 2 and 50 percent are type 3.
i) What is the probability that a randomly chosen flashlight will give more that 100 hours of
use?
ii) Given that a flashlight lasted over 100 hours, what is the conditional probability that it
was a type j flashlight, j=1,2,3?

5
SOLUTION: 2.8. P(A) = (.7)(.2) + (.4)(.3) + (.3)(.5) = 0.41
P(F1 /A) = (.7)(.2)/.41 = 14/41
P(F2 /A) = (.4)(.3)/.41 = 12/41
P(F3 /A) = (.3)(.5)/.41 = 15/41

0.7 F 0.7 × 0.2

F1 =

F10 =
0.3
2
0.
= -1
p pe
Ty
0.4 F 0.4 × 0.3
F2 =
p = 0.3
A
Type-2
F20 =
0.6
p
=
Ty

0.
5
p e-

F 0.3 × 0.5
3

0.3
F3 =

F30 =
0.7

Exercise

1. Three major parties A, B, C are contending for power in the elections of a state and
the chance of their winning the election is in the ratio 1: 3: 5. The parties A, B, C
respectively have probabilities of banning the online lottery 2/3, 1/3, 3/5. What is the
probability that there will be a ban on the online lottery in the state? What is the
probability that the ban is from the party C?

2. A bag contains three coins, one of which is two headed and the other two are normal
and fair. A coin is chosen at random from the bag and tossed four times in succession.
i) Find the probability that head turns up each time. ii) Given that head turns up
each time in succession, what is the probability that it was the two headed coin?

3. In a bolt factory there are four machines A, B, C, D manufacturing respectively 20%,

15%, 25%, 40% of the total production. Out of these 5%, 4%, 3%, 2% are defective.
If a bolt drawn at random was found defective what is the probability that it was
manufactured by A or D?

6
4. A company is manufacturing ball pens in two writing colors blue and red make packets
of 10 pens with 5 pens of each color. In a particular shop it was found that after sales,
packet one contained 3 blue and 2 red pens, packet two contained 3 blue and 5 red
pens. On the demand of a customer for a pen, the packet was drawn at random and
a pen was taken out. It was found blue. Find the probability that packet one was
selected.

5. The chance that a doctor will diagnose a disease correctly is 60%. The chance that
a patient will die after correct diagnose is 40% and the chance of death by wrong
diagnosis is 70%. If a patient dies, what is the chance that his disease was correctly
diagnosed?

6. A laboratory blood test is 95% effective in detecting a certain disease when it is, in
fact, present . however,the test also yields a ”false positive” result for 1% of the healthy
person tested.If 0.5% of the population actually has the disease, what is the probability
that a person has the disease given that the test result is positive?

7. In answering a question on a multiple-choice test, a student either knows the answer

or guesses.Let p be the probability that the student knows the answer and 1-p be
the probability that the student guesses . Assume that a stdent who guesses at the
answer will be correct with probability 1/m, where m is the number of multiple-choice
alternatives. What is the conditional probability that a student knew the answer to a
question given that he or she answered its correctly?

8. A insurance company belives that people can be divided into two classes:those who
are accident prone and those who are not.The company’s statistics show that an
accident -prone person will have an accident at some time within a fixed 1-year period
with probability 0.4,whereas this probability decreases to 0.2 for a person who is
not accident prone.If we assume that 30 percent of the population is accident prone,
what is the probability that a new policyholder will have an accident within a year of
purchasing a policy?

Suppose that the new policy holder has an accident within a year of purchasing a
policy. What is the probability that he or she is accident prone.

9. Suppose a drug test is 99% sensitive and 99% specific. That is, the test will produce
99% true positive results for drug users and 99% true negative results for non-drug
users. Suppose that 0.5% of people are users of the drug. If a randomly selected
individual tests positive, what is the probability that he is a user? Ans:-0.332

10. In a class 70% are boys and 30% are girls. 5% of boys, 3% of the girls are irregular to
the classes. What is the probability of a student selected at random is irregular to the
classes and what is the probability that the irregular student is a girl?

11. In a certain day care 30% of the children have grey eyes , 50% have blue and 20% eyes
are in other colours. One day they play a game together.In first run, 65% of the grey
eye ones, 82% of the blue eyes and 50% of the children with other eye were selected.
Now if a child is selected randomly from the class and we know that he /she was not
in the first game, what is the probability that the child has blue eyes?
7
Self-test problems

1. A total of 46 percent of the voters in a certain city classify themselves as independents,

whereas 30 percent classify themselves as liberals and 24 percent say that they are
conservatives. In a recent local election, 35 percent of the independents, 62 percent of
the liberals and 85 percent of the conservatives voted. A voter is chosen at random.
Given that this person voted in the local election, what is the probability that he or
she is a) an independent? b) A liberal? c) A conservative? d) What fraction of votes
participated in the local election?

2. A family has j children with probability pj , where p1 = 0.1, p2 = 0.25, p3 = 0.35, p4

= 0.3. A child from this family is randomly chosen. Given that this child is the eldest
child in the family, find the conditional probability that the family has (a) Only 1 child;
(b) 4 children. Redo (a) and (b) when the randomly selected child is the youngest child
of the family.

3. On rainy days, Joe is late to work with probability 0.3; on nonrainy days, he is late with
probability 0.1. With probability 0.7, it will rain tomorrow. (a) Find the probability
that Joe is early tomorrow. (b) Given that Joe was early, what is the conditional
probability that it rained?

4. Stores A, B, C have 50, 75, and 100 employees, and, respectively, 50, 60, and 70 percent
of these are women. Resignations are equally likely among all employees, regardless
of sex. One employee resigns and this is a woman. What is the probability that she
works in store C?

5. Urn A has 5 white and 7 black balls. Urn B has 3 white and 12 black balls. We flip a
fair coin. If the outcome is heads, then a ball from urn A is chosen. If the outcomes
are tails, then a ball from urn B is chosen. Suppose a white ball is selected, what is
the probability that the coin landed tails?

6. A worker has asked her supervisor for a letter of recommendation for a new job. She
estimates that there is an 80 percent chance that she will get the job if she receives
a strong recommendation, a 40 percent chance if she receives a moderately good rec-
ommendation, and a 10 percent chance if she receives a weak recommendation. She
further estimates that the probabilities that the recommendation will be strong, mod-
erate, and weak are .7, .2, and .1, respectively.(a)How certain is she that she will receive
the new job offer?(b)Given that she does receive the offer, how likely should she feel
that she received a strong recommendation? a moderate recommendation? a weak
recommendation?(c) Given that she does not receive the job offer, how likely should
she feel that she received strong recommendation? A moderate recommendation? A
weak recommendation?

8
3 Random Variables
Frequently, when an experiment is performed, we are interested mainly in some function of the
outcome as opposed to the actual outcome itself. For instance, in tossing dice, we are often
interested in the sum of the two dice and are not really concerned about the separate values of
each die. These quantities of interest, or, more formally, these real-valued functions defined on the
sample space, are known as random variables.

EXAMPLE: 3.1. Three balls are to be randomly selected without replacement from an urn
containing 20 balls numbered 1 through 20. If we bet that at least one of the balls that are
drawn has a number as large as or larger than 17, what is the probability that we win the
bet?

SOLUTION: 3.1. Let X denotes the largest number among the three balls that are selected. Then
X takes the values from 3, 4, ..., 20. Total possible outcomes of selecting 3 balls out of 20 is 20C 3 .
The number of selection that result in the event {X = i} is equal to the number of selections that
result in the ball numbered i and two of the balls numbered 1 through i − 1 being chosen. Hence the
number of selections is 1C 1 × i−1C 2 . Therefore, the required probability is
i−1
C2
P {X = i} = 20
C3
Probability of our winning the bet is

P {X ≥ 17} = P {X = 17} + P {X = 18} + P {X = 19} + P {X = 20} = 0.508

For a random variable X, the function F defined by

F (x) = P {X ≤ x} −∞<x<∞

is called the cumulative distribution function, or, more simply, the distribution function, of X. It
is to be noted that F (x) is a non decreasing function of x.

3.1 Discrete Random Variables

A random variable that can take on at most a countable number of possible values is said to be
discrete. For a discrete random variable X, we define the probability mass function p(a) of X by

p(a) = P {X = a}

Remark 1. If X must assume one of the values x1 , x2 , ..., then

1. p(xi ) ≥ 0 for i = 0, 1, 2, ...

2. p(x) = 0 for all other values of x

∞
X
3. p(xi ) = 1
i=0

9
Definition 1. If X is a discrete random variable having a probability mass function p(x),
then the expectation, or the expected value, of X, denoted by E[X], is defined by
∞
X
E[X] = xi p(xi )
i=0

QUESTION: 3.1. Find E[X], where X is the outcome when we roll a fair die.

Definition 2. If X is a random variable with mean µ, then the variance of X, denoted by

V ar(X), is defined by
2
V ar(X) = E[(X − µ)2 ] = E[X 2 ] − E[X]

QUESTION: 3.2. Calculate V ar(X) if X represents the outcome when a fair die is rolled.

Definition 3. The square root of the V ar(X) is called the standard deviation of X, and we
denote it by SD(X). That is, q
SD(X) = V ar(X)

Exercise

1. i) Find k ii) evaluate P {X < 6}, P {X ≥ 6} and P {3 <

X ≤ 6} iii) Mean and variance, iv) cumulative distribution
xi 0 1 2 3 4 5 6 7
p(xi ) 0 k 2k 2k 3k k 2 2k 2 7k 2 + k

2. A random experiment of tossing a ’die’ twice is performed. Let X denotes the random
variable of sum of two numbers turning up on the toss. Compute E[X] and standard
deviation.

3. A school class of 120 students is driven in 3 buses to a symphonic performance. There

are 36 students in one of the buses, 40 in another, and 44 in the third bus. When the
buses arrive, one of the 120 students is randomly chosen. Let X denote the number of
students on the bus of that randomly chosen student, and find E[X]. Ans: 40.2667

4. If random variable X take the value 1,2,3,4 such that

2P {X = 1} = 3P {X = 2} = P {X = 3} = 5P {X = 4}, find the probability distribu-
tion function of X.

10
5. Find the value of k such that the following distribution represents a finite probability
distribution. Hence find its mean and standard deviation. Also find P {X ≤ 1},
P {X > 1} and P {−1 < X ≤ 2}
xi -3 -2 -1 0 1 2 3
p(xi ) k 2k 3k 4k 3k 2k k

6. A random variable X take the values -3, -2, -1, 0,1, 2, 3 such that P {X = 0} = P {X <
0} and P {X = −3} = P {X = −2} = P {X = −1} = P {X = 1} = P {X = 2} =
P {X = 3}. Find the probability distribution.

7. From a sealed box containing a dozen apples it was found that 3 apples are perished.
Obtain the probability distribution of the number of perished apples when 2 apples
are drawn at random. Also find the mean and variance of this distribution. (Ans:
mean= 1/2 and variance = 15/44)

3.2 Continuous Distribution

In previous section, we considered discrete random variables—that is, random variables whose set
of possible values is either finite or countably infinite. However, there also exist random variables
whose set of possible values is uncountable. Two examples are the time that a train arrives at
a specified stop and the lifetime of a transistor. Let X be such a random variable. We say that
X is a continuous random variable if there exists a non negative function f , defined for all real
x ∈ (−∞, ∞), having the property that, for any set B of real numbers,
Z
P {X ∈ B} = f (x)dx (1)
B

The function f is called the probability density function of the random variable X.
Z ∞
Remark 2. • f (x)dx = 1
−∞
Z b
• P {a ≤ X ≤ b} = f (x)dx
a
Z a
• If we let a = b, we get P {X = a} = f (x)dx = 0
a
Z a
• Cumulative distribution function F (a) = P {X ≤ a} = P {X < a} = f (x)dx
−∞

d
• Differentiating both sides of the previous equation we get da F (a) = f (a)

11
EXAMPLE: 3.2. The lifetime in hours of a certain kind of radio tube is a random variable
having a probability density function given by
(
0, x ≤ 100
f (x) = 100
x2
, x > 100

What is the probability that exactly 2 of 5 such tubes in a radio set will have to be replaced
within the first 150 hours of operation? Assume that the events Ei , i = 1, 2, 3, 4, 5, that the
ith such tube will have to be replaced within this time are independent.

SOLUTION: 3.2. Z 150 Z 150

P (Ei ) = f (x)dx = 100 x−2 dx = 1/3
0 100
2 3
5 1 2
Since, Ei are independent events, we have the desired probability is C 2 3 3

Remark 3. If X is a continuous random variable with probability density function f (x),

Z ∞
then E[g(X)] = g(x)f (x)dx
−∞

Exercise

1. Suppose that X is a continuous random variable whose probability density function is

given by (
c(4x − 2x2 ), 0 < x < 2
f (x) =
0, otherwise
a) What is the value of c?
b) Find P {X > 1}

2. The amount of time in hours that a computer functions before breaking down is a
continuous random variable with probability density function given by
(
λe−x/100 , x≥0
f (x) =
0, x, 0

What is the probability that

a) a computer will function between 50 and 150 hours before breaking down?
b) it will function for fewer than 100 hours?
Ans: .383, .632

12
3. A continuous random variable has the distribution function

0,

 x≤1
F (x) = c(x − 1)4 , 1≤x≤3


1, x > 3

Find c and also the probability density function.

4. The time t years required to complete a software project has probability density func-
tion (
kt(1 − t), 0 ≤ t ≤ 1
f (t) =
0, otherwise
Find k and also the probability that the project will be completed in less than 4 months.
Ans: 7/27

5. Find k such that (

kx2 , 0 < x < 3
f (x) =
0, otherwise
is a probability density function. Also compute (i) P {1 < X < 2} (ii) P {X ≤ 1}, (iii)
P {X > 1}, (iv) mean and variance.

6. A random variable X has the density function f (x) = k/(1 + x2 ), −∞ < x < ∞.
Determine k and hence evaluate (i) P {X ≥ 0} (ii) P {0 < X < 1}.
Ans: 1/2, 1/4

7. The probability density function of X, the lifetime of a certain type of electronic device
(measure in hours) is given by
(
0, x ≤ 10
f (x) = 10
x2
, x > 10

(a) Find P {X > 20}

(b) What is the cumulative distribution function of X?
(c) What is the probability that of 6 such types of devices, at least 3 will function for
at least 15 hours?

8. The density function of X is given by

(
a + bx2 , 0 ≤ x ≤ 1
f (t) =
0, otherwise

3
If E[X] = , find a and b.
5
9. Consider the function
(
C(2x − x2 ), 0 < x < 5/2
f (t) =
0, otherwise
13
Could f be a probability density function? If so, determine C.