Chapter 5.

A survey of probability concepts

1. What is a probability?

1) Probability: A value between 0 and 1, inclusive, describing the

relative possibility (chance or likelihood) an event will occur.

2) Interpretation of probabilities
• A value near zero means the event is not likely to happen.
• A value near one means it is likely.

3) 3 basic key words

• Experiment: The observation of some activity or the act of
taking some measurement.
• Outcome: The particular result of an experiment.
• Event: The collection of one or more outcomes of an
experiment.

• Example: Take a look at the following table.

1
2. Approaches to assigning probabilities

1) Three approaches to assigning probabilities

• Classical
• Empirical

2) Classical probability
• Based on the assumption that the outcomes of an experiment
are equally likely.
• To find the probability of a particular outcome, we divide the
number of favorable outcomes by the total number of possible
outcomes:
Number of favorable outcomes
Probability of an event = .
Total number of possible outcomes

• Example: Consider an experiment of rolling a six-sided die.

What is the probability of the event “an even number of spots
appear face up”?
The possible outcomes are:

There are three “favorable” outcomes (a two, a four, and a six)

in the collection of six equally likely possible outcomes. (To
emphasize the equal likelihood, we use an expression, a “fair”
die.) Therefore:
3
Probability of an even number = = 0.5.
6

2
• Note!
a. Mutually exclusive: Events are mutually exclusive if the
occurrence of any one event means that none of the others
can occur at the same time. (In a die-tossing experiment,
every outcome will be either an even number or an odd
number.)
b. Collectively Exhaustive: Events are collectively exhaustive
if at least one of the events must occur when an experiment
is conducted. (In a die-tossing game, the set of outcomes is
collectively exhaustive.)
c. If the set of events is collectively exhaustive and the events
are mutually exclusive, the sum of the probabilities equals 1.

3) Empirical probability
• Relative frequencies.
• The probability of an event happening is the fraction of the
time similar events happened in the past.
• Formula:
Number of times the event occurs
Empirical Probability = .
Total number of observations

• Law of large numbers

a. The empirical probability is based on the law of large
numbers.
b. Definition: Over a large number of trials the empirical
probability of an event will approach its true probability.
c. The key point is that more observations will provide a more
accurate estimate of the probability.

3
• Example of LLN: Suppose we toss a fair coin. The result of
each toss is either an H or a T. If we toss the coin many times,
the probability of getting H’s will get close to 0.5. The
following table (on 131) reports the results of an experiment of
tossing a fair coin 1, 10, 50, 100, 500, 1,000 and 10,000 times
and computing the relative frequency of H’s.

• Example of empirical probability: On February 1, 2003, the

Space Shuttle Columbia exploded. This was the second
disaster in 113 space missions for NASA. On the basis of this
information, what is the probability that a future mission is
successfully completed?
# of successful flights
Probability of a successful flight = .
Total # of flights
To simplify, letters or numbers may be used. P stands for
probability, and in this case P(A) stands for the probability that
a future mission is successfully completed.
111
𝑃𝑃(𝐴𝐴) = = .98.
113

4
4. Some rules for computing probabilities

1) Rules of addition
• Special Rule of Addition: If two events A and B are mutually
exclusive, the probability of one or the other event’s occurring
equals the sum of their probabilities.
𝑃𝑃(𝐴𝐴 𝑜𝑜𝑜𝑜 𝐵𝐵) = 𝑃𝑃(𝐴𝐴 ∪ 𝐵𝐵) = 𝑃𝑃(𝐴𝐴) + 𝑃𝑃(𝐵𝐵)
(Call 𝐴𝐴 ∪ 𝐵𝐵 as A union B)

• Recall that mutually exclusive means when an event occurs,

none of the others can occur at the same time.

• Complement Rule: The complement rule is used to determine

the probability of an event occurring by subtracting the
probability of the event not occurring from 1.
𝑃𝑃(𝐴𝐴) + 𝑃𝑃(~𝐴𝐴) = 1 or 𝑃𝑃(𝐴𝐴) = 1 − 𝑃𝑃(~𝐴𝐴).
(Call ~𝐴𝐴 (or 𝐴𝐴𝑐𝑐 ) as not A or A complement)

5
• Example: A sample of employees of World Enterprises is to be
surveyed about a new health care plan. The employees are
classified as follows:

Classification Event # of Employees P(Event)

Supervisors A 120 0.06
Maintenance B 50 0.025
Production C 1,460 0.73
Management D 302 0.151
Secretarial E 68 0.034
a. What is the probability that the first person selected is:
(i) either in maintenance or a secretary?
𝑃𝑃(𝐵𝐵) + 𝑃𝑃(𝐸𝐸 ) = 0.025 + 0.034 = 0.059

(ii) not in management?

1 − 𝑃𝑃(𝐷𝐷) = 1 − 0.151 = 0.849.

b. Draw a Venn diagram illustrating your answers to part (a).

c. Are the events in part (a)(i) complementary or mutually

exclusive or both? Mutually exclusive but not
complementary.

6
• General Rule of Addition: What if the events are not mutually
exclusive? In that case the general rule of addition is used.
The probability that event A or B happens, i.e., P(A or B), is
computed by the following formula:
P(A or B) = P(A) + P(B) - P(A and B)
𝑃𝑃(𝐴𝐴 ∪ 𝐵𝐵) = 𝑃𝑃(𝐴𝐴) + 𝑃𝑃(𝐵𝐵) − 𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵)
(Call 𝐴𝐴 ∩ 𝐵𝐵 as A intersection B)

• Joint probability: A probability that measures the likelihood

two or more events will happen concurrently.

• Example: What is the probability that a card chosen at random

from a standard deck of cards will be either a king or a heart?
Card (Event) Probability Explanation
King (A) 4 4 kings in the deck of
52 52 cards
Heart (B) 13 13 hearts in a deck of
52 52 cards
King of Heart 1 1 king of heart in a
(A and B) 52 deck of 52 cards

𝑃𝑃(𝐴𝐴 ∪ 𝐵𝐵) = 𝑃𝑃(𝐴𝐴) + 𝑃𝑃(𝐵𝐵) − 𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵)

4 13 1 16
= + − = 𝑜𝑜𝑜𝑜 0.3077.
52 52 52 52

7
2) Rules of multiplication (How to get the joint probability? – Part 1)
• Special Rule of multiplication
a. Independence: Two events A and B are independent if the
occurrence of one has no effect on the probability of the
occurrence of the other. Note! The independency of A and
B does not mean that they are mutually exclusive.

b. Special Rule of multiplication: For two independent events

A and B, the probability that A and B will both occur is
found by multiplying the two probabilities.
𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵) = 𝑃𝑃(𝐴𝐴)𝑃𝑃(𝐵𝐵).

• Example: A survey by the American Automobile Association

(AAA) revealed 60 percent of its members made airline
reservations last year. Two members are selected at random.
What is the probability both made airline reservations last year?
Solution:
The probability that the first member made an airline
reservation last year is .60 written as 𝑃𝑃(𝑅𝑅1 ) = 0.60. The
probability that the second member selected made a
reservation is also .60, so that 𝑃𝑃(𝑅𝑅2 ) = 0.60. Since the
number of AAA members is very large, you may assume that
𝑅𝑅1 and 𝑅𝑅2 are independent.

𝑃𝑃(𝑅𝑅1 ∩ 𝑅𝑅2 ) = 𝑃𝑃(𝑅𝑅1 )𝑃𝑃(𝑅𝑅2 ) = (0.6)(0.6) = 0.36

Similarly,
𝑃𝑃(𝑅𝑅1 ∩ 𝑁𝑁𝑅𝑅2 ) = 𝑃𝑃(𝑅𝑅1 )𝑃𝑃(𝑁𝑁𝑁𝑁2 ) = (0.6)(0.4) = 0.24
𝑃𝑃(𝑁𝑁𝑅𝑅1 ∩ 𝑅𝑅2 ) = 𝑃𝑃(𝑁𝑁𝑅𝑅1 )𝑃𝑃(𝑅𝑅2 ) = (0.4)(0.6) = 0.24
𝑃𝑃(𝑁𝑁𝑅𝑅1 ∩ 𝑁𝑁𝑅𝑅2 ) = 𝑃𝑃(𝑁𝑁𝑁𝑁1 )𝑃𝑃(𝑁𝑁𝑁𝑁2 ) = (0.4)(0.4) = 0.16

8
• General rule of multiplication
a. Conditional Probability
– A conditional probability is the probability of a
particular event occurring, given that another event has
occurred.
– The probability of the event A given that the event B has
occurred is written P(A|B).

b. General Multiplication Rule

– We use the general rule of multiplication to find the
joint probability of two events when the events are not
independent.
– It states that for two events, A and B, the joint
probability that both events will happen is found by
multiplying the probability that event A will happen by
the conditional probability of event B occurring given
that A has occurred.

– Formula:
P(A ∩ B) = P(A)P(B|A)

• Note! We can also apply this general rule to the independent

events, which means that 𝑃𝑃(𝐵𝐵|𝐴𝐴) = 𝑃𝑃(𝐵𝐵) or 𝑃𝑃(𝐴𝐴|𝐵𝐵) = 𝑃𝑃(𝐴𝐴)
for the independent events, A and B.

• Example 1: For two independent events, A and B, P(A) =0.4

and P(B)=0.2.
a. Find 𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵). 0.08

b. Find 𝑃𝑃(𝐴𝐴|𝐵𝐵). 0.4

c. Find 𝑃𝑃(𝐴𝐴 ∪ 𝐵𝐵) 0.52

9
• Example 2: A golfer has 12 golf shirts in his closet. Suppose 9
of these shirts are white and the others blue. He gets dressed
in the dark, so he just grabs a shirt and puts it on. He plays
golf two days in a row and does not do laundry. What is the
likelihood both shirts selected are white?

Solution:
The event that the first shirt selected it white is 𝑊𝑊1 . The
probability is 𝑃𝑃(𝑊𝑊1 ) = 9/12. The event that the second shirt
selected is also white is identified as 𝑊𝑊2 . The conditional
probability that the second shirt selected is white, given that
the first shirt selected is also white, is 𝑃𝑃(𝑊𝑊2 |𝑊𝑊1 ) = 8/11. To
determine the probability of 2 white shirts being selected we
use formula:
9 8
𝑃𝑃(𝑊𝑊1 ∩ 𝑊𝑊2 ) = 𝑃𝑃(𝑊𝑊1 )𝑃𝑃(𝑊𝑊2 |𝑊𝑊1 ) = � � � � = 0.55.
12 11

10
5. Contingency Tables (How to get the joint probability? – Part 2)
1) Contingency table: Contingency table is a table used to
classify sample observations according to two or more
identifiable characteristics.

• Example 1: A survey of 150 adults classified each as to

gender and the number of movies attended last month.
Each respondent is classified according to two criteria—the
number of movies attended and gender.

a. Determine 𝑃𝑃(𝑀𝑀)? 70/150=0.47

b. Determine 𝑃𝑃(𝑁𝑁0 |𝑊𝑊)? 40/80=1/2

c. Determine P(𝑀𝑀 ∩ 𝑁𝑁1 )?

70 20
P(M ∩ N1 ) = P(M)P(N1 |M) = = 0.13.
150 70

11
6. Principles of counting

1) Multiplication
• Example: An automobile dealer wants to advertise that for
$29,999 you can buy a convertible, a two-door sedan, or a
four-door model with your choice of either wire wheel covers
or solid wheel covers. How many different arrangements of
models and wheel covers can the dealer offer?

• Multiplication formula: If there are m ways of doing one thing,

and n ways of doing another thing, there are m × n ways of
doing both.

12
2) Permutation
• Example 1: Jack has three T-shirts (white, grey, and dark blue)
and a three pairs of pants (white, grey, and dark blue).
a. In how many different ways, can he wear a shirt and pants?
(9)

b. Assume that he doesn’t match a T-shirt with the same color

of pants. In how many different ways, can he wear a shirt
and pants?
(6)

• The permutation is an arrangement of objects or things

wherein order is important. That is, each time, the objects are
placed in a different order, a new permutation results.

• A permutation is any arrangement of r objects selected from n

possible objects. The order of arrangement is important in
permutations.

𝑛𝑛!
𝑛𝑛𝑃𝑃𝑟𝑟 = ,
(𝑛𝑛 − 𝑟𝑟)!
where:
𝑛𝑛 is the total number of objects.
𝑟𝑟 is the number of objects selected.

• Note: The 𝑛𝑛! is a notation called “n factorial.” For example,

4! means 4 times 3 times 2 times 1 = 4 × 3 × 2 × 1 = 24.

13
3) Combination
• If the order of the selected objects is not important, any
selection is called a combination.

• Example 1: If executives Anne, Barbara, and Chelsea are to

be chosen as a committee to negotiate a merger, there is only
one possible combination of these three; the committee of
Anne, Barbara, and Chelsea is the same as the committee of
Barbara, Chelsea, and Anne.

• Formula:
𝑛𝑛!
𝑛𝑛𝐶𝐶𝑟𝑟 =
𝑟𝑟! (𝑛𝑛 − 𝑟𝑟)!

• Example 2: There are 12 players on the Carolina Forest High

School basketball team. Coach Thompson must pick five
players among the twelve on the team to comprise the starting
lineup. How many different groups are possible?

12! (12∙11∙10∙9∙8)∙7!
12𝐶𝐶5 =
5!7!
=
(5∙4∙3∙2∙1)∙7!

14
7. Bayes’ theorem

4) Probability revision using Bayes’ theorem

• Prior probability: We begin with initial or prior probability
estimates for specific event of interest.
• New information: We obtain additional information about the
events.
• Application of Bayes’ theorem
• Posterior probability: We update the prior probability values
by calculating revised probabilities, referred to as posterior
probabilities.

5) Bayes’ theorem (for two mutually exclusive events 𝐴𝐴1 and 𝐴𝐴2 )
𝑃𝑃(𝐴𝐴1 ∩ 𝐵𝐵) 𝑃𝑃(𝐴𝐴1 )𝑃𝑃(𝐵𝐵|𝐴𝐴1 )
𝑃𝑃(𝐴𝐴1 |𝐵𝐵) = =
𝑃𝑃(𝐵𝐵) 𝑃𝑃(𝐴𝐴1 )𝑃𝑃(𝐵𝐵|𝐴𝐴1 ) + 𝑃𝑃(𝐴𝐴2 )𝑃𝑃(𝐵𝐵|𝐴𝐴2 )
𝑃𝑃(𝐴𝐴2 ∩ 𝐵𝐵) 𝑃𝑃(𝐴𝐴2 )𝑃𝑃(𝐵𝐵|𝐴𝐴2 )
𝑃𝑃(𝐴𝐴2 |𝐵𝐵) = =
𝑃𝑃(𝐵𝐵) 𝑃𝑃(𝐴𝐴1 )𝑃𝑃(𝐵𝐵|𝐴𝐴1 ) + 𝑃𝑃(𝐴𝐴2 )𝑃𝑃(𝐵𝐵|𝐴𝐴2 )

6) Bayes’ theorem (for n mutually exclusive events 𝐴𝐴1 , 𝐴𝐴2 , … , 𝐴𝐴𝑛𝑛 )

𝑃𝑃(𝐴𝐴𝑖𝑖 ∩ 𝐵𝐵) 𝑃𝑃(𝐴𝐴𝑖𝑖 )𝑃𝑃(𝐵𝐵|𝐴𝐴𝑖𝑖 )
𝑃𝑃(𝐴𝐴𝑖𝑖 |𝐵𝐵) = = 𝑛𝑛
𝑃𝑃 (𝐵𝐵) ∑𝑗𝑗=1 𝑃𝑃�𝐴𝐴𝑗𝑗 �𝑃𝑃�𝐵𝐵|𝐴𝐴𝑗𝑗 �

15
7) Example for two-event case

Historical data suggest that the quality ratings of the two

suppliers are shown in the following table:

Percentage Percentage
Good Parts Bad Parts
Supplier 1 98 2
Supplier 2 95 5

Currently, 65% of the parts purchased by the company are from

supplier 1 and the remaining 35% are from supplier 2. Given the
information that the part is bad, what is the probability that it
came from supplier 1 and what is the probability that it came
from supplier 2?

Let 𝐴𝐴1 denote the event that a part is from supplier 1 and 𝐴𝐴2
denote the event that a part is from supplier 2. If a part is selected
at random we would assign the prior probabilities:
𝑃𝑃(𝐴𝐴1 ) = 0.65 and 𝑃𝑃(𝐴𝐴2 ) = 0.35
If we let 𝐺𝐺 denote the event that a part is good and 𝐵𝐵 denote the
event that a part is bad, the information in the above table
provides the following conditional probabilities:
𝑃𝑃(𝐺𝐺|𝐴𝐴1 ) = 0.98 𝑃𝑃(𝐵𝐵|𝐴𝐴1 ) = 0.02
𝑃𝑃(𝐺𝐺|𝐴𝐴2 ) = 0.95 𝑃𝑃(𝐵𝐵|𝐴𝐴2 ) = 0.05
Thus, the probability that a bad part came from supplier 1:
(𝐴𝐴1 ∩ 𝐵𝐵) 𝑃𝑃(𝐴𝐴1 )𝑃𝑃(𝐵𝐵|𝐴𝐴1 )
𝑃𝑃(𝐴𝐴1 |𝐵𝐵) = =
𝑃𝑃(𝐵𝐵) 𝑃𝑃(𝐴𝐴1 )𝑃𝑃(𝐵𝐵|𝐴𝐴1 ) + 𝑃𝑃(𝐴𝐴2 )𝑃𝑃(𝐵𝐵|𝐴𝐴2 )
(0.65)(0.02) 0.0130
= = = 0.4262
(0.65)(0.02) + (0.35)(0.05) 0.0305

16
Similarly,
(𝐴𝐴2 ∩ 𝐵𝐵) 𝑃𝑃 (𝐴𝐴2 )𝑃𝑃(𝐵𝐵|𝐴𝐴1 )
𝑃𝑃(𝐴𝐴2 |𝐵𝐵) = =
𝑃𝑃(𝐵𝐵) 𝑃𝑃(𝐴𝐴1 )𝑃𝑃(𝐵𝐵|𝐴𝐴1 ) + 𝑃𝑃(𝐴𝐴2 )𝑃𝑃(𝐵𝐵|𝐴𝐴2 )
(0.35)(0.05) 0.0175
= = = 0.5738
(0.65)(0.02) + (0.35)(0.05) 0.0305

17