Chapter 5

STATS 10 Introduction to Statistical Reasoning

Maria Cha
Randomness

• What is random?
: no predictable pattern occurs and no digit
is more likely to appear than any other.

2
Randomness to Probability
• A random phenomenon
is a situation in which
we know what
outcomes could
happen, but we don't
know which particular
outcome did or will
happen.

• However, we can
calculate the probability
with which each
outcome will happen .

3
Randomness to Probability
• Let’s say there are 1000 songs in your iTunes
Library and only 5 of these songs are by Justin
Bieber. What is the probability that next time you
hit ‘shuffle’ you get a Justin Bieber song?

• A : 0.5
• B : 0.05
• C : 0.005
• D:5

4
Probability
• The probability of an event is its relative
frequency. We have two kinds of probabilities:

• 1. Theoretical probability: the relative frequency

at which an event happens after infinitely many
repetitions.
• Ex: We may not be able to predict which
song we play each time we hit shuffle,
however we know that in the long run 5 out of
1000 songs will be Justin Bieber songs.

5
Probability
• 2. Empirical probability: relative frequencies
based on an experiment or on observations of a
real-life process.
• Ex: We listen to 100 songs on shuffle and 4
of them are Justin Bieber songs.The
empirical probability is 4/100=0.04

• Difference : Theoretical probabilities are always

the same value; while empirical probabilities
change with every experiment.

6
Probability
• We need both theoretical and empirical
probabilities.
• It is impossible to carry out an experiment that will
provide the exact value of a theoretical probability
: we use empirical probabilities to 1) estimate and
to 2) test theoretical probabilities.

• 1) Estimate when it is too difficult to compute a

theoretical probability.
• 2) Test to verify the theoretical probability is
correct.

7
1. Finding Theoretical Probabilities

8
Terminologies
• For any random phenomenon, each attempt is
called a trial and each trial generates an outcome.
• Each time you hit shuffle is a trial.
• The song that plays as a result of hitting shuffle
is an outcome.
• Sample space is the collection of all possible
outcomes of a trial
• Sample space is the entire iTunes library of
1000 songs.
• A combination of outcomes is called an event
• For example, “playing two Justin Bieber songs
in a row in shuffle” is an event.
9
Terminologies
• The trials are independent : the outcome of one trial
doesn’t influence or change the outcome of another.

• If the iTunes shuffle is truly random then the songs

played are independent of each other.
• In other words, the iTunes shuffle is memoryless, it
does not say to itself “Wait, I just played an Justin
Bieber song, I shouldn't play one again”
• Similarly, if the genders of kids a couple has are
independent, then the probability of having a boy for a
second child doesn't change based on whether or not
the first child of the couple was a girl.

10
Terminologies
Example : Die roll

• Trial: Each die roll

• Outcome: A die has six
sides
• Sample Space:
• Rolled once:
S = {1,2,3,4,5,6}
• Independence: The
outcome of one die roll
does not affect the
outcome of the next

11
Theoretical Probability: Properties
• Probabilities are always numbers between 0 and
1.
0 ≤ 𝑃𝑟𝑜𝑏(𝑒𝑣𝑒𝑛𝑡) ≤ 1
• If 0, the event never happens.
• If 1, the event always happens.
• Probabilities can be expressed as fractions,
decimals, or percents: ½, 0.5, 50%

• The probability of the set of all possible

outcomes, sample space (𝑆) of a trial must be 1.

𝑃𝑟𝑜𝑏 𝑆 = 1
12
Calculating probability of an event
• When the outcomes are equally likely, the
probability that a particular event A occurs is
just the number of outcomes that make up that
event A, divided by the total number of equally
likely outcomes in the sample space, S.

Number of outcomes in A
𝑃𝑟𝑜𝑏 𝐴 =
Number of all possible outcomes in S

13
Complement Rule
• The set of outcomes that are not in the event A is
called the complement of A, denoted Ac. The
probability of an event occurring is 1 minus the
probability that it does not occur.
𝑃𝑟𝑜𝑏 𝐴! = 1 − 𝑃𝑟𝑜𝑏 𝐴
or,
𝑃𝑟𝑜𝑏 𝐴 = 1 − 𝑃𝑟𝑜𝑏 𝐴!

*Recall that 𝑃𝑟𝑜𝑏 𝑆 = 1.

14
Complement Rule
• Solve simple exercises using the complement rule:

• Exercise: Coin toss

• Denote the events:
𝐻: getting a ‘head’ from a coin toss
𝑇: getting a ‘tail’ from a coin toss
• If the coin is fair, the equation below holds:
𝑃𝑟𝑜𝑏 𝐻 + 𝑃𝑟𝑜𝑏 𝑇 = 0.5 + 0.5 = 1
• If the coin is not fair and we are given that
𝑃𝑟𝑜𝑏 𝐻 = 0.4. What is 𝑃𝑟𝑜𝑏 𝑇 ?

15
Probability of multiple events
• 1. Disjoint Events (Mutually Exclusive events) : Events
that have no outcomes in common (and, thus, cannot
occur together)
• The outcome of a coin toss cannot be a head and a tail
• A student cannot fail and pass a class
• A card drawn from a deck cannot be an ace and a
queen
• 2. Combined Events
• 1) Event ‘A and B’ : Event that Event A and B
happen at the same time.
• 2) Event ‘A or B’ : Either Event A or Event B, or both
happen.

16
Probability of multiple events
• Venn Diagram for the events where 1) disjoint, 2) A
and B, 3) A or B

• Find these events in the example.

17
Finding Theoretical probabilities
• Example : We recorded whether the students
attended the class and also whether they pass or fail
the quiz after the class. (Event A : A student attended
the class ; Event B : A student passed the quiz )
Pass Fail Total
Attend 35 5 40
Absence 0 10 10
Total 35 15 50

• Define Prob(A) : Probability that a randomly selected

student attended the class.

18
Finding Theoretical probabilities
• Example : We recorded whether the students
attended the class and also whether they pass or fail
the quiz after the class. (Event A : A student attended
the class ; Event B : A student passed the quiz )
Pass Fail Total
Attend 35 5 40
Absence 0 10 10
Total 35 15 50

• Calculate Prob(A), Prob(B), Prob(Ac) and Prob(Bc).

• Find the disjoint event(s).
• Define events ‘A and B’, ‘A or B’ and find their
probabilities.
19
Addition Rule
• Now we want to generalize the probability of event ‘A or
B’ in an equation.
Prob(A or B) = Prob(A) + Prob(B) – Prob(A and B)

• (Special case) If A and B are disjoint, i.e. Prob(A and

B)=0
Prob(A or B) = Prob(A) + Prob(B)

20
Addition Rule
• Find the probabilities using the addition rule.
Pass (B) Fail (Bc) Total
Attend (A) 35 5 40
Absence (Ac) 0 10 10
Total 35 15 50

• Probability that a randomly selected student either

attended the class or passed the quiz:
• Prob(A or B) = Prob(A) + Prob(B) - Prob(A and B)
= 40/50 + 35/50 – 35/50 = 0.8
• Probability that a randomly selected student either not
attended the class or passed the quiz:
• Prob(Ac or B) = Prob(Ac) + Prob(B)
= 10/50 + 35/50 = 0.9
21
Conditional Probability
• Given that a randomly selected student attended the
class, what is the probability that the student passed the
quiz?
Pass (B) Fail (Bc) Total
Attend (A) 35 5 40
Absence (Ac) 0 10 10
Total 35 15 50

• Conditional probabilities: we focus on just one group of

objects and imagine taking a random sample from that
group alone. Probability of Event A given that Event B
occurred is,
𝑃𝑟𝑜𝑏 (𝐴 𝑎𝑛𝑑 𝐵 )
𝑃𝑟𝑜𝑏(𝐴|𝐵) =
𝑃𝑟𝑜𝑏 (𝐵)
22
Conditional Probability
• Again, given that a student attended the class, the
probability that the student passed the quiz is,
Prob(B|A) = Prob(B and A) / Prob(A)
= 35/50 / 40/50 =0.875
• Given that a student passed the quiz, find the probability
that the student actually attended the class.
Prob(A|B) = Prob(A and B) / Prob(B)
= 35/50 / 35/50 = 1
This is called ‘posterior probability.’
Pass (B) Fail (Bc) Total
Attend (A) 35 5 40
Absence (Ac) 0 10 10
Total 35 15 50

23
Conditional Probability
• Note1: “Given that” vs. “And”
• Probability that a randomly selected student passed
a quiz given that the student attended the class :
Prob(Pass|Attend)
• Probability that a randomly selected student who
attended the class passed a quiz : Prob(Pass and
Attend)

• Note2: Flipping the condition

• Prob(B|A) ≠ Prob(A|B)
• Or,
Prob(B|A) ≠ 1/Prob(A|B)

24
Independence
• The occurrence of one event doesn’t influence or
change the occurrence of another.

• Conditional Probability and Independence : Event A

and B are independent if knowledge that one event
has happened tells you nothing about whether or
not the other event has happened. In mathematical
notation,
Prob(A|B)=Prob(A)

25
Independence
• If A and B are independent,
Prob(A|B) = Prob(A)

• By definition,
Prob(A|B) = Prob(A and B) / Prob(B),
It satisfies,
Prob(A) = Prob(A and B) / Prob(B)
or
Prob(A and B) = Prob(A) * Prob(B),
if A and B are independent.

26
Independence
Pass (B) Fail (Bc) Total
Attend (A) 35 5 40
Absence (Ac) 0 10 10
Total 35 15 50

• Check if the event A and event B are independent :

Check either 1) Prob(A|B) = Prob(A), or 2) Prob(A and
B) = Prob(A) * Prob(B)
• 1) Prob(A|B) = Prob(A and B) / Prob(B)
= 35/50 / 35/50 = 1
Prob(A) = 40/50 = 0.8 (not equal)
• 2) Prob(A and B) = 35/50 = 0.7
Prob(A) * Prob(B) = 40/50 * 35/50 = 0.56 (not equal)

• Conclusion: They are not independent.

27
Multiplication Rule
• Now we want to generalize the probability of event
‘A and B’ in an equation.

Prob(A and B) = Prob(A|B) * Prob(B)

= Prob (B|A) * Prob(A)

• (Special case) If A and B are independent, i.e.

Prob(A|B)=Prob(A) and Prob(B|A)=Prob(B)

Prob(A and B) = Prob(A) * Prob(B)

28
Application
• (Addition rule : special case) if A and B are disjoint,
Prob(A or B) = Prob(A) + Prob(B)

Thus, if A1, A2, …, and An are all mutually exclusive,

Prob(A1 or A2 or … or An )
= Prob(A1) + Prob(A2) + … + Prob(An)

• (Multiplication rule : special case) If A and B are

independent,
Prob(A and B) = Prob(A) * Prob(B)

Thus, if A1, A2, …, and An are all independent,

Prob(A1 and A2 and … and An )
= Prob(A1)*Prob(A2)*… *Prob(An)
29
Application
• If the event Ac is a complement event of the event A,
then
P(Ac)=1-P(A)

• Likewise, for any event B,

P(Ac|B)=1-P(A|B)
! !! !"# !
(proof) ! !! ! =
! !
! ! − ! ! !"# !
=
! !
! ! !"# !
=1−
! !
= 1 − !(!|!)
30
Example - Guessing at Random
• In a multiple choice exam there are 5 questions and 4
choices for each question (a, b, c, d). Nancy has not
studied for the exam at all, and decided to randomly
guess the answers.

• What is the probability that she gets the ‘first’ question

right, 𝑃(𝐴!)? (Denote the event 𝐴!: she gets the first
question right.)

• What is the probability that she gets the ‘second’

question right, 𝑃(𝐴")? (Denote the event 𝐴": she gets
the second question right.)

• Are the two events above independent?

31
Example - Guessing at Random
• What is the probability that the first question she
gets right is the 5th question?

𝑃 𝐴!" 𝑎𝑛𝑑 𝐴"# 𝑎𝑛𝑑 𝐴"$ 𝑎𝑛𝑑 𝐴"% 𝑎𝑛𝑑 𝐴&

= 𝑃(𝐴!" )×𝑃(𝐴"# )×𝑃(𝐴"$ )×𝑃(𝐴"% )×𝑃(𝐴& )
= 0.75 × 0.75×0.75×0.75×0.25 = 0.0791

• What is the probability that she gets all the

questions right?
• A. 0.255
• B. 0.755
• C. Don’t know

32
Example - Guessing at Random
• What is the probability that she gets at least one
question right?

• If there are 5 questions, the possible number of

questions she gets right are:
S = {0,1,2,3,4,5}
• We are interested in instances where she gets at
least 1 question right:
S = {0,1,2,3,4,5}
• So we can divide up the sample space into two
categories:
S = {0, at least 1}

33
Example - Guessing at Random
• What is the probability that she gets at least one question
right?

• Direct approach :
P(1 question correct or 2 questions correct or 3 questions
correct or … all questions correct)
= P(1 question correct) + P(2 questions correct) +…
+ P(all questions correct)

• Since the event “0 question correct” is the complement

event of the event “at least 1 question correct”, we may
approach this with
1- P(0 question correct) = 1- P(Wrong and Wrong …
and Wrong)=1 - 0.755 = 0.7627
34
Exercise
• Suppose that we are planning a seminar on Saturday
for UCLA undergraduate students. It is known that the
probability that a randomly chosen UCLA student is on
campus on Saturday is 0.3. If a randomly chosen
student is on campus, the probability of the student
attends the seminar is 0.7. However, if a student is not
on campus, then the probability that the student
attends the seminar is 0.2.

• 1. Define the events.

• A : A student is on campus on Saturday
• B : A student attends the seminar

35
Exercise
• Suppose that we are planning a seminar on Saturday
for UCLA undergraduate students. It is known that the
probability that a randomly chosen UCLA student is on
campus on Saturday is 0.3. If a randomly chosen
student is on campus, the probability of the student
attends the seminar is 0.7. However, if a student is not
on campus, then the probability that the student
attends the seminar is 0.2.

• 2. Summarize the information given in the problem.

• P(A) = 0.3
• P(B|A) = 0.7
• P(B|Ac)=0.2

36
Exercise
• Q1. What is the probability that a randomly chosen
UCLA student is on campus on Saturday and attends
the seminar?

• Q2. What is the probability that a randomly chosen

UCLA student is NOT on campus on Saturday but
attends the seminar?

• Q3. What is the probability that a randomly chosen

UCLA student DOES NOT attend the seminar?

• Q4. Given that a student attends the seminar, what is

the probability that the student is NOT on campus on
Saturday?
37
Independent vs. Disjoint
Disjoint events cannot be independent
• Since we know that disjoint events have no outcomes in common,
knowing that one occurred means the other didn't happen.
• Thus, the probability of the one event occurring changed based on
our knowledge that the other occurred.
• Therefore the two events are not independent.

Non-disjoint events may or may not be independent

• Just because two events can occur at the same time does not
mean they need to also be dependent.
• It is possible that two randomly selected students get an A in a
class, but how they do in the class may have nothing to do with
each other.
• However if you and a close friend of yours that you study with both
get an A in a class, how well the two of you do in that class are
possibly dependent on each other.
38
2. Finding Empirical Probabilities

39
Empirical Probabilities
• Relative frequencies based on an experiment or on
observations of a real-life process.

• Source :
• 1. Observed data : From the observations of
real-life events, we can estimate the probability.
• 2. Simulation : If we can’t find the data or the
situation is too complex for us to find the
empirical probability of some random event, we
can simulate the situation.

40
Law of Large numbers
• The Law of Large Numbers (LLN) says that the long-run
relative frequency of repeated independent events gets
closer and closer to the true relative frequency as the
number of trials increases.
• If a coin is tossed many times, the overall percentage of
heads should settle down to about 50% as the number
of tosses increases.

41
Law of Large numbers
• This is why we define probability as the long run
relative-frequency of an event.

• The common misunderstanding of the Law of large

numbers is that random phenomena are supposed
to compensate for whatever happened in the past;
this is just not true and if also called law of
averages (or gambler's fallacy.)

42
Law of averages
• When tossing a fair coin, if heads comes up on
each of the first 10 toss, what do you think the
chance is that another head will come up on the
next toss?
HHHHHHHHHHHHHH?

• The probability is still 0.5, or there is still a 50%

chance that another head will come up on the next
toss.
Prob(H on 11th toss) = Prob(T on 11th toss) = 0.5

• The coin is not due for a tails.

43