5/15/24, 10:42 AM Probability

X
Probability

Probability Theory
Introduction
The science of statistics is based upon considerations of "probability".
But what does "probability mean"? The only thing we can say for
certain is that it is a number between 0 and 1 (it is never <0 or >1.0).
What does it represent? We also know that probability refers to
uncertain events. A typical example of an event is: "it will rain
tomorrow between 3pm and 4pm". Nobody can know for
certain whether or not this will happen. Another example of an event
is: "I toss a coin in the air and it lands with heads upwards" (we will
use this event a lot!). Probability refers to uncertain events.

The definition of probability used by statisticians is that it represents

an individual's degree of belief or estimate of the likelihood of an
uncertain event occurring.

As you can see, this depends on the individual. This is not surprising
because, for example, in the rain example above, if you asked 100
people for their probability estimate (between 0% - it certainly will not
happen, and 100% - it certainly will happen) you will most likely get
100 different answers. Nobody can be right or wrong in this
estimate.

If this is so, how can we "calculate" probabilities? and in this

course we will spend a lot of time doing just that!

The answer is we cannot calculate probabilities. We can just use

logic to work out what is the most rational number to take based
on any relevant facts that we are aware of.

https://lds.maiasoft.com/must/emodule.jsp 1/12
5/15/24, 10:42 AM Probability

So we do not just pluck a number out of thin air! We need to make a

rational estimate based on what we know. For the rain example, if it
X
had rained every day between 3 and 4 for the past week, we might
well assess the chance for tomorrow as fairly high. It might depend
whether or not we had heard the weather forecast as well. And so on.
On the other hand if somebody asks for probability that a foreign
football team in a country you know nothing about, will beat another
team in that country then you have little information to base a sensible
assessment and might just say 50%. If you know the person asking
you and you suspect their motives, maybe you might change your
view accordingly.

The point is that the idea of "calculating" the likelihood of a future

event about which you have little information is plain silly - you cannot
do it. On the other hand in the situations we will be covering we will
have a have a lot of relevant numerical information in which case we
can make rational, sensible estimates.

Probability

➤ Probability is a number between 0 and 1

A probability of 0 represents something impossible (things start
➤
to fall up instead of down)
➤ A probability of 1 represents something certain (it will rain in
Kuala Lumpur!)
A number in between represents something uncertain
➤
(anything in the future)
The nearer to 1 the probability, the more likely you believe it
➤
will happen and the nearer to 0 the less likely
A probability of ½ represents a belief that something may or
➤
may not happen with equal probability

Probability represents a person's degree of uncertainty about

something.

Events

https://lds.maiasoft.com/must/emodule.jsp 2/12
5/15/24, 10:42 AM Probability

Events in probability theory are things which may or may not

➤
occur
X
Example: I find a $100 note on the road
➤
Example: I get home in less than 1 hour after class
➤
We assign a probability to an event
➤
➤ If the event is called E then the probability that E occurs is
called p(E) or Prob(E)

The Universal Set

Events E are subsets of a universal set Ω which contains all
➤
possible events.
Note: all possible events does not mean anything which is
➤
possible to occur in the whole universe! It is the set of all
possible outcomes of an uncertain situation.
➤ The Event with nothing in it is called the empty set and
denoted by the Greek symbol phi written φ
➤ By definition P(φ)=0 and P(Ω)=1
Example: toss a coin. The event is which side lands face upwards.
The universal set for this experiment is Heads and Tails there are no
other possibilities (we normally discount the coin landing on its edge!).
If we throw a 6 sided dice the universal set is the set {1,2,3,4,5,6}
depending on how many spots are face upwards. Note: P(Ω)=1
means simply that something must happen. We may not know what
exactly, but definitely something must happen! Even if we do not know
whether a coin will land Heads or Tails, we know it will land on one of
them for sure. So the probability of Heads OR Tails must be 1.0

Union/Intersection of events.
DISJOINTNESS
We use A∪B to mean A union B (all elements in either or A or
➤
B or both)
A∩B means A intersect B (all elements in both A and B)
➤

https://lds.maiasoft.com/must/emodule.jsp 3/12
5/15/24, 10:42 AM Probability

➤ We call two events disjoint if their intersection is empty. i.e

A∩B = φ This means that EITHER A can happen OR B can
X
happen but they cannot both happen at the same time.

Examples of disjoint events

A=you get Heads when you toss a coin
B=you get Tails when you toss a coin

You throw 2 dice

A=event you score 2 or 3
B=event you score 11 or 12

Non-disjoint events
You throw two dice.
A=event that you score 2,3,4,5 or 6
B=event that you score 3,5,7,10,12

In this case the intersection A∩B is the event that you score 3 or 5 so
that A happens AND B happens at the same time.

When we say all possible events we are referring to a particular

situation. Typical examples we will use are things like:

➤ Whether or not you get Heads or Tails if you toss a coin

➤ The number you get when you throw a dice
The total you get when you throw two dice
➤
➤ The range of values which are possible representing the time it
takes you to travel from home to work.

Universal set = all possibilities

Example: throw a dice Ω={1,2,3,4,5,6}
throw 2 dice Ω={2,3,4,5,6,7,8,9,10,11,12}

https://lds.maiasoft.com/must/emodule.jsp 4/12
5/15/24, 10:42 AM Probability

THE COMPLEMENTARY EVENT

X

Given an event called E the complementary event is the event that E

does NOT happen.
We denote the event complementary to E as Ec

Example: Throw a dice. E=event that you get 1 or 2 then Ec is the

event that you get 3,4,5 or 6

Example: Toss a coin. E = event you get Heads then Ec is the event
you get Tails

THE ADDITION LAW OF PROBABILITY

If A and B are two DISJOINT events then P(A∪B) = P(A)+P(B)
NOTE: the events MUST be DISJOINT - otherwise it is not true.

For more than two DISJOINT events we can extend it to eg

P(A∪B∪C∪D) = P(A)+P(B)+P(C)+P(D)

This is saying that if two events cannot happen together (they are
disjoint) then the probability that either A OR B occurs is the sum of the
probabilities of A and B, that is: P(A)+P(B).

Example: throw 2 dice

Let A=[score 2]
Let B=[score 3 or 4]

We will see later that Prob(A)=1/36 and Prob(B)=5/36

So Prob[score 2 or 3 or 4] = 1/36 + 5/36 = 1/6

In fact we calculate Prob(B) using the addition law as Prob(3)+Prob(4)

A∪B= [score: 2,3 or 4]

https://lds.maiasoft.com/must/emodule.jsp 5/12
5/15/24, 10:42 AM Probability

The complementary event again

Suppose E is any event and Ec is its complementary event. What can

we say? First we can say that E and Ec are disjoint (because an
event happens or does not happen we can't have both!!)

BUT we can also say that EITHER E happens OR it does not happen -
there are no other possibilities.

Technically we can say E ∪ Ec = Ω (the universal set)

But we know that P(Ω)=1.0 so from the addition law we have the very
important conclusion that

P(E) + P( Ec) = 1
which means that the probability that something occurs plus the
probability that it does NOT occur = 1.0

Why you may ask is this important? The answer is that it is

sometimes MUCH easier to calculate Prob(Ec)
rather than P(E) so we can derive P(E) = 1-P(Ec)

We will use this a lot: P(E)=1-P(Ec )

Examples

Selection of an event from many (basis of

much of what will follow)
https://lds.maiasoft.com/must/emodule.jsp 6/12
5/15/24, 10:42 AM Probability

We often have a situation where we want to calculate the probability of

the occurrence of an event when there are many possible (random)
X
possibilities.
The rule is that the probability of the event is, assuming all
outcomes are equally likely

the number of ways the event can happen

number of all possible random outcomes
We will look at this in a lot more detail for specific types of simple
events (see module Sets and Counting). But for now let us use the
example of tossing two coins. Suppose we want the probability that
the coins are showing different faces. We can reason like this:

1) Assume both coins are fair that means that Prob(H) = Prob(T) =
50% THIS IS AN ASSUMPTION
2) The number of possible outcomes is 4 all of which are equally likely
according to assumption 1. The four outcomes are: {HH,HT,TH,TT}
where the order of H, T indicates face of coin 1, followed by face of
coin 2.
3) The outcomes we are interested in are {HT, TH} and there are two.
4) We deduce the probability (meaning our sensible, logical
assessment based on our assumptions) is 2/4 or 1/2 = 0.5 or 50%.

Random Variables

➤ A random variable is a number which depends on a

random event AND (more or less) we are given the
probability of every possible value that the random
variable can take.
➤ Example: the number shown when you throw a dice - we
need the probability of every possible number which can
occur
➤ Example: the height of a person selected at random - this
is a continuous random variable and we cannot give a
probability for every value rather we must give a
probability for every range of possible values
https://lds.maiasoft.com/must/emodule.jsp 7/12
5/15/24, 10:42 AM Probability

Example: the time it takes you to travel to work/college -

➤
this is also a continuous random variable
X

NOTE 1: We normally denote a random variable by capital letters

such as X and Y

NOTE 2: Many of the probabilities we study are in fact the probability

that a random variable X=some value x
We write that as: Prob[X=x]. The full specification of Prob[X=x] for all
possible values of x is called the probability distribution of X

All random variables are defined by their probability distribution.

We will study random variables a lot more later.

Note: we are just introducing the term "random variable" at this

stage and don't yet do anything with it.

Examples of Random Variables

BACK TO ADDITION LAW - what about intersecting events

What if A and B not disjoint

https://lds.maiasoft.com/must/emodule.jsp 8/12
5/15/24, 10:42 AM Probability

Suppose that A and B are events which are not disjoint. This means
that A∩B ≠ φ i.e. A∩B is not empty - in other words there are
outcomes which are in both A and B. What can we say about A∪B?

If we look at the Venn diagram we can see that A∪B = A ∪ (B\A)

where we use the notation B\A to mean everything which is in B but
not in A. In other words A∪B is just A ∪ the part of B which is not in
the intersection of A and B. The notation A-B is sometimes used
instead of A\B

Because the events A and (B\A) are disjoint (by definition - because
anything in B\A cannot be in A) we know that:

P(A∪B)=P(A)+P(B\A)

That is just from the addition law. So to finish we need to work out
what is P(B\A). But from the diagram we can see that the event

B=(A∩B) ∪ P(B\A)

Because events (A∩B) and (B-A) are disjoint (remember B-A contains
nothing in A)

https://lds.maiasoft.com/must/emodule.jsp 9/12
5/15/24, 10:42 AM Probability

So from the addition law we see that:

P(B)=P(A∩B) + P(B\A)
X
So now we can say that:

P(B\A) = P(B)-P(A∩B)

∴ from the first equation we have finally:

P(A∪B)=P(A)+P(B)-P(A∩B)
You need to remember this!

Summary so far
➤ Probability is a number between 0 and 1 representing a
person's degree of belief that an event will or will not occur
➤ 0=impossibility; 1=certainty
➤ Probabilities are assigned to events (things which happen)
➤ If A and B are events then A∪B means EITHER A OR B
occurs, or both occur
➤ A∩B means both A AND B occur together
➤ A and B are disjoint if A∩B = ∅ the empty event which means
A and B cannot occur together
➤ If A and B are disjoint events then P(A∪B)=P(A)+P(B) this is
the addition law - it is an axiom of probability
➤ If A and B are NOT disjoint then P(A or B)=P(A)+P(B)-P(A and
B)
➤ The universal set Ω is the set of all possible outcomes and
P(Ω)=1

MULTIPLICATION LAW
➤ It is not necessarily true that P(A∩B)=P(A)P(B)
➤ For us, we will always assume it IS true

https://lds.maiasoft.com/must/emodule.jsp 10/12
5/15/24, 10:42 AM Probability

In advanced probability is it not always true that P(A∩B)=P(A)P(B) but

in the cases we will consider it will always be true. This is the
X
multiplication law:

P(A and B)=P(A)P(B)

Example: what is the probability that you throw a dice twice and it
comes up 6 first time and 5 the second time? Ans = Prob(6) x Prob(5)
= ⅙ x ⅙ = 1/36

The addition and multiplication laws are all you need to

calculate all the probabilities we will come across.

Example:

What is the probability you get a 4 if you throw 2 dice?

The universal set is the set of all possible values from 2 dice.

Each dice can take 6 values, and for each value of dice 1 dice 2 can
take 6 values

There are therefore 36 possible outcomes (think of one dice being

red and the other green). We assume they all have equal
probabilities.

The question is this: out of the 36 possibilities how many of them have
a total of 4?

Or equivalently, how can you get 4 if you throw a red dice and a green
dice?

Answer: if we write the red dice value first followed by the green dice
value (eg if red=3 and green=4 we write 3:4) then the only way we can
get 4 as the sum of the two dice is from: 1:3, 2:2, 3:1 in other words
there are three possibilities for our event out of a total of 36 so we
would say the probability of getting 4 is 3/36 = 1/12 = about 8.33%

https://lds.maiasoft.com/must/emodule.jsp 11/12
5/15/24, 10:42 AM Probability

X
Self-test questions

__________________________________
(c) Maiasoft Educational Software 2017-2024

https://lds.maiasoft.com/must/emodule.jsp 12/12