Probability
Some slides developed by Mine Çetinkaya-Rundel of OpenIntro
The slides may be copied, edited, and/or shared via the CC BY-SA license
Chapter 2: Probability
●A random process is a
situation in which we know
what outcomes could
happen, but we don’t know
which particular outcome will
happen.
●Examples: coin tosses, die
rolls, iTunes shuffle, whether
the stock market goes up or
down tomorrow, etc.
●It can be helpful to model a
process as random even if it
is not truly random.
Probability
There are several possible interpretations of probability but they (almost)
completely agree on the mathematical rules probability must follow.
●P(A) = Probability of event A
● 0 ≤ P(A) ≤ 1
Frequentist interpretation:
●The probability of an outcome is the proportion of times the outcome
would occur if we observed the random process an infinite number of
times.
Bayesian interpretation:
●A Bayesian interprets probability as a subjective degree of belief: For
the same event, two separate people could have different viewpoints
and so assign different probabilities.
●Largely popularized by revolutionary advance in computational
technology and methods during the last twenty years.
Random Phenomena
Outcome is
Uncertain
In the short-run
outcome are highly
random
In the long-run,
outcomes are very
predictable
Probability quantifies
long-run randomness
Law of large numbers
Law of large numbers states that as more observations are collected, the
proportion of occurrences with a particular outcome, p
̂n, converges to the
probability of that outcome, p. As # of trials
increase, the
proportion of
an outcome
approaches a
particular
number
(probability):
In the long run,
1/6 of die
Tosses is 6
Figure The Cumulative Proportion of Times that a 6 Occurs
for a Simulation of 10,000 Rolls of a Fair Die
Law of large numbers (cont.)
When tossing a fair coin, if heads comes up on each of the first 10 tosses,
what do you think the chance is that another head will come up on the next
toss? 0.5, less than 0.5, or more than 0.5?
●The probability is still 0.5, or there is still a 50% chance that another
head will come up on the next toss.
P(H on 11th toss) = P(T on 11th toss) = 0.5
● The coin is not “due” for a tail.
●The common misunderstanding of the LLN is that random processes are
supposed to compensate for whatever happened in the past; this is just
not true and is also called gambler’s fallacy (or law of averages).
Disjoint and non-disjoint outcomes
Disjoint (mutually exclusive) outcomes: Cannot happen at the same time.
●The outcome of a single coin toss cannot be a head and
a tail.
●A student both cannot fail and pass a class.
●A single card drawn from a deck cannot be an ace and
a queen.
●Note: For disjoint events P(A and B) = 0, so the above formula simplifies to P(A
or B) = P(A) + P(B).
Probability of Union of Two Events
Addition Rule for the union of any two events,
For the union of two events P(A or B) = P(A) + P(B) – P(A and B)
If disjoint, P(A and B) = 0, so
P(A or B) = P(A) + P(B)
Figure The Probability of the Union, Outcomes in A or B or Both
Non-disjoint outcomes: Can happen at the same time.
●A student can get an A in Stats and A in Econ in the same semester.
Intersection & Union of A and B
Intersection is all outcomes in both A and B
Union is all outcomes in either A or B or both
Figure The Intersection and the Union of Two Events. Intersection means A
occurs and B occurs, denoted “A and B.”
Probability of an Event
Probability of event A, P(A), is sum
of outcome probabilities in event
When all outcomes
are equally likely:
Probability distributions
●Probability distributions
A probability distribution lists all possible events and the probabilities with which
they occur.
●The probability distribution for the gender of one kid:
●Rules for probability distributions:
1. The events listed must be disjoint
2. Each probability must be between 0 and 1
3. The probabilities must total 1
Probability distributions
Figure Proposed distributions of US household incomes
Sample space and complements
Sample space is the collection of all possible outcomes of a trial.
●A couple has one kid, what is the sample space for the gender of this
kid? S = {M, F}
●A couple has two kids, what is the sample space for the gender of
these kids?
Complementary events are two mutually exclusive events whose
probabilities that add up to 1.
●A couple has one kid. If we know that the kid is not a boy, what is
gender of this kid? { M, F } Boy and girl are complementary
outcomes.
●A couple has two kids, if we know that they are not both girls, what
are the possible gender combinations for these kids?
S = { MM, FF, FM, MF }
S = {MM, FF, FM, MF}
Complement of Event A
All outcomes in sample space not in A
