College Mathematics: Learning Worksheets Chapter 15
Name ________________________________ Date ______________ Class ____________
Goal: To solve problems using Markov chains
1. A Markov process has two states, A and B. The probability of going from state A to state
B in one trial is 0.25, and the probability of going from state B to state A in one trial is 0.35.
a) Draw the transition diagram.
Find the transition matrix.
AB
b) If
0.4 .6S, find 12
, and .SS
10
SSP
21
SSP
Section 15-1 Properties Of Markov Chains
Definition: A transition matrix is a constant square matrix P of order n such that the entry in
the ith row and jth column indicates the probability of the system moving from the ith state to
the jth state on the next observation or trial.
Theorem: If P is the transition matrix and 0
Sis an initial state matrix for a Markov chain,
k
439
3. Two popular brands of macaroni and cheese, Smack A Lot and Good Eats are sold at a
grocery store. The buying habits of the customers are followed for several weeks during an
advertising campaign. It is found that 20% of those using Smack A Lot will switch to Good
Eats and that 40% of those who started by buying Good Eats will switch to Smack A Lot.
a) Draw a transition diagram.
b) Find the transition matrix.
0.8 0.2
0.4 0.6
S
PG
c) If 50% of the customers used Smack A Lot at the start of the advertising
campaign, what percentage will be using Smack A Lot after 1 week? after 2 weeks?
4. The buying patterns of customers who buy two brands of running shoes, Flying Feet and
Great Grips, are noted each year. It is found that 80% customers who buy Flying Feet one
year will buy Great Grips the next year and 20% will buy Flying Feet again the next year.
Also, it is found that 30% of the customers who buy Great Grips will buy Flying Feet the
next year and 70% will buy Great Grips again.
a) Draw a transition diagram.
b) Find the transition matrix.
FG
c) If 60% of the customers bought Flying Feet at the start and 40% bought Great
Grips, what percentage will buy Flying Feet after the first year? Great Grips? After 2
years?
00.6 0.4S
10
SSP
21
SSP
441
5. If a voter votes Republican in one election, the probability that the voter will vote
Democratic in the next election is .15 and the probability the voter will vote for an
independent candidate is .05. If a voter votes Democratic in one election, the probability that
the voter will vote Republican in the next election is .05 and the probability that the voter
will vote for an independent candidate is .1. If a voter votes for an independent candidate in
one election, the probability that the voter will vote Republican in the next election is .3 and
the probability that the voter will vote Democratic in the next election is .5. Assume that
these are the only three choices available to the voter.
a) Draw the transition diagram.
0.15
R D 0.85
0.2
b) Find the transition matrix.
0.8 0.15 0.05
R
DI
R
College Mathematics: Learning Worksheets Chapter 15
442
c) If 42% of the electorate votes Republican one year and 48% vote Democratic, find
the percentage of voters who vote Republican the next year. The percentage who vote
Democratic the next year. The percentage of voters who vote for an independent
candidate. After 2 years?
00.42 0.48 0.1S
10
0.39 0.521 0.089
SSP
21
0.36475 0.54585 0.0894
SSP
After one year, 39% will vote Republican, 52.1% will vote Democratic, and 8.9%
will vote Independent.
College Mathematics: Learning Worksheets Chapter 15
Name ________________________________ Date ______________ Class ____________
Goal: To solve problems using Markov chains
1. A Markov process has two states, A and B. The probability of going from state A to state
B in one trial is 0.35, and the probability of going from state B to state A in one trial is 0.25.
Find the stationary matrix that describes the long-run behavior of this process.
[] []
12 12
0.65 0.35
0.25 0.75
ss ss
⎡⎤
=
⎢⎥
⎣⎦
121 12
0.65 0.25 0.35 0.25 0
sss ss
+=−+=
⎧⎧
Section 15-2 Regular Markov Chains
Definition: Stationary Matrix for a Markov Chain
The state matrix 12
[]
n
Sss sis a stationary matrix for a Markov chain with
i
n
445
4. The buying patterns of customers who buy two brands of running shoes, Flying Feet and
Great Grips, are noted each year. It is found that 90% customers who buy Flying Feet one
year will buy Great Grips the next year and 10% will buy Flying Feet again the next year.
Also, it is found that 20% of the customers who buy Great Grips will buy Flying Feet the
next year and 80% will buy Great Grips again. In the long run what percentage of the
customers will be using Flying Feet? Great Grips?
[] []
12 12
0.1 0.9
0.2 0.8
ss ss
⎡⎤
=
⎢⎥
⎣⎦
121 12
0.1 0.2 0.9 0.2 0
sss ss
+= −+=
⎧⎧
22
2
2
0.9 0.9 0.2 0
1.1 0.9
0.8182
ss
s
s
−+ + =
=
≈
1
1
1 0.8182
0.1818
s
s
=−
≈
446
5. If a voter votes Republican in one election, the probability that the voter will vote
Democratic in the next election is 0.15 and the probability the voter will vote for an
independent candidate is 0.05. If a voter votes Democratic in one election, the probability
that the voter will vote Republican in the next election is 0.05 and the probability that the
voter will vote for an independent candidate is 0.1. If a voter votes for an independent
candidate in one election, the probability that the voter will vote Republican in the next
election is 0.3 and the probability that the voter will vote Democratic in the next election is
0.5. Assume that these are the only three choices available to the voter. If this trend holds
up, what percentage of the voters will vote Republican in the long run? Democratic?
Independent?
123 123
0.8 0.15 0.05
0.05 0.85 0.1
0.3 0.5 0.2
ss s ss s
1231 123
1232 123
0.8 0.05 0.3 0.2 0.05 0.3 0
0.15 0.85 0.5 0.15 0.15 0.5 0
ssss sss
ssss sss
Using Gauss-Jordan elimination to solve this system of four equations with three
College Mathematics: Learning Worksheets Chapter 15
Name ________________________________ Date ______________ Class ____________
Goal: To solve application problems using Markov chains
1. A computer game has two levels. Level one is called Flying Low and level two is called
Flying High. To win the game the player must successfully complete Flying Low before
getting to Flying High. On their first attempt at playing the game, 30% of players are able to
successfully navigate Flying Low and move on to Flying High, 50% make an error and are
eliminated from the game, and the rest continue to play at the Flying Low level. After
making it to Flying High, 10% of the players successfully navigate Flying High and win the
game, 15% make a fatal error and are eliminated from the game, and the rest continue to play
at the Flying High level.
a) Draw a transition diagram.
0.3
FLYING FLYING
0.5 0.15 0.1
1
Section 15-3 Absorbing Markov Chains
The limiting matrix: 0
0
I
PFR
where
1
FIQ
(F is called the fundamental matrix for P).
College Mathematics: Learning Worksheets Chapter 15
b) Find the transition matrix P.
0.2 0.3 0.5 0
FL FH L W
FL
c) Write the transition matrix P in a standard form.
1000
L W FL FH
L
d) Subdivide matrix P and then find matrix R and matrix Q.
1000
L W FL FH
L
e) Find matrix F. (Remember
1
FIQ
).
00.8 0 4
College Mathematics: Learning Worksheets Chapter 15
f) Find FR.
0 4 0.15 0.1 0.6 0.4
g) Write the limiting matrix P.
1000
0.85 0.15 0 0
0.6 0.4 0 0
h) In the long run, what percentage of players will lose before getting to Flying
High?
i) In the long run, what percentage of players will lose after getting to Flying High?
j) In the long run, what percentage of players that make it to Flying High will win the
game?
k) What is the average number of trials that a player spends in Flying Low?
l) What is the average number of trials that a player spends in Flying High?
College Mathematics: Learning Worksheets Chapter 15
450