EXERCISE 15-3 15-21
36. (A) S0P = [0 0 1]
100
010
.6 .4 0
= [.6 .4 0]
100
.6 .4 0
38. (A) S0P = [0 0 1]
100
100
100
= [1 0 0]
100
100
40. (A) S0P = [0 0 0 1]
1000
0100
.55 .45 0 0
.65 .35 0 0
= [.65 .35 0 0]
1000
0100
.55 .45 0 0
.65 .35 0 0
42. True. See the definition on page 470.
46. False. See the discussion and on page 474, after Matched Problem 3.
15-22 CHAPTER 15: MARKOV CHAINS
50. By Theorem 2, P has a limiting matrix:
1000
0100
1000
0100
A B C D
1 000
0100
A
B
52. By Theorem 2, P has a limiting matrix:
4 ≈
10000
01000
P8 ≈
01000
.5519 .3572 .0051 .0479 .0378
.0510 .8505 .0055 .0517 .0412
,
16 ≈
1 0000
0 1000
32 ≈
1 0000
0 1000
A B C D E
1 0000
A
15-24 CHAPTER 15: MARKOV CHAINS
56. P4 ≈
1000
.6528 .1296 0 .2176
.6459 .2013 .0625 .0903
, P8 ≈
1000
.7374 .0168 0 .2458
.8177 .0387 .0039 .1397
1000
.75 0 0 .25
.85 0 0 .15
0001
58. Let S = [x 1 – x 0 0], 0 ≤ x ≤ 1. Then
SP = [x 1 – x 0 0]
1000
0100
= [x 1 – x 0 0]
60. (A) Tk+1 = I + Q + Q2 + … + Qk+1
62. A transition matrix for this problem is:
F B M T
F=Fully qualified mechanic
1000
F
0.2
.3 .5
EXERCISE 15-3 15-25
The limiting matrix for M has the form:
I
.4 0
-1
2
5
0
-1
2
2
0.2 .45.55
Therefore, F T B M
1000
0100
F
T
64. A transition matrix in standard form for this problem is:
A B C N
1000
0100
A
B
0010
.16 .56 .28 0
C
N
15-26 CHAPTER 15: MARKOV CHAINS
Copyright © 2019 Pearson Education, Inc.
(A) In the long run, 16% of the employees will elect to join plan A;
56% plan B; 28% plan C.
(B) On the average, it takes 4 years for an employee to decide to join a plan.
66. Let I denote ICU, C denote CCW, D denote “died”, and R denote “released”. A transition matrix in standard
form for this problem is:
D R I C
P =
1000
0100
.02 0 .38 .6
.01 .19 .05 .75
D
R
I
C
For this matrix, we have
R = .02 0
.01 .19
and Q = .38 .6
.05 .75
The limiting matrix for P has the form:
P = 0
0
I
FR
-1
.4 4.96
.01 .19
.0576 .9424
and D R I C
P =
1000
0100
D
R
68. A transition matrix in standard form for this problem is:
L R F B
P =
1000
0100
11 1
0
44 2
11
00
22
L
R
F
B
CHAPTER 15 REVIEW 15-27
For this matrix we have:
R =
11
44
1
20
and Q =
1
2
1
2
0
0
The limiting matrix for P has the form:
P = 0
0
I
FR
1
2
0
10
-1
1
2
1
-1
42
33
24
33
1
20
51
66
Now, L R F B
1000
L
6.
(B) The average number of exits a rat placed in room B will choose
33
CHAPTER 15 REVIEW
1. S1 = S0P = [.3 .7] .6 .4
= [.32 .68]
A
B
A
2. A is an absorbing state; the chain is absorbing since it is possible to go from state B to state A.
3. There are no absorbing states since there are no 1’s on the main diagonal. P is regular since
15-28 CHAPTER 15: MARKOV CHAINS
4. P = 01
has no absorbing states. Since Pk, k = 1, 2, 3, …, alternates between 01
and 10
,
5. States B and C are absorbing. The chain is absorbing; it is possible to go from the nonabsorbing state A to
A B C
7. P =
010
.1 0 .9
A
B
There are no absorbing states.
010
.1 0 .9
(15-1, 15-2, 15-3)
A B C
8. P =
010
.1 .2 .7
A
B
A B C
9. P =
001
.1 .2 .7
A
B
.02 .74 .24
A B C D
10. P =
.3 .2 0 .5
0100
00.2.8
A
B
C
15-30 CHAPTER 15: MARKOV CHAINS
From the first and third equations, we have s1 = 5
6s2 and s3 = 1
4s2.
Substituting these values into the fourth equation, we get
5
6s2 + s2 + 1
4s2 = 1 or 25
12 s2 = 1 and s2 = .48
.4 .48 .12
C
A B C
15. For P =
100
010
.3 .1 .6
A
B
C
we have R = [.3 .1] and Q = [.6]. The limiting matrix P has the form
100
2
and FR = 5
[.3 .1] = [.75 .25].
2
from C to either A or B. (15-3)
A B C D
16. For P =
1000
0100
.1 .5 .2 .2
.1 .1 .4 .4
A
B
C
D
.1 .1
.4 .4
CHAPTER 15 REVIEW 15-31
The limiting matrix P has the form
P = 0
0
I
FR
1
1 0 .2 .2
0 1 .4 .4
1
.8 .2
.4 .6
1
41
55
3
2
55
We use row operations to find the inverse:
1
410
5
1
10
44
5
1
10
44
5
1
10
4
5
4
~
31
10 22
01 12
31
22
12
12
12
.1 .5
.1 .1
.3 .7
A B C D
and P =
1000
0100
A
B
A B A B
A
A
(15–3)
A B C
18. P =
.4 .6 0
.5 .3 .2
A
B
, P4 ≈
.4066 .4722 .1212
.3935 .4895 .117
,
.4001 .4798 .1201
.4 .48 .12
C