CHAPTER 2 REGULAR MARKOV CHAINS
2.1)
2.1a)
5.02.02.01.01
4321State
RepairNW 1,
Action eMaintenanc State
1
2
3
4
1
0.2074
0.4773
0.1458
0.1695
P4 = 2
0.2097
0.4798
0.1305
0.18
3
0.2133
0.489
0.1252
0.1725
4
0.2094
0.4818
0.1407
0.1681
42
2.1b)
2
3
1
0.1
0
0.2
0.5
Z = 2
0.3
0
0
0
3
0.2
0
0.3
0
4
0.1
0
0.3
0.4
1
2
3
4
1
0.081
0
0.164
0.15
Z3 = 2
0.03
0
0.069
0.075
3
0.044
0
0.085
0.08
4
0.078
0
0.154
0.139
1
0.2
10
2
1
0.1282
(4) 3 (1)
22
fZf
= 2
0.0555
3
0.0673
4
0.1204
(4)
42
f
0.1204
2.1c) Make target recurrent state 1 an absorbing state.
1
Vector of MFPTs to state 1= 2
3.33333
3
3.80952
4
4.68254
m
4.68254
2.2)
2.2a)
Nothing Do(mD) 4
Repair of 1Day (NW1) 1
Action eMaintenancState
State 1 2 3 4 5
101000
4 0.2 0 0.5 0.3 0
1
2
3
4
5
1
0.174
0.15
0.403
0.143
0.13
2
0.1775
0.174
0.4096
0.112
0.127
P4 = 3
0.1689
0.177
0.3841
0.105
0.165
4
0.1749
0.188
0.4
0.092
0.145
5
0.1784
0.179
0.4115
0.106
0.126
53
2.2b)
11
1
2
3
4
5
1
0
1
0
0
0
Z = 2
0.1
0
0
0.2
0.5
3
0.3
0
0
0
0
4
0.2
0
0
0.3
0
5
0.1
0
0
0.3
0.4
1
2
3
4
5
1
0.09
0.1
0
0.21
0.2
2
0.072
0.09
0
0.143
0.13
Z3 = 3
0.03
0
0
0.06
0.15
4
0.038
0.06
0
0.067
0.1
5
0.068
0.1
0
0.131
0.114
1
0
(1)
3
f
2
0.2
3
0.7
4
0.5
5
0.2
3
1
0.165
(4) 3 (1)
33
fZf
= 2
0.1155
3
0.06
4
0.0655
5
0.1083
(4)
53
f
0.1083
2.2c) Make target recurrent state 1 an absorbing state.
1
2
4.76984
Vector of MFPTs to state 1= 3
3.33333
4
3.80952
5
4.68254
51
m
4.68254
2.2d)
m
8.97531
12
2.3)
2.3a)
repair ofday secondin g,Not Workin(D2) 2
repair ofday first in g,Not Workin(D1) 1
nDescriptioState
123
3
2
State 0 1 2 3
200
3100 0
Pq
q
qq qq

1
2
3
0
0.9
0.1
0
0
P = 1
0.5
0
0.5
0
2
0.6
0
0
0.4
3
1
0
0
0
0
1
2
3
0
0.849
0.086
0.045
0.02
2.3b) P3 = 1
0.9
0.075
0.025
0
2
0.876
0.094
0.03
0
3
0.86
0.09
0.05
0
(3)
32
2.3c)
0
1
2
3
0
0.9
0.1
0
0
Z = 1
0.5
0
0.5
0
2
0.6
0
0
0
3
1
0
0
0
0
1
2
0
0.86
0.09
0.05
Z2 = 1
0.75
0.05
0
2
0.54
0.06
0
3
0.9
0.1
0
3
0
0
0
14
11
11
Beginning Order,
Inventory, State 0 1 2 3
03210
033
State 0 1 2 3
0 0.014 0.076 0.303 0.607
nn
nn
nn nn
Xc
Xc
P(d ) P(d ) P(d ) P(d )
P


t
393 0.607 0 0
2 0.090 0.303 0.607 0
3 0.014 0.076 0.303 0.607
2.4b)
State = 0
1
2
3
p(0) = 1
0
0
0
p(1) = p(0)P = 0.3
0.7
0
0
p(2) = p(1)P = 0.48998
0.21
0.30002
0
p(3) = p(2)P = 0.46701
0.342986
0.090006
0.1
2.4c) If no orders are placed tomorrow, then the inventory level at the beginning of
1212223
nn

2.4d) Make the target recurrent state 0 an absorbing state.
0
1
2.544529
Vector of MFPTs to state 0 = 2
4.506342
3
6.510956
20
m
4.5063
2.4e) Mean recurrence time
00
m
4.2155
15
2.5)
43210 years,in ICan of Age
ii
1
life of 1year during failing
3
3
3
2
32
2
1
21
1
0
10
0
S
r
S
r
S
r
S
r
i
State 0 1 2 3
37
000
10 10
31000
2.5b)
State = 0
1
2
3
p(0) = 1
0
0
0
p(1) = p(0)P = 0.3
0.7
0
0
p(2) = p(1)P = 0.48998
0.21
0.30002
0
p(3) = p(2)P = 0.46701
0.342986
0.090006
0.1
2.5d) Make the target recurrent state 0 an absorbing state.
º
ª
»
»
»
º
«
«
«
ª
01
0001
04286.005714.0
0001
3
1
0
16
º
ª
º
ª
1429.04286.01
1
04286.01
1
1
Hence the vector of mean times until a component of age i years is replaced is
00
30
2.10002
1
m
m
ªº
ªº
«»
«»
¬¼
¬¼
2.6)
State 0 1 2 3
0 0.30 0.10 0.20 0.40
3 0 0 0.30 0.70
2.6b)
State = 0
1
3
p(0) = 0
1
0
p(1) = p(0)P = 0.3
0.1
0.4
p(2) = p(1)P = 0.12
0.1
0.56
p(3) = p(2)P = 0.066
0.088
0.612
2.6c)
S
[0. 036 0. 084 0. 24 0. 64]
2.6d) Make the target recurrent state 0 an absorbing state.
0
17
0
27.77778
Vector of MFPTs to state 0 = 1
37.77778
2
41.11111
20
m
41.11111
2.7)
State01234
0 0.76 0.24 0 0 0
3 0.34 0.16 0.18 0.08 0.24
4 0.34 0.16 0.18 0.08 0.24
2.7b)
State = 0
1
2
3
4
p(0) = 0
0
1
0
0
p(1) = p(0)P = 0.5
0.18
0.08
0.24
0
p(2) = p(1)P = 0.624
0.1872
0.0928
0.038
0.058
p(3) = p(2)P = 0.68058
0.1968
0.06963
0.03
0.023
2.7c)
2.7d) Make the target recurrent state 1 an absorbing state.
1
0
4.16667
Vector of MFPTs to state 1 = 2
4.59592
3
4.77049
4
4.77049
41
m
4.77049
2.8)
2.8a) Problem 1.7 has the following transition probability matrix.
18
State 0 1 2 3
01 0 0
pp
2.8b)
)1)((
rpr
S
, where
rp
s)1(
)1(
3
()(1)(1)
rp r ps

rp
)1(
2.8d)
State 0 1 2 3
1 0.2736 0.5728 0.1536 0
P
2.8e)
State = 0
1
2
3
p(0) = 0
0
1
0
p(1) = p(0)P = 0
0.2736
0.5728
0.1536
p(2) = p(1)P = 0.074857
0.313436
0.42542
0.186286
p(3) = p(2)P = 0.142647
0.313897
0.35889
0.184568
2.8g) Make the target recurrent state 3 an absorbing state.
3
0
49.42627
Vector of MFPTs to state 3 = 1
45.2596
2
31.32731
23
m
31.32731
2.9)
19
2.9a)
State01234
1 0.14 0.10 0.20 0.18 0.38
2 0 0.14 0.10 0.20 0.56
P
2.9b)
State = 0
1
2
3
4
p(0) = 0
0
0
1
0
p(1) = p(0)P = 0
0
0.14
0.1
0.76
p(2) = p(1)P = 0
0.0196
0.028
0.1444
0.808
p(3) = p(2)P = 0.002744
0.00588
0.02694
0.13669
0.82775
2.9c)
S
[0. 00066 0. 003584 0. 022098 0. 136092 0. 837566]
2.9d) Make the target
recurrent state 4 an absorbing
state.
0
2.75166
Vector of MFPTs to state 4 = 1
2.20784
2
1.76239
3
1.38526
14
m
2.20784
2.10)
20
2.10a)
State 12345
3 (W,I) 0 0.8 0.2 0 0
P
2.10b)
State = 1
2
3
4
5
p(0) = 1
0
0
0
0
p(1) = p(0)P = 0.4
0
0.6
0
0
p(2) = p(1)P = 0.16
0.48
0.36
0
0
p(3) = p(2)P = 0.1984
0.3456
0.3696
0.0864
0
2.10c)
S
[0.17728 0.37989 0.34514 0.07275 0.02494]
2.10d) Make the target recurrent state 3 an absorbing state.
3
1
1.66667
Vector of MFPTs to state 3 = 2
2.37175
4
3.44708
5
3.80032
53
m
3.80032