2
CHAPTER 1 MARKOV CHAIN STRUCTURE AND MODELS
1.1)
4.03.02.01.04
03.05.02.03
007.03.02
5.02.02.01.01
4321State
Nothing Do WP4,
Nothing DomD 3,
Nothing DoMD 2,
RepairUnder NW 1,
Action eMaintenanc State
P
1.2)
Nothing Do(mD) 4
Repair of 1Day (NW1) 1
Action eMaintenancState
State 1 2 3 4 5
101000
4 0.2 0 0.5 0.3 0
1.3)
State 0 1 2 3 4
01 0 0 0
4100 0 0
pp
1.4)
To display
P
, a large transition matrix with 14 rows and 14 columns,
P
is bisected
L
LF
3
State1234567
101 000 0
2001 0 00
70000000
80000000
90000000
140000000
F
pp
pp
P
State 8 9 10 11 12 13 14
10000000
20000000
6000 000
71 0 0 0 00
801 000 0
9001 000
130000010
140000001
L
p
Ppp
pp
pp
4
1.5)
0 if ),3min( 11 nnn XAX
012 34
01 234
01234
20
300
pp ppp
ppppp
1.6)
)1,min(4
11
nnn
VXX
00
State 0 1 2 3 4
01 000
pp
1.7)
When the store has no customers,
nn
1
When the store has 1 or 2 customers,
)1()|1(
1
rpiXiXP
nn
1
nn
When the store has 3 customers,
5
rXXP
nn
1)3|3(
1
nn
1
State 0 1 2 3
01 0 0
1 (1 ) (1 )(1 ) (1 ) 0
pp
Ppr pr p r p r
1.8)
11
11
Beginning Order,
Inventory, State 0 1 2 3
03210
033
nn
nn
nn nn
Xc
Xc
P(d ) P(d ) P(d ) P(d )
t
1.9)
life of 1year during failing
3210 years,in ICan of Age
3
43
3
2
32
2
1
21
1
0
10
0
S
r
S
r
S
r
S
r
i
i
00
State 0 1 2 3
0100
rr
1.10)
11 12
State 1 2 3 4
1
pp bkb
State P E
PP
22211211
6
kppppg
PE
24231413
1.10b) From the calculations for part (a), the transition probabilities,
ij
g
, for the lumped
210
)1()]12()11([
00101
¸
·
¨
§
XPXXPXXP
)0(
2
)0(
1
pp
1.10d) Clearly,
PP
g
depends on the choice of the initial state probability vector. Note that
22211211
)0(
2
)0(
1
)0(
2
)0(
1
pp
pp
PP
which is independent of the choice of the initial state probability vector.
)0(
2
)0(
1
)0(
22423
)0(
11413
)()(
pp
pppppp
g
PE
)0(
44241
)0(
33231
)()(
pppppp
g
)0(
4
)0(
3
pp
EE
7
1.11)
1 and 0 if 0
11
d
nnn
WXX
01234 5
0123 45
State 0 1 2 3 4
0
1
wwwww w
wwww ww
1.12) States 1, 2, and 3 are transient. States 4 and 5 are absorbing.
State 4` 5 1 2 3
4Discharged 1 0 0 0 0
1.13) States 1, 2, and 3 are transient. States 4 and 5 are recurrent. They form a recurrent
closed class of states. State 6 is absorbing.
000001Discharged6
321546`State
»
«
8
1.14)
State 1 2 3 4 5
1(I,I) 1 0 0 0
4 (W,W) 0 (1 ) (1 )(1 ) (1 )
pp
rq r q r q r q