47
v1(2) = 660.8
4.6
State 1 2 5 3 4
2, On course to LEO 0 1 0 0 0 0
I
ªº
State Revenue (in millions of dollars)
14
21
A
q
ªº
«»
3
4
4.6b) U = (I – Q)
-1
= 3
1.13475
0.070922
4
0.42553
1.276596
1
2
5
F = UD = (I – Q)
-1
D = 3
0.624113
0.368794
0.00709
4
0.234043
0.638298
0.12766
UqT = 3
4
The expected total revenue (in millions of dollars) earned by launching a rocket, given that
the rocket starts in state 3, is
$2 4 6 ( )
ff fUq  
4.6c) If, upon launch, the rocket is observed to start in state 3, the expected total revenue
earned by launching the rocket into LEO, state 2, is found by calculating
2
()
T
Uq
.
23 4
210 0
0.4(1) 0.3(0.369) 0.2(0.638)
100
20
I
½
°
ªº
°
«»
°
«»
°
°
°
°
ªº
°
ªº
«»
°
°
¿
3
4
49
The investor’s two alternative policies can be represented as multichain MCRs
with the following transition probability matrices and associated reward vectors.
Policy 1
º
ª
I
0001020
000010
15105200State
º
ª
A
q
5020
200
RewardState
Policy 2
º
ª
I
14.038.008.022.018.020
10.030.022.014.024.010
0001015
20100155State
º
ª
A
q
d
d
20
10
1515
RewardState
The expected total rewards produced by both policies will be equated. The equation
relating the expected total rewards for both policies will be solved to find the break even
value of the dividend
d
. The fundamental matrix for policy 1 is
8721.15426.10342.115
8098.04164.1086.25
15105
20.33.72
º
ª
1
000.
The vector,
T
Uq
, is computed below.
5 2.086 1.4164 .8098 5 4.3122
15 1.0342 1.5426 1.8721 15 4.4489
dd
dd
ªºªºªº
«»«»«»
«»«»«»
¬¼¬¼¬¼
0
50
The component,
10
( ) 3.7968
T
Uq d
, represents the expected total dividend earned before
the stock is sold, given that she paid $10 for the stock. Reason:
10
()U
= sum of entries in
When the stock is sold, the chain will be absorbed in state $0 or $20. The equation
for calculating the matrix of absorption probabilities for an absorbing multichain, given
that the process starts in a transient state, is
1
20,50,5
5
200
ff
3493.06507.015
06.010.0
8721.15426.10342.115
If the investor buys the stock for $10, then under policy 1 she will eventually sell it for
0,10
20,10 f
when the stock is sold, she will receive an expected selling price of
20,100,1010
A
The investor’s expected total reward, given that she bought the stock for $10, is the
expected dividends received before selling plus the expected selling price, or
10 10
( ) ( ) 3.7968 6.704
TA
Uq Fq d
The fundamental matrix for policy 2 is
51
3683.19181.04719.020
5138.0987.09064.10
20100
18.26.66
º
ª
1
000.
The vector,
T
Uq
, is computed below.\
»
º
«
ª
»
º
«
ª
»
º
«
ª
d
d
4072.3
0
5138.987.09064.1
00
1
The component,
dUq
T
8934.2)(
10
, represents the expected total dividend earned before
When the stock is sold, the chain will be absorbed in state $5 or $15. The equation
for calculating the matrix of absorption probabilities for an absorbing multichain, given
that the process starts in a transient state, is
1
15,05,0
0
155
ff
3683.19181.04719.020
4768.05283.020
22.018.0
If the investor buys the stock for $10, then under policy 2 she will eventually sell it for
5,10
15,10
when the stock is sold, she will receive an expected selling price of
15,105,1010
A
52
The investor’s expected total reward, given that she bought the stock for $10, is the
expected dividends received before selling plus the expected selling price, or
Equating the expected total rewards produced by both policies,
If
62.2$!d
, the investor should follow policy 1 by selling the stock as soon as the price
If
62.2$d
, the investor should follow policy 2 by selling as soon as the price becomes
4.8
4.8a)
)P(w)P(w)P(w
wP)P(w)P(w)P(w
wP)P(w)P(w
)P(w)P(w
P
nnn
nnnn
nnn
nn
21004
)3(2103
0)3(212
00321
4321State
t
d
d
)3(21004
)3(2103
0)3(21)0(2
0032)1()0(1
4321State
nnnn
nnnn
nnnn
nnnn
wP)P(w)P(w)P(w
wP)P(w)P(w)P(w
wP)P(w)P(wwP
)P(w)P(wwPwP
P
53
1.04.03.02.004
1.04.03.02.03
01.04.03.02.02
001.04.03.02.01
4321State
P
In the matrix below,
kR
denotes the release of
k
units of water, and
kS
denotes the
spilling away of
k
units of surplus water.
1 and 22204
22223
0222or 12
0022or 1or 01
4321State
SRRR
RRRR
RRRR
RRRR
P
The expected rewards in every state are calculated below.
2.8$
9$)1.04.03.02.0(9$)]3()2()1()0([9$
3 nnnn WPWPWPWPq
4 nnnnn WPWPWPWPWPq
The MCR for the volume of water in the dam under this release policy is
State 1 2 3 4 State Reward
1 0.9 0.1 0 0 1 6
,
2 0.5 0.4 0.1 0 2 8.2
3 0.2 0.3 0.4 0.1 3 9
4 0 0.2 0.3 0.5 4 8.7
Pq
4.8b)
v(2) =
v(1) =
v(0) =
State
v(3) = 0
q+ P v(3)
q +Pv(2)
q+ Pv(1)
0
0
6
12.22
18.536
1
0
8.2
15.38
22.175
2
0
9
17.13
24.649
3
0
8.7
17.39
25.61
4.8c)
S
[0.8146 0.1517 0.0281 0.0056]
4.8d)
0
1
2
3
0
0.81
0.09
0
0
αP = 1
0.45
0.36
0.09
0
2
0.18
0.27
0.36
0.09
3
0
0.18
0.27
0.45
0
1
2
3
0
8.489559
1.28464
0.194048
0.03175
1
()IP
D
1
6.811292
2.71202
0.409658
0.06703
2
5.988057
1.75114
1.942879
0.31793
3
5.168741
1.74722
1.087847
1.99619
0
63.49408
2
70.53954
3
72.49714
4.9d)
0
1
2
0
0.27
0.18
0.45
αP = 1
0.36
0.45
0.09
2
0.27
0.54
0.09
0
1
2
0
4.026194
3.624376
2.34943
3.135246
4.836066
2
3.055061
3.945118
3
0
0
0
1
0
2
0
0
2
56
2
15.58442
2.33766
1.818182
0
7767.97
UqT = 1
6434.63
2
3615.58
(UqT)0 =
7767.97
4.10d) The MCR for an engineer who follows the recommended annual course loads is
State 3 0 1 2 State Revenue (in thousands of dollars)
31000 3 80
2 0.45 0.55 0 0 2 120 15(1) 105
q
0
35877.78
4.10e) UqT = 1
31977.78
2
19837.78
(UqT)0 = 35877.78
4.11)
4.11a)
State 0 1 2 State Expected cost per plan for employees who switch
0 P.P.O. 0.85 0.12 0.03 0 $870 200[(0.12)($30) (0.03)($25)]
1 H.M.O. 0.15 0.75 0.10 1 $1620 300[(0.15)($20) (0.10)($24)]
2 H.D.P. 0.04 0.16 0.80 2 $54
4 100[(0.04)($32) (0.16)($26)]
4.11b)
v(2) =
v(1) =
v(0) =
State
v(3) = 0
q+ P v(3)
q +Pv(2)
q+ Pv(1)
0
0
870
1820.22
2817.77
1
0
1620
3019.9
4285.28
2
0
544
1273.2
2118.55
4.11c)
S
[0.41312 0.349939 0.236938]
4.11d)
0
1
2
0
0.765
0.108
0.027
αP = 1
0.135
0.675
0.09
2
0.036
0.144
0.72
0
1
2
0
31.68091
21.60684
10
0
68005.47
65817.44
2
44535.38