Chapter 16 – Markov Processes
True / False
1. Markov processes use historical probabilities.
a.
True
b.
False
True
1
Market share analysis
2. All entries in a matrix of transition probabilities sum to 1.
a.
True
b.
False
3. All Markov chain transition matrices have the same number of rows as columns.
a.
True
b.
False
4. A unique matrix of transition probabilities should be developed for each customer.
a.
True
b.
False
5. The probability that the system is in state 2 in the 5th period is π5(2).
a.
True
b.
False
False
1
Market share analysis
6. The fundamental matrix is used to calculate the probability of the process moving into each absorbing state.
a.
True
b.
False
True
1
7. Steady state probabilities are independent of initial state.
Chapter 16 – Markov Processes
a.
True
b.
False
True
1
8. A Markov chain cannot consist of all absorbing states.
a.
True
b.
False
False
1
9. If an absorbing state exists, then the probability that a unit will ultimately move into the absorbing state is given by the
steady state probability.
a.
True
b.
False
False
1
10. All Markov chains have steady-state probabilities.
a.
True
b.
False
False
1
11. All entries in a row of a matrix of transition probabilities sum to 1.
a.
True
b.
False
12. A state i is a transient state if there exists a state j that is reachable from i, but the state i is not reachable from state j.
a.
True
b.
False
13. A state i is an absorbing state if pii = 0.
a.
True
b.
False
Chapter 16 – Markov Processes
False
1
14. When absorbing states are present, each row of the transition matrix corresponding to an absorbing state will have a
single 1 and all other probabilities will be 0.
a.
True
b.
False
True
1
15. For Markov processes having the memoryless property, the prior states of the system must be considered in order to
predict the future behavior of the system.
a.
True
b.
False
False
1
16. The sum of the probabilities in a transition matrix equals the number of rows in the matrix.
a.
True
b.
False
True
1
17. Transition probabilities are conditional probabilities.
a.
True
b.
False
18. A state, i, is an absorbing state if, when i = j, pij = 1.
a.
True
b.
False
True
1
19. If a Markov chain has at least one absorbing state, steady-state probabilities cannot be calculated.
a.
True
b.
False
True
Chapter 16 – Markov Processes
20. State j is an absorbing state if pij = 1.
a.
True
b.
False
False
1
Absorbing states
Multiple Choice
21. In Markov analysis, we are concerned with the probability that the
a.
state is part of a system.
b.
system is in a particular state at a given time.
c.
time has reached a steady state.
d.
transition will occur.
b
1
Introduction
22. For a situation with weekly dining at either an Italian or Mexican restaurant,
a.
the weekly visit is the trial and the restaurant is the state.
b.
the weekly visit is the state and the restaurant is the trial.
c.
the weekly visit is the trend and the restaurant is the transition.
d.
the weekly visit is the transition and the restaurant is the trend.
a
1
Market share analysis
23. A transition probability describes
a.
the probability of a success in repeated, independent trials.
b.
the probability a system in a particular state now will be in a specific state next period.
c.
the probability of reaching an absorbing state.
d.
None of the alternatives is correct.
b
1
Introduction
24. The probability of going from state 1 in period 2 to state 4 in period 3 is
a.
p12
b.
p23
c.
p14
d.
p43
1
Steady-state probabilities
Chapter 16 – Markov Processes
1
Market share analysis
25. The probability that a system is in a particular state after a large number of periods is
a.
independent of the beginning state of the system.
b.
dependent on the beginning state of the system.
c.
equal to one half.
d.
the same for every ending system.
1
Market share analysis
26. At steady state
a.
π1(n+1) > π1(n)
b.
π1 = π2
c.
π1 + π2 ≥ 1
d.
π1(n+1) = π1
d
1
Market share analysis
27. Analysis of a Markov process
a.
describes future behavior of the system.
b.
optimizes the system.
c.
leads to higher order decision making.
d.
All of the alternatives are true.
1
28. If the probability of making a transition from a state is 0, then that state is called a(n)
a.
steady state.
b.
final state.
c.
origin state.
d.
absorbing state.
d
1
Absorbing states
29. Absorbing state probabilities are the same as
a.
steady state probabilities.
b.
transition probabilities.
c.
fundamental probabilities.
Chapter 16 – Markov Processes
d.
None of the alternatives is true.
d
1
Fundamental matrix
30. The probability of reaching an absorbing state is given by the
a.
R matrix.
b.
NR matrix.
c.
Q matrix.
d.
(I − Q)−1 matrix
b
1
Fundamental matrix
Subjective Short Answer
31. Calculate the steady state probabilities for this transition matrix.
1
32. Two airlines offer conveniently scheduled flights to the airport nearest your corporate headquarters. Historically,
flights have been scheduled as reflected in this transition matrix.
Current
Next Flight
Flight
Airline A
Airline B
Airline A
.6
.4
Airline B
.2
.8
a.
If your last flight was on B, what is the probability your next flight will be on A?
b.
If your last flight was on B, what is the probability your second next flight will be on A?
c.
What are the steady state probabilities?
a.
b.
c.
1/3, 2/3
1
State of the system, steady-state probabilities
33. The matrix of transition probabilities below deals with brand loyalty to Bark Bits and Canine Chow dog food.
Current
Purchase
Next Purchase
Bark Bits
Canine Chow
Bark Bits
.75
.25
Canine Chow
.20
.80
Chapter 16 – Markov Processes
a.
What are the steady state probabilities?
b.
What is the probability that a customer will switch brands on the next purchase after a large
number of periods?
b.
P(switching) = (4/9)(1/4) + (5/9)(1/5) = 2/9
1
Market share analysis
34. Bark Bits Company is planning an advertising campaign to raise the brand loyalty of its customers to .80.
a.
The former transition matrix is
What is the new one?
b.
What are the new steady state probabilities?
c.
If each point of market share increases profit by $15000, what is the most you would pay for
the advertising?
c.
The increase in market share is .5 − .444 = .056.
(5.6 points)($15,000/point) = $84,000 value for the campaign.
1
Market share analysis
35. The daily price of a farm commodity is up, down, or unchanged from the day before. Analysts predict that if the last
price was down, there is a .5 probability the next will be down, and a .4 probability the price will be unchanged. If the last
price was unchanged, there is a .35 probability it will be down and a .35 probability it will be up. For prices whose last
movement was up, the probabilities of down, unchanged, and up are .1, .3, and .6.
a.
Construct the matrix of transition probabilities.
b.
Calculate the steady state probabilities.
Chapter 16 – Markov Processes
36. Appointments in a medical office are scheduled every 15 minutes. Throughout the day, appointments will be running
on time or late, depending on the previous appointment only, according to the following matrix of transition probabilities:
Previous
Appointment
Next Appointment
On Time
Late
On Time
.75
.25
Late
.30
.70
a.
The day begins with the first appointment on time. What are the state probabilities for
periods 1, 2, 3 and 4?
b.
What are the steady state probabilities?
a.
1
State of the system, steady-state probabilities
37. A city is served by three cable TV companies: Xcellent Cable, Your Cable, and Zephyr Cable. A survey of 1000 cable
subscribers shows this breakdown of customers from the beginning to the end of August.
Company on
August 1
Company on August 31
Xcellent
Your
Zephyr
Xcellent
300
50
50
Your
10
200
40
Zephyr
40
80
230
a.
Construct the transition matrix.
b.
What was each company’s share of the market at the beginning and the end of the month?
c.
If the current trend continues what will the market shares be?
b.
Market shares Aug 1 X: .40, Y: .25, Z: .35
Market shares Aug 31 X: .35, Y: .33, Z: .32
c.
Steady states: .2156, .48125, .30315
1
Market share analysis
38. A television ratings company surveys 100 viewers on March 1 and April 1 to find what was being watched at 6:00
p.m. — the local NBC affiliate’s local news, the CBS affiliate’s local news, or “Other” which includes all other channels
and not watching TV. The results show
1
Steady-state probabilities
Chapter 16 – Markov Processes
March 1
Choice
Record of Switches During March to
Number
NBC
CBS
Other
NBC
30
—
5
10
CBS
40
15
—
5
Other
30
5
5
—
a.
What are the numbers in each choice for April 1?
b.
What is the transition matrix?
c.
What ratings percentages do you predict for May 1?
1
Market share analysis
39. Accounts receivable have been grouped into the following states:
State 1: Paid
State 2: Bad debt
State 3: 0-30 days old
State 4: 31–60 days old
Sixty percent of all new bills are paid before they are 30 days old. The remainder of these go to state 4. Seventy percent of
all 30 day old bills are paid before they become 60 days old. If not paid, they are permanently classified as bad debts.
a.
Set up the one month Markov transition matrix.
b.
What is the probability that an account in state 3 will be paid?
a.
1
Accounts receivable analysis
Chapter 16 – Markov Processes
40. The medical prognosis for a patient with a certain disease is to recover, to die, to exhibit symptom 1, or to exhibit
symptom 2. The matrix of transition probabilities is
Recover
Die
S1
S2
Recover
1
0
0
0
Die
0
1
0
0
S1
1/4
1/4
1/3
1/6
S2
1/4
1/8
1/8
1/2
a.
What are the absorbing states?
b.
What is the probability that a patient with symptom 2 will recover?
a.
States 1 and 2 (recover and die) are the absorbing states.
The probability that a person with symptom 2 will recover is .633.
1
Absorbing states
41. Rent-To–Keep rents household furnishings by the month. At the end of a rental month a customer can: a) rent the item
for another month, b) buy the item, or c) return the item. The matrix below describes the month-to-month transition
probabilities for 32-inch stereo televisions the shop stocks.
This
Month
Next Month
Rent
Buy
Return
Rent
.72
.10
.18
Buy
0
1
0
Return
0
0
1
What is the probability that a customer who rented a TV this month will eventually buy it?
P(Buy) = .357, P(Return) = .643
1
Absorbing states
42. A recent study done by an economist for the Small Business Administration investigated failures of small business.
Failures were either classified as due to poor financing, poor management, or a poor product. The failure rates differed for
new businesses (under one year old) versus established businesses (over one year old.)
As the result of the economist’s study, the following probabilities were determined. For new businesses the probability of
failure due to financing was .15, due to management .20, and due to product .05. The corresponding probabilities for
established businesses were .10, .06, and .03 respectively.
a.
Determine a five-state Markov Chain transition matrix with states for new, established, and
each of the three failure states. Write it in the form of I, O, R, and Q submatrices.
b.
Determine the probability that a new business will survive during the next three years.
c.
What proportion of new businesses eventually fail due to:
(1) poor financing? (2) poor management? (3) poor product?
a.
Estab.
Chapter 16 – Markov Processes
43. On any particular day an individual can take one of two routes to work. Route A has a 25% chance of being
congested, whereas route B has a 40% chance of being congested.
The probability of the individual taking a particular route depends on his previous day’s experience. If one day he takes
route A and it is not congested, he will take route A again the next day with probability .8. If it is congested, he will take
route B the next day with probability .7.
On the other hand, if on a day he takes route B and it is not congested, he will take route B again the next day with
probability .9. Similarly if route B is congested, he will take route A the next day with probability .6.
a.
Construct the transition matrix for this problem. (HINT: There are 4 states corresponding to
the route taken and the congestion. The transition probabilities are products of the
independent probabilities of congestion and next day choice.)
b.
What is the long-run proportion of time that route A is taken?
b. .36 + .12 = .48
1
Steady-state probabilities
44. Henry, a persistent salesman, calls North’s Hardware Store once a week hoping to speak with the store’s buying agent,
Shirley. If Shirley does not accept Henry’s call this week, the probability she will do the same next week is .35. On the
other hand, if she accepts Henry’s call this week, the probability she will not do so next week is .20.
a.
Construct the transition matrix for this problem.
b.
How many times per year can Henry expect to talk to Shirley?
c.
What is the probability Shirley will accept Henry’s next two calls if she does not accept his
call this week?
d.
What is the probability of Shirley accepting exactly one of Henry’s next two calls if she
accepts his call this week?
a. The transition matrix is:
This Week’s Call
b. .394
c. (1) .47; (2) .39; (3) .14
1
Probabilities at stage n
Chapter 16 – Markov Processes
45. Rent-To–Keep rents household furnishings by the month. At the end of a rental month a customer can: a) rent the item
for another month, b) buy the item, or c) return the item. The matrix below describes the month-to-month transition
probabilities for 32-inch stereo televisions the shop stocks.
Next Month
Rent
Buy
Return
Rent
.72
.10
.18
This
Month
Buy
0
1
0
Return
0
0
1
What is the probability that a customer who rented a TV this month will eventually buy it?
46.
Joe Ferris, a stock trader at the brokerage firm of Smith, Jones, Johnson, and Thomas, Inc. has noticed that price changes
in the shares of Dollar Department Stores at each trade are dependent upon the previous trade’s price change. His
observations can be summarized by the following transition matrix.
Current
Next Price Change
Price Change
+1/8
0
-1/8
+1/8
.7
.2
.1
0
.3
.4
.3
-1/8
.2
.1
.7
a. What is the long-run average change in the value of a share of Dollar Department Stores’ stock per trade?
b. If the shares of Dollar Department Stores are currently traded at $18 and the last trade was at 17 7/8, what is the
probability the shares will sell at 18 in two trades?
Chapter 16 – Markov Processes
47.
Joe Isley, the owner of Big I HiFi, believes that the store’s inventory can be modeled as a Markov process. If items are
either classified as in stock, out of stock, discontinued from stock or put on clearance sale, then the following transition
matrix has been estimated:
Next Month
Chapter 16 – Markov Processes
This Month
In Stock
Out of Stock
Discontinued
Clearance Sale
In Stock
.67
.20
.05
.08
Out of Stock
.48
.42
.10
0
Discontinued
0
0
1
0
Clearance Sale
0
0
0
1
Discontinued
Clearance Sale
Out of Stock
1
0
0
1
.05
.08
.10
0
a. Rewrite the transition matrix for the problem in the form of I, O, R, and Q submatrices.
b. Compute the fundamental matrix for this problem.
c. What is the probability of an item currently in stock being out of stock in two months?
Chapter 16 – Markov Processes
48.
The evening television news broadcast that individuals view on one evening is influenced by which broadcast they viewed
previously. An executive at the C network has determined the following transition probability matrix describing this
phenomenon.
Current Network
News Watched
Next Network News Watched
A
C
N
A
.80
.12
.08
C
.08
.85
.07
N
.08
.09
.83
a. Which network has the most loyal viewers?
b. What are the three networks’ long-run market shares?
c. Suppose each of the three networks earns $1,250 in daily profit from advertising revenue for each 1,000,000 viewers it
has. If on the average 40,000,000 people watch the evening television news, compute the long run average daily profit
each network generates from its evening news broadcast.
49.
Precision Craft, Inc. manufactures ornate pedestal sinks. On any day, the status of a given sink is either: a) somewhere in
the normal manufacturing process, b) being reworked because of a detected flaw, c) finished successfully, or d) scrapped
because a flaw could not be corrected. The transition matrix is:
Tomorrow’s Status
Today’s Status
In-Process
Rework
Finished
Scrapped
In-Process
.30
.15
.50
.05
Rework
.40
.10
.30
.20
Finished
0
0
1
0
Scrapped
0
0
0
1
a. What is the probability of a sink eventually being finished if it is currently in process?
b. What is the probability of a sink eventually being scrapped if it is currently in rework?
c. What is the probability that a sink currently in rework will have a “finished” status either tomorrow or the next day?
(HINT: there are three ways this can happen.)
Chapter 16 – Markov Processes
50.
Southside College has modeled its student loan program as a Markov process. Each year a student with a prior loan
borrows again, defers repayment for a year, makes payments, pays the loan balance in full, or defaults on repayment. The
transition matrix is as follows:
Next Year
This Year
Borrowing
Deferring
Paying
Paid-Off
Default
Borrowing
.60
.30
0
.10
0
Deferring
.15
0
.65
.10
.10
Paying
0
0
.75
.15
.10
Paid-Off
0
0
0
1
0
Defaulted
0
0
0
0
1
a. If currently a student is making payments on his/her loan, what is the probability the loan will be paid in full
eventually?
b. Is the probability of eventually defaulting greater for a student who is currently borrowing more or a student who is
making payments?
c. What is the probability a student who is borrowing this year will repay the loan balance in full in two years or less?
Essay
51. Explain the concept of memorylessness.
52. Where is a fundamental matrix, N, used? How is N computed?
53. Why is a computer necessary for some Markov analyses?
54. What assumptions are necessary for a Markov process to have stationary transition probabilities?
55. Give two examples of how Markov analysis can aid decision making.
Chapter 16 – Markov Processes
56. Discuss three types of information provided by analysis of a Markov process.