1. Introduction 2. Example 3. Main Menu 4. Printing 5. Graphs 6. Modules

Markov Analysis

A Markov Chain is described by a transition matrix that gives the probability of going from state to state. For example consider the following.
From/To State 1 State 2 State 3
State 1 .7 .1 .2
State 2 .05 .85 .1
State 3 .05 .05 .9

If we are in state 1 then there is a 70% chance that we are in state 1 at the next stage, a 10% chance that we move to state 2 and a 20% chance that we move to state 3. There are essentially two types of questions that need to be answered for Markov Chains. One is, where will we be after a small number of steps while the other is where will we be after a large number of steps. Often times this depends on the state in which we start.

The data for this example is shown below. The first column ('initial') is indicating that we have an equal chance of starting in any of the three states. This column does not have to contain probabilities (see example 3). The extra data above the data (number of transitions) indicates that we want to look at the results after 3 transitions.


The results contain three different types of answers. The top 3 by 3 table contains the 3 step transition matrix (which is independent of the starting state). The next row gives the probability that we end in state 1 or 2 or 3 which is a function of the initial state probabilities. The last row gives the long run probability or the percentage of times we spend in each state.

The figure below displays the multiplications through 3 transitions (as requested in the extra data box above the data.

Example 2 - A complete analysis

Consider the Markov Chain that is displayed below. The chain consists of three different types of states. State 1 is absorbing, states 3 and 4 together form a closed recurrent class while state 2 is transient.

The first output table is as before describing long run behavior.

In addition the software can classify the states as shown below:

Finally, the software can compute the usual matrices after sorting the data.

1. Introduction 2. Example 3. Main Menu 4. Printing 5. Graphs 6. Modules

[Link to the POM Catalog Page + Link to the QM Catalog Page + Software Download Page
Link to The Decision Science Catalog Page + On Line User Guide + Feedback]

© Prentice-Hall, Inc.
A Simon & Schuster Company
Upper Saddle River, New Jersey 07458

Legal Statement