For a Markov process with absorbing states, we define Π(j) = state vector at stage j N = fundamental matrix
I = identity matrix
Q = matrix of transition probabilities between non-absorbing states
R = matrix of transition probabilities between non-absorbing states and absorbing states
The limiting state probabilities equal:

Question

Quizplus · Accepted Answer

The formula for the limiting state vector is Π(infinity)= Π(1) N (I-Q)^(-1) R. Since the absorbing states have probability 1 of being reached eventually, the probability of ending up in any other state approaches zero as the number of stages goes to infinity. Therefore, the contribution of the matrix R to the limiting state vector is Π(infinity) = Π(1) N R.

For a Markov Process with Absorbing States, We Define Π(J)