Question 2. [11 Marks]. Consider a Markov process {X(t)}t≥0 with state space S ={0, 1, 2, 3} and Q-matrix, or generator,Q =−q0 2 0 02 −q1 4 00 4 −q2 20 0 1 −q3 .Determine the following quantities.(a) The expected holding time of each state (i.e. the expected amount of time spent in eachstate before a jump).(b) The stationary distribution πQ associated with Q.(c) The proportion of time the process spends in state 3 in the long run.(d) The expected return time, m2 = E2[T2], for state 2.(e) The transition matrix R of the embedded chain.(f) A stationary distribution πR of the embedded chain. Is the stationary distribution πRthat you found unique? Explain your reasoning.2

(a) The expected holding time of each state is the reciprocal of the rate at which the process leaves that state. This is given by the negative of the diagonal elements of the Q-matrix. So, the expected holding times are 1/q0, 1/q1, 1/q2, and 1/q3 for states 0, 1, 2, and 3 respectively.

(b) The stationary distribution πQ is the left eigenvector of the Q-matrix associated with the eigenvalue 0, normalized to sum to 1. To find this, we solve the system of linear equations (πQ)Q = 0 and sum(πQ) = 1.

(c) The proportion of time the process spends in state 3 in the long run is given by the corresponding element of the stationary distribution, πQ(3).

(d) The expected return time m2 for state 2 is the reciprocal of the stationary distribution of state 2, so m2 = 1/πQ(2).

(e) The transition matrix R of the embedded chain is obtained by normalizing the off-diagonal elements of the Q-matrix by the rate of leaving each state. That is, R(i, j) = Q(i, j) / -Q(i, i) for i ≠ j.

(f) A stationary distribution πR of the embedded chain is the left eigenvector of the R-matrix associated with the eigenvalue 1, normalized to sum to 1. To find this, we solve the system of linear equations (πR)R = πR and sum(πR) = 1. The stationary distribution πR is unique if the embedded chain is irreducible and aperiodic, which means that it is possible to get from any state to any other state in a finite number of steps and the greatest common divisor of the lengths of all cycles is 1.

(a) The expected holding time of each state is the reciprocal of the diagonal elements of the Q-matrix. Therefore, the expected holding times are 1/q0, 1/q1, 1/q2, and 1/q3 for states 0, 1, 2, and 3 respectively.

(b) The stationary distribution πQ associated with Q is the solution to the system of equations πQ = πQ * Q, subject to the condition that the elements of πQ sum to 1. This is a system of linear equations that can be solved using standard techniques.

(c) The proportion of time the process spends in state 3 in the long run is given by the corresponding element of the stationary distribution πQ.

(d) The expected return time, m2 = E2[T2], for state 2 is the expected time to return to state 2 starting from state 2. This can be computed using the formula m2 = 1 + ∑(πQ(i) * m2(i)) for i ≠ 2, where m2(i) is the expected time to go from state i to state 2.

(e) The transition matrix R of the embedded chain is obtained by normalizing the off-diagonal elements of the Q-matrix. The (i, j) element of R is given by Q(i, j) / -Q(i, i) for i ≠ j, and 0 for i = j.

(f) A stationary distribution πR of the embedded chain is the solution to the system of equations πR = πR * R, subject to the condition that the elements of πR sum to 1. This is also a system of linear equations that can be solved using standard techniques. The stationary distribution πR is unique if and only if the embedded chain is irreducible and aperiodic. If the chain is not irreducible, there may be multiple stationary distributions. If the chain is not aperiodic, the stationary distribution may not exist.