1. Quiz Question Consider the Markov chain (Xn, n = 0, 1, 2, . . . ) with state space S = {1, 2, 3, 4}and transition diagramP =1/3 1/3 1/3 01/4 1/2 1/4 00 1/2 0 1/20 1 0 0 .(a) Draw the transition diagram.(b) Find P (X1 = 1, X2 = 2, X3 = 3 | X0 = 2).(c) Find P (X2 = 2, X3 = 1 | X0 = 4, X1 = 2).(d) Find p(2)21 = P (X2 = 1 | X0 = 2).(e) Find P (X7 = 1 | X5 = 2).

Quiz question For each of the following transition matrices for Markov chains with state spaceS = {1, 2}, write down the Full Balance Equations and find the equilibrium distribution(s).(a) P =0.4 0.60.8 0.2(b) P =0.82 0.180.82 0.18

Quiz Question Consider the Markov chain with transition diagram345210.10.50.50.60.30.50.20.40.30.50.50.20.4i.e., with transition matrixP =0.6 0.4 0 0 00.5 0.5 0 0 00 0.5 0 0.5 00 0.5 0.3 0 0.20 0.1 0.2 0.4 0.3 .(a) Starting from state 5, what is the most likely two-step path? What is the probability(starting from state 5) of following that two-step path?(b) In this Markov chain, can you get from every state to every other state eventually?

Question 2(c)Let 𝑋1 → 𝑋2 → 𝑋3 → · · · → 𝑋𝑛 form a Markov chain in this order. Thus, the joint probabilityof 𝑋1, . . . , 𝑋𝑛 are given by𝑝(𝑥1, 𝑥 − 2, . . . , 𝑥𝑛) = 𝑝(𝑥𝑛 |𝑥𝑛−1) 𝑝(𝑥𝑛−1 |𝑥𝑛−2) · · · 𝑝(𝑥2 |𝑥1) 𝑝(𝑥1).1. Express 𝐼 (𝑋1; 𝑋2, . . . , 𝑋𝑛) in terms of its entropy and conditional entropy [2 Marks]2. Simplify the entropy expression you derived above to reduce 𝐼 (𝑋1; 𝑋2, . . . , 𝑋𝑛) to itssimplest form. Please do not use other methods to simplify, you will not receive anymarks.

If the initial state probability distribution of Markov chain is ( ) ( )and transition probability matrix of the chain is ( ). Compute theprobability distribution of the chain after 2 steps

Problem 4. Consider a Markov process that transit between states 0,1 and 2, with thetransition probability matrixP =0 1 20 1 0 01 0.1 0.6 0.32 0 0 1From this matrix we see that once the process reaches state 0 or state 2, it stays thereforever. Hence these two states are called absorbing states. If the process starts at state 1,the process may stay in state 1 for a duration but eventually the process will leave state1 and enter one of the absorbing states.1. What is the probability that the process twill enter into state 2 given the processstarts at state 1?2. On average how long does it take to reach one of the absorbing states?