Exercise: Path calculation

Question

0.0/3.0 points (graded)
Consider a Markov chain with the following transition probability graph:

This figure depicts a Markov chain with seven states. The possible states are 1, 2, up to 7. 1, 2, and 3 are depicted as a row of three circles, in this order, at the upper-left of the figures. 4, 5, 6, and 7 are depicted as four squares arranged in a square, with S_4 and S_5 to the right of S_3 (in this order), 6 directly below 4, and 7 directly below 5. In this figure, transitions are depicted by arrows from a circle (representing the source) to a circle (representing the destination). The source and the destination may be the same, and transition arrows may be labelled with a number or a variable indicating the probability that if we start at the source, we will be at the given destination at the next step. In the row consisting of 1, 2, and 3, there are self-loops within 1 and 3, which are circular arrows with both the tail and the head at the same state. These self-loops are labelled with probabilities of 0.4 and 0.6, respectively. There are also transitions between 1 and 2, as well as between 2 and 3; the probability from 1 to 2 is 0.6 and from 2 to 1 is 0.2, while the probability from 2 to 3 is 0.8 and from 3 to 2 is 0.4. Connecting the two components (the line with three states and the square with four states),there's only one transition from 4 to 3 with probability 0.4. Within the square, there's a transition from 4 to 5 with probability 0.3 and 4 to 6 with probability 0.3. From state 5, the only transition is to state 6, with probability 1; similarly, from state 7, the only transition is to state 5 with probability 1. Lastly, from state 6, there's a transition to state 4 with probability 0.2 and to state 7 with probability 0.8.
\mathbf{P}(X_1=6,X_2=4,X_3=3\mid X_0=4)=\,
unanswered
\mathbf{P}(X_{103}=3 \mid X_{100}=1)=\,

Answer 1

To calculate the path probabilities in the given Markov chain, we need to multiply the transition probabilities along the desired path.

1) To calculate P(X1=6, X2=4, X3=3 | X0=4), we start at state 4 and have the following transitions:
- From state 4 to state 6 with probability 0.3
- From state 6 to state 4 with probability 0.2
- From state 4 to state 3 with probability 0.4

Therefore, the path probability is:
P(X1=6, X2=4, X3=3 | X0=4) = 0.3 * 0.2 * 0.4 = 0.024

2) To calculate P(X103=3 | X100=1), we start at state 1 and need to reach state 3 after 103 steps. We can calculate this by raising the transition matrix to the power of 103 and looking at the entry in the row corresponding to state 1 and the column corresponding to state 3.

Unfortunately, the transition probability matrix is not given in the exercise. Without the transition probabilities, it is not possible to calculate the desired path probability.