Question

Consider a Markov chain {X0,X1,…}, specified by the following transition probability graph.

P(X2=2∣X0=1)= - unanswered

Find the steady-state probabilities ð1, ð2, and ð3 associated with states 1, 2, and 3, respectively.

ð1= - unanswered
ð2= - unanswered
ð3= - unanswered

For n=1,2,…, let Yn=Xn−Xn−1. Thus, Yn=1 indicates that the nth transition was to the right, Yn=0 indicates that it was a self-transition, and Yn=−1 indicates that it was a transition to the left.

limn→∞P(Yn=1)= - unanswered

Is the sequence Y1,Y2,… a Markov chain?

- unanswered

Given that the nth transition was a transition to the right (Yn=1), find (approximately) the probability that the state at time n−1 was state 1 (i.e., Xn−1=1). Assume that n is large.

- unanswered

Suppose that X0=1. Let T be the first positive time index n at which the state is equal to 1.

E[T]= - unanswered

Does the sequence X1,X2,X3,… converge in probability to a constant?

- unanswered

Let Zn=max{X1,…,Xn}. Does the sequence Z1,Z2,Z3,… converge in probability to a constant?

- unanswered

Answer 1

4) No
7) No
8) Yes

Answer 2

1. 0.44
2. 1/9
2/9
6/9
3. 1/9

Answer 3

5. 2/5

Answer 4

Somebody knows the answer to Q6...E[T]

Answer 5

E[T]=9

Answer 6

You guys are amazing, I love you