Consider a Markov chain {X0,X1,…}, specified by the following transition probability graph.

P(X2=2∣X0=1)= - unanswered

Find the steady-state probabilities ð1, ð2, and ð3 associated with states 1, 2, and 3, respectively.

ð1= - unanswered
ð2= - unanswered
ð3= - unanswered

For n=1,2,…, let Yn=Xn−Xn−1. Thus, Yn=1 indicates that the nth transition was to the right, Yn=0 indicates that it was a self-transition, and Yn=−1 indicates that it was a transition to the left.

limn→∞P(Yn=1)= - unanswered

Is the sequence Y1,Y2,… a Markov chain?

- unanswered

Given that the nth transition was a transition to the right (Yn=1), find (approximately) the probability that the state at time n−1 was state 1 (i.e., Xn−1=1). Assume that n is large.

- unanswered

Suppose that X0=1. Let T be the first positive time index n at which the state is equal to 1.

E[T]= - unanswered

Does the sequence X1,X2,X3,… converge in probability to a constant?

- unanswered

Let Zn=max{X1,…,Xn}. Does the sequence Z1,Z2,Z3,… converge in probability to a constant?

- unanswered