Let Θ1, Θ2, W1, and W2 be independent standard normal random variables. We obtain two observations,

Question

X1=Θ1+W1,X2=Θ1+Θ2+W2.
Find the MAP estimate θ^(theta hat)=(θ^1,θ^2) of (Θ1,Θ2) if we observe that X1=1, X2=3.
1.θ^1 (theta hat)=?
2.θ^2(theta hat)=?

Answer 1

2. Answer is 1

Answer 2

1. Answer is also 1

Answer 3

As usual, we focus on the exponential term in the numerator of the expression given by Bayes' rule. The prior contributes a term of the form

e^(-1/2*(Θ^2_1Θ^2_2))

Conditioned on (Θ_1, Θ_2) = (θ_1, θ_2), the measurements are independent. In the conditional universe, X_1 is normal with mean θ_1, X_2 is normal with mean θ_1 + θ_2, and both variances are 1. Thus, the term makes a contribution of the form

e^(-1/2*(x_1-θ_1)^2) * e^(-1/2*(x_2-θ_1-θ_2)^2)

We substitute X_1 = 1 and X_2 = 3, and in order to find the MAP estimate, we minimize the expression

-1/2*(θ^2_1+θ^2_2+(θ_1-1)^2+(θ_1+θ_2-3)^2)

Setting the derivatives (with respect to θ_1 and θ_2) to zero, we obtain:

θ'_1+(θ'_1-1)+(θ'_1+θ'_2-3) = 0, θ'_2+(θ'_1+θ'_2-3) = 0,

or

3θ'_1+θ'_2 = 4, θ'_1+2θ'_2 = 3

Either by inspection, or by substitution, we obtain the solution θ'_1 = 1, θ'_2 = 1.