Setting these equal to zero and isolating terms with a and b to one side, we obtain a system of linear equations
\displaystyle \mathbb E[Y] \displaystyle = a + \mathbb E[X] b
\displaystyle \mathbb E[XY] \displaystyle = \mathbb E[X] a + \mathbb E[X^2] b
Multiplying the first equation by \mathbb E[X] and subtracting from the second equation gives
(\mathbb E[X^2] - \mathbb E[X]^2)b = \mathbb E[XY] - \mathbb E[X]\mathbb E[Y] \quad \implies \quad b = \frac{\textsf{Cov}(X,Y)}{\textsf{Var}(X)}.
Plugging this value back into the first equation to solve for a gives
a = \mathbb E[Y] - \frac{\textsf{Cov}(X,Y)}{\textsf{Var}(X)} \mathbb E[X].
We now compute the Hessian
H = \left(\begin{array}{cc} f_{aa} & f_{ab} \\ f_{ba} & f_{bb} \end{array}\right)
to make sure that this pair (a,b) critical point is a local minimum. The determinant of H at this value (a,b) is
-\textsf{Var}(X)
4\textsf{Var}(X)
\mathbb E[X]
\textsf{Cov}(X,Y)
1 answer