Question

Setting these equal to zero and isolating terms with a and b to one side, we obtain a system of linear equations

\displaystyle \mathbb E[Y] \displaystyle = a + \mathbb E[X] b
\displaystyle \mathbb E[XY] \displaystyle = \mathbb E[X] a + \mathbb E[X^2] b
Multiplying the first equation by \mathbb E[X] and subtracting from the second equation gives

(\mathbb E[X^2] - \mathbb E[X]^2)b = \mathbb E[XY] - \mathbb E[X]\mathbb E[Y] \quad \implies \quad b = \frac{\textsf{Cov}(X,Y)}{\textsf{Var}(X)}.

Plugging this value back into the first equation to solve for a gives

a = \mathbb E[Y] - \frac{\textsf{Cov}(X,Y)}{\textsf{Var}(X)} \mathbb E[X].

We now compute the Hessian

H = \left(\begin{array}{cc} f_{aa} & f_{ab} \\ f_{ba} & f_{bb} \end{array}\right)

to make sure that this pair (a,b) critical point is a local minimum. The determinant of H at this value (a,b) is



-\textsf{Var}(X)

4\textsf{Var}(X)

\mathbb E[X]

\textsf{Cov}(X,Y)