Question

Consider the linear regression model introduced in the slides and lecture, restated below:

Linear regression model : (\mathbf X_1, Y_1),\ldots , (\mathbf X_ n,Y_ n) \in \mathbb {R}^ d \times \mathbb {R} are i.i.d from the linear regression model Y_ i={\boldsymbol \beta }^\top \mathbf X_ i + \varepsilon _ i, \quad \varepsilon _ i \stackrel{iid}{\sim } \mathcal{N}(0,1) for an unknown {\boldsymbol \beta }\in \mathbb {R}^ d and \mathbf X_ i \sim \mathcal{N}_ d(0,I_ d) independent of \varepsilon _ i.

Suppose that {\boldsymbol \beta }= \mathbf{1} \in \mathbb {R}^ d, which denotes the d-dimensional vector with all entries equal to 1.

What is the mean of Y_1?

\mathbb E[Y_1]=\quad
unanswered
What is the variance of Y_1? (Express your answer in terms of d.)

\textsf{Var}(Y_1)=\quad

Answer 1

The mean of Y_1 can be found by substituting {\boldsymbol \beta }=\mathbf{1} and \mathbf X_1=0 into the linear regression model:

\begin{align*}
\mathbb E[Y_1] &= {\boldsymbol \beta }^\top \mathbf X_1 + \mathbb E[\varepsilon_1] \\
&= \mathbf 1^\top \mathbf 0 + \mathbb E[\varepsilon_1] \\
&= 0 + \mathbb E[\varepsilon_1] \\
&= 0.
\end{align*}

Therefore, the mean of Y_1 is 0.

To find the variance of Y_1, we can use the fact that \varepsilon_i \sim \mathcal{N}(0,1). Since \varepsilon_1 is independent of \mathbf X_1, we have:

\textsf{Var}(Y_1) = \textsf{Var}({\boldsymbol \beta }^\top \mathbf X_1 + \varepsilon_1).

Since \mathbf{1}\in \mathbb{R}^d, we have:

{\boldsymbol \beta }^\top \mathbf X_1 = \mathbf{1}^\top \mathbf{0} = 0.

Therefore, the variance of Y_1 reduces to:

\textsf{Var}(Y_1) = \textsf{Var}(\varepsilon_1) = 1.

Therefore, the variance of Y_1 is 1.