Question

In the setting of deterministic design for linear regression, we assume that the design matrix \mathbb {X} is deterministic instead of random. The model still prescribes \mathbf Y= \mathbb {X} {\boldsymbol \beta }+ {\boldsymbol \varepsilon }, where {\boldsymbol \varepsilon }= (\varepsilon _1, \ldots , \varepsilon _ n) is a random vector that represents noise. Take note that the only random object on the right hand side is \mathbf\varepsilon, and that Y is still random.

For the rest of this section, we will always assume (\mathbb {X}^ T \mathbb {X})^{-1} exists; i.e. \mathrm{rank}(\mathbb {X}) = p.

Recall that the Least-Squares Estimator \hat{{\boldsymbol \beta }} has the formula

\hat{{\boldsymbol \beta }} = (\mathbb {X}^ T \mathbb {X})^{-1} \mathbb {X}^ T \mathbf Y.

If we assume that the vector {\boldsymbol \varepsilon } is a random variable with mean \mathbb E[{\boldsymbol \varepsilon }] = 0, then in the deterministic design setting: “The LSE \hat{{\boldsymbol \beta }} is a random variable, with mean..." (choose all that apply)



0

(\mathbb {X}^ T \mathbb {X})^{-1} \mathbb {X}^ T \mathbb E[\mathbf Y]

\mathbb {X}^ T \mathbb {X} \beta

\beta

\epsilon