Let \Sigma denote a covariance matrix for some random vector \mathrm{{\boldsymbol X}} \in \mathbb {R}^ d. (Assume that \mathbf E[\left\| \mathrm{{\boldsymbol X}} \right\| _2^2] < \infty.)

Question

Which of the following properties does \Sigma necessarily have? (Choose all that apply.)

Symmetric

Positive Semidefinite

Positive Definite

Orthogonal

Answer 1

The properties that the covariance matrix \(\Sigma\) necessarily have are:

1. Symmetric: The covariance matrix \(\Sigma\) is always symmetric because the covariance between two random variables \(X_i\) and \(X_j\) is the same as the covariance between \(X_j\) and \(X_i\).

2. Positive Semidefinite: The covariance matrix \(\Sigma\) is always positive semidefinite because for any vector \(v\) in \(\mathbb{R}^d\), we have \(v^T\Sigma v \geq 0\). This property ensures that the variances and covariances are non-negative.

3. Orthogonal: The covariance matrix \(\Sigma\) does not necessarily have to be orthogonal. The orthogonality of a matrix refers to its columns or rows being mutually orthogonal unit vectors. Covariance matrices can have non-zero off-diagonal elements, indicating that the variables are correlated.

4. Positive Definite: The covariance matrix \(\Sigma\) may or may not be positive definite. A covariance matrix is positive definite if and only if all its eigenvalues are positive. It is positive semidefinite if and only if all its eigenvalues are non-negative. So, while positive semidefiniteness is guaranteed, positive definiteness depends on the specific correlations and variances of the random variables.