Question

Given two data points in 2 dimensions:

\displaystyle \displaystyle \mathbf{x}^{(1)} \displaystyle = \displaystyle (x^{(1)}, y^{(1)})= (0,1)
\displaystyle \mathbf{x}^{(2)} \displaystyle = \displaystyle (x^{(2)}, y^{(2)})=(0,-1)
Notice that this sample is already centered, with sample mean 0 in both x- and y- coordinates.

Without computation, find the direction of largest variance. (This is the first principal component \text {PC1}.



(1,0)

(0,1)

(1,1)
unanswered
Let us now compute the first principal component and check if it matches with the above. Find the sample covarance matrix \mathbf{S} for the given data.

Recall: Sample covarance matrix
Show

(Enter as a matrix, e.g. type [[1,2],[3,4]] for the matrix \begin{pmatrix} 1& 2\\ 3& 4\end{pmatrix}.)

\mathbf{S}=\quad
unanswered
[Math Processing Error]
What are the eigenvalues \lambda _1,\lambda _2 of the covariance matrix \mathbf{S}? In other words, find pairs of scalar \lambda and \mathbf v_\lambda such that \mathbf{S}\mathbf v_\lambda =\lambda \mathbf v_\lambda, i.e. the matrix \mathbf{S} acts on a vector \mathbf v_\lambda by rescaling it.

Review: eigenvalues and eigenvectors
Show

(Enter the eigenvalues in decreasing order, i.e. \lambda _1> \lambda _2. )

\lambda _1=\quad
unanswered
\lambda _2=\quad
unanswered
Find an eigenvector corresponding to the largest eigenvalue \lambda _1. The span of this vector is called the first principal component (PC1) .

(All non-zero scalar multiples will be accepted as correct.)

\mathbf v_{\text {PC1}}=\quad
unanswered
{::}
(Is your answer for the first principal component the same as your answer in the first part is this question, the direction of largest variance? )

Answer 1

Yes, the direction of the largest variance is the same as the first principal component.