The word eigen, having a German origin, means characteristics. The eigenvalues and eigenvectors give the characteristic, but of what? Let’s understand it through a geometric example.
The linear transformations such as scaling, rotation, and shearing can be expressed using matrices. For example, by applying a vertical scaling of +2 to every vector of a square, will transform the square into a rectangle. In the same way, by applying a horizontal shear to the square, it becomes a parallelogram.
Note that during these transformations, some of the vectors remain on the same line (span) as they were earlier. As shown in the figure,
Note that, in the above figure, after vertical scaling of +2, every vector’s (except horizontal and vertical ones) direction has changed. These two vectors are special and are the characteristic of this particular transform. Hence, these are called eigenvectors.
An eigenvector is a vector, which after applying the linear transformation, stays in the same span i.e. changes by only a scalar factor.
The eigenvalue is how much the eigenvectors are transformed (stretched or diminished).
In the same way, the horizontal shear transformation to square gives only one eigenvector (horizontal one) having an eigenvalue of +1.
In 180 degree rotation of square, all vectors are still laying on the same span, but their direction is reversed. Hence, all vectors are eigenvectors, having an eigenvalue of -1.
In case of 3d rotation transformation of cube, the eigenvector gives the axis of rotation.
Suppose we have a transformation matrix
A and we apply this transformation to vector
x. This will be equivalent to stretching (or diminshing) the vector
x by a scalar factor
I is the identity matrix.
The above euqation has a non-zero solution iff the determinant of the matrix
(A − λI) is zero i.e.
Evaluating this determinant gives the characteristic polynomial i.e.
The solution of this equation gives the eigvenvalues. Put these eigenvalues in the original expression to get their corresponding eigenvectors.