Today I’m going to talk to you about a very important and widely used concept in the field of machine learning — eigenvalues and eigenvectors of matrices.

So let’s look at the definition, which itself is pretty simple, let’s say we have A matrix A of order N and A real number, so that we can find a non-zero vector x that satisfies:


If we can find it, we’ll weigh itIs the eigenvalue of matrix A, and the non-zero vector x is the eigenvector of matrix A.


Geometric meaning


It’s really hard to tell anything by light from the above formula, but if we combine it with the geometric meaning of the matrix transformation, it becomes a lot clearer.

We know that for an n-dimensional vector x, if we multiply it by A square matrix A of order n, we get Ax. Geometrically, I’m applying a linear transformation to some vector x. The resulting vector y and the original vector x have changed direction and length.

But, for A particular matrix A, there’s always some vector x in A particular direction, such that Ax and x don’t change in direction, just in length. Let’s think of the change in length as a coefficient, so for vectors like this, called eigen vectors of matrix A,That’s the special value of this eigenvector.


Solving process


Let’s do a very simple transformation of the original:


I here is the identity matrix, and if you expand it out, you get a homogeneous system of linear equations with n elements. We’re already familiar with this, that for a homogeneous linear system to have a non-zero solution, you need the determinant of the coefficients


Not zero, which means the rank of the coefficient matrix is less than n.

We expand this determinant:


This is a toIs a system of n-degree equations with one unknown, and the n-degree equations have n solutions in the complex set. If we look at the equation above, we can see thatOnly on the positive diagonal, and obviously, the eigenvalues of A are the solutions to the system. Because the n-degree system has n solutions in the complex set, the matrix A has n eigenvalues in the complex set.

Let’s try this as an example:

Assumptions:


then, we apply the formula of finding the root to getThe two root, there are:.

This conclusion can be extended to all n, that is to say, for A square matrix A of order n, we can get:


case


Let’s look at an example:


Us into, can be obtained:


So:You can see it

whenWhen:




By solving, we can get:All,The vectors are all eigen vectors of A.

Similarly, whenWhen:


By solving, we can get:All,The vectors are all eigen vectors of A.


Use Python to solve for eigenvalues and eigenvectors


In our previous articles, we introduced the power of Python for computational science, and this time it is no exception for solving eigenvalues and eigenmatrices. By using the library functions in Numpy, we can double compute eigenvalues and eigenvectors in one line of code very easily.

Let’s look at the code:

import numpy as np

a = np.mat([[3.1], [1.3]])
lam, vet = np.linalg.eig(a)
Copy the code

The np.linalg.eig method returns two values, the first value is the eigenvalue of the matrix, the second value is the eigenvector of the matrix, let’s look at the result:

Why is the eigenvector 0.707? Because Python automatically does the unitization for us, and the vectors returned are all unit vectors, which is really sweet.


conclusion


So that’s the end of the introduction to eigen values and eigen vectors of matrices, for algorithm engineers, as opposed to how to compute eigen vectors and eigen values. It is even more important to understand their conceptual and geometric meanings, because they are widely used in machine learning, and in many dimensionality reduction algorithms, eigenvalues and eigenvectors of matrices are used extensively.

The principle of dimensionality reduction algorithm is not described here, but will be updated in a future article. Those of you who are interested can look forward to it.

This is the end of the article, this is the last article on linear algebra topic, a short six articles of course can not cover all the knowledge of linear algebra in the discipline, but in practice, the commonly used content has basically included. We’ll be starting a new Python feature next week, so look forward to it.

If you feel that you have gained something, please click on the following or forward, your support is my biggest motivation.