Understand the meaning of matrix rank and determinant in one article

As an engineering student, we have been using linear algebra knowledge such as matrices and determinants for a long time. In this article, I would like to talk about these issues, namely, what is area and what is the high-dimensional generalization of area.

1 What is area?

When it comes to what is area, you may first think of the length*width that we often use in our daily lives. Is this true? In fact, the area we are talking about here is actually the basic unit of area in Euclidean space geometry: the area of ​​a parallelogram. The definition of the area of ​​a parallelogram in geometry is the length of two adjacent sides multiplied by the sine of the angle between them.

But when we face some more general situations and higher-dimensional mathematical problems, we need to generalize the definition of area. First of all, we should note that area is a scalar, which is the result of multiplying two vectors from two adjacent sides. Therefore, we need to regard area as a mapping relationship.

Here V can be regarded as a proper quantity, V*V represents two proper ordered pairs, so f is naturally the required area.

Now we will prove that this mapping is a linear mapping, please sit tight:

Now let's take a simple example. Let's assume that the first vector is (1.0) and the second vector is (0,1). That is to say, the two vectors are positive unit vectors on the X-axis and Y-axis, respectively. Then the quadrilateral formed by these two vectors is actually a square. According to the definition of area, it is actually * width = 1*1 = 1

So we can get:

Now suppose that the first vector is scaled by a times, the area of ​​the quadrilateral will also become a times the corresponding area, and the area will also become a times the original area. If the second vector is scaled by b times, the area will also become b times the original area. If we scale both vectors by ab times at the same time, the area will also become ab times the original area. This shows that the mapping of the area is linear for the scalar product of the other two operands, as follows:

In fact, in actual situations, the mapping of area is also linear with respect to the vector addition of its operands (vectors). Since the operation of vector addition itself is a linear one, the mapping of area is also a linear mapping. Now I would like to explain some consequences of the linearity of mapping addition through a few examples.

The parallelogram formed by two collinear vectors is a line, so the area is 0. Now assume that the area mapping is a linear mapping about a proper addition, then we have the following result

In fact, a theory is used here:

That is to say, after exchanging the order of mutually perpendicular operands, the mapping of the area becomes a negative value. Whether it is positive or negative depends on your definition. In general, we put the vector of the X axis in front and the vector of the Y axis in the back. The area of ​​a parallelogram from the X axis to the Y axis is generally regarded as a positive sign.

2 Applications in three-dimensional space

In three-dimensional space, we generally use the right-hand rule to conduct experiments. If the square on the X-axis is the head and the positive direction of the Y-axis is the tail, the right-hand rule tells me that the outward direction of the paper is the positive direction of the area. If it is the other way around, the inward direction of the paper is the positive direction of the area. It is opposite to the direction of the specified positive and negative signs. Now the geometric meaning of the positive and negative signs is more obvious.

Now let's assume that the area of ​​the parallelogram formed by any two vectors in the plane can be expressed using the formula:

Here, it is not difficult to see that the so-called area is actually the determinant of a 2*2 matrix:

Just like the picture below:

In fact, our first row is our first row vector (a, b), the second row is the second row vector (c, d), or the first column is the rank of the first column vector (a, b), and the second column is the rank of the second column vector (c, d). Of course, this depends on whether we write the vector as a row vector or a column vector.

3 Calculation of properties of determinants

In the above reasoning, we can easily find that the value of the determinant is irrelevant whether the vectors related to the determinant are written as horizontal rows of column vectors or vertical rows of row vectors. This is why, when calculating the determinant, the status of the row and column is equal. And we should also note that according to the above analysis, the area is negative when the order of the vectors is exchanged. This is why in the determinant, the negative sign should be taken once when the column vector or row vector is exchanged once. In addition, the properties of other calculations of the determinant are actually reflected in the linearity of the area mapping.

So, to sum up, the determinant itself is actually a generalization of the form of area. In fact, it is the volume of a generalized quadrilateral defined in N dimensions formed by N vectors under a given set of bases. In fact, this is the essence of the determinant.

4 A generalization of the determinant

Based on the above conclusion, we can easily generalize it to a calculation of three-dimensional volume:

Here we should note that the definition of a determinant is actually the product of elements from different columns in each row and the sign is related to the so-called inverse order. What is inverse imaginary? The so-called inverse order, its geometric meaning is that after a positive direction is specified (for example, the order from 1, 2, 3, 4, 5...N is defined as a positive sign), exchanging any pair of numbers will take a negative sign once. We have seen this property in the area function mentioned above. In fact, volume, generalized volume in higher dimensions, also has a positive direction, but it is difficult to illustrate it with the right-hand rule (and cross product). The limitation of the right-hand rule is also one of the motivations for generalizing high-dimensional area to be expressed as a determinant.

The property that the sign can be changed by exchanging any bunch of indices is actually called antisymmetry. At this point, if you are good at thinking, you will wonder why we need to take the product of elements in different rows and columns. Because if any two elements are in the same row and column, then they exchange their column indices, the product remains unchanged but the sign is opposite. Therefore, the product must be 0, which is one of the reasons why it is not reflected in the determinant value.

The definition of determinant is actually quite complicated. It actually comes from the antisymmetry of the vast area mapping. In fact, the area mapping is a 2-dimensional one. If we expand the two-dimensional to multiple dimensions, we can actually find that the R-dimensional form is exactly the same as the R*R determinant form.

In fact, here we can summarize what the various dimensions represent. Two dimensions represent the area in the plane, three dimensions are naturally the volume in three-dimensional space, and four dimensions are the hypervolume in four-dimensional space. And so on. In the above reasoning, we found that the matrix written by the base coordinates given by these vectors must be a square matrix, and the determinant of the matrix corresponds to the area or volume. I believe that such generalization proofs can be seen in any linear algebra book. I am just speaking human language.

5 Determinant and matrix inverse

We know many theorems, such as the matrix with a determinant of 0 is irreversible, and the matrix with a determinant of non-zero is reversible. At this time, we can't help but ask how the determinant representing the area is combined with the reversibility of linear change.

At this point we should understand the geometric meaning of linear change. Now let me explain:

If we write a set of linearly independent vectors in space in the form of column vectors, then the volume of the N-dimensional volume they span is not zero. According to the above analysis, its value is given by the determinant. After the vector is transformed by linear transformation A, the new vector form is as follows:

Note that A is an N*N matrix and the vector is a column vector.

Before the transformation, the volume of the N-dimensional body is:

After the transformation, the volume of the N-dimensional volume is (note that the second equation actually explains how the geometric meaning defines matrix multiplication, that is, the multiplication of the N*N matrix A and another N*N matrix consisting of N column vectors):

If the determinant of A is not zero, it means that after this transformation, the volume of the N-dimensional body is not NULL. Combining the properties of linear independence and volume, we can say:

If the determinant of A is not zero, then A can map a set of linearly independent vectors into a new set of linearly independent vectors; A is reversible (one-to-one mapping, fidelity mapping, KERNEL is {0})

If the determinant of A is zero, then A maps a set of linearly independent vectors into a set of linearly dependent vectors.

If the determinant of A is negative, then A will change the direction of the original N-dimensional volume.

From linear independence to linear dependence, some information is lost (for example, collapsing into colinear or coplanar), so this transformation is obviously irreversible. Whether the linear independence is directly related to the volume of the N-dimensional body, and this volume value is related to the determinant of A. Therefore, we have established a geometric relationship between the determinant of A and whether it is reversible.

For example, let's assume that A is a 3D matrix. If there are three linearly independent vectors before mapping, we know that the volume they span is not 0; after mapping, the new vectors they correspond to can also span a parallelepiped, then the volume of this parallelepiped is the original volume multiplied by the determinant of A.

Obviously, if the determinant of A is 0, then the volume of the new "parallelepiped" after the transformation will inevitably be 0. According to the above conclusion, we have: this set of new vectors after the transformation is linearly related.

in conclusion:

Whether the determinant of the linear transformation A is zero represents the fidelity of its mapping, that is, whether it can transform a set of linearly independent vectors into another set of vectors that maintain independence.

6 Rank

But sometimes, although the determinant A cannot make a large number of vectors in space linearly independent, it can ensure that a small number of vectors are linearly independent. This number of vectors is often less than the dimension of the linear space. This number is called the rank of the linear transformation A.

For example: a 3*3 matrix A with a rank of 2, because the rank is less than 3, then any 3D hexahedron will have a volume of 0 after its transformation, degenerate a face, but there is still a face with a non-zero area, and it is still a face with a non-zero area after the transformation.

So the rank of a linear transformation is nothing more than the maximum dimension of a geometric shape that can maintain a non-zero volume after the transformation.

By understanding the geometric meaning of rank, determinant, and reversibility, we can arbitrarily construct a linearly changing A, so that it can either preserve all geometric bodies, or reduce dimensions to geometric bodies with specific dimensions and structures, or compress them into lower-dimensional geometric bodies. Therefore, it can be regarded as a "dimensionality reduction attack".

Reasoning in higher dimensions, I hope that those who are interested can prove it by themselves, and if you don’t understand the problem, you can also comment below the article. I hope to communicate with you more, thank you for your advice.

This article is reproduced from Leifeng.com. If you need to reprint, please go to Leifeng.com official website to apply for authorization. This article was written by Xia Hongjin and originally published on the author's personal blog.