Introduction
This post’s title is intentionally vague. Usually, I write an introduction that describes the path the post will walk and then I meander down that path from beginning to end, and write a conclusion that sums it all up. After thinking about how to write this post in the most engaging way, it occurred to me that mathematics has often felt the most satisfying to me when I’ve felt as though I was discovering definitions and theorems rather than being read them by professors. My hope is that in addition to conveying the beauty and ingenuity behind what follows, I am able to also pass along some of the wonder that I myself felt during journey. With that, follow me…
Review of linear operators
Suppose we have a vector space $V$. One of the most important objects (if not the most important) we study in linear algebra are structure preserving maps called linear transformations (a.k.a. linear maps). As a quick review, a transformation is (loosely) a function that takes a vector from one vector space and turns it into a vector in another using some rule. In order for $T$ to be called a linear transformation, we need the following two properties to hold:
- If $u$ and $v$ are vectors, it must be the case that $T(u + v) = T(u) + T(v)$.
- If $v$ is a vector and $c$ is a member of the $V$’s underlying field, it must be the case that $T(cv) = cT(v)$.
In English, (1) and (2) say that linear transformations must preserve addition and scalar multiplication – that is, adding/scaling then mapping must produce the same result as mapping then adding/scaling. For this post, we’re going to focus our attention on linear operators, or linear maps from $V$ to itself.
Invariant subspaces
So suppose $T$ is an operator on $V$. It’s natural to wonder how $T$ behaves with respect to subspaces of $V$. In the case of an operator, it will always be the case that $T$ maps a subspace $U$ to some subspace of $V$, but how $T$ transforms an arbitrary choice of subspace is unclear. Let’s simplify a little bit and think about the neatest-possible set of outputs that $T$ might produce as it acts on vectors from $U$; we are going to focus on subspaces which, with respect to $T$, are in some sense self-contained. In other words, let’s require that $T$ map every vector in $U$ get back into $U$ somewhere. (A more compact way of phrasing our requirement is that we want $T$ to be an operator on $U$.) If $T$ behaves this way with respect to $U$, we say that $U$ is invariant under $T$. (In technical jargon, we say a subspace $U$ is invariant under $T$ if for every vector $u \in U$, $T(u) \in U$.)
I don’t know about you, but I still don’t feel on sure enough footing; let’s simplify further. Instead of letting our invariant subspaces get big and complicated, let’s restrict our focus to $T$’s invariant subspaces of the lowest possible dimension. As a brief aside, the dimension of a vector space is the smallest number of vectors, linear combinations of which can comprise an entire space. For example, consider $\mathbf{R}^2$ (the $(x,y)$ plane). It is a vector space of dimension 2. Why? Because I can make the two vectors $(0,1)$ and $(1,0)$ into any vector (coordinate pair) that I want! How? Notice that $(a,b) = a \times (1,0) + b \times (0,1)$. In the $\mathbf{R}^2$ example, we call ${(0,1), (1,0)}$ a basis (“a” basis because there are infinitely many others). A vector space’s dimension is defined as the size of any basis (all bases are the same size).
Eigenvalues
In light of our detour, what might a one-dimensional invariant subspace look like? Well, a one dimensional subspace has a basis of size one, which means that the subspace is made up of linear combinations of a single vector, i.e. its scalar multiples! In math terms – and in this case, I actually think the symbols help – a one dimensional subspace $U$ looks like
where $F$ is $V$’s underlying field.
Now let’s say this low-dimensional subspace is invariant under $T$. This would mean that $T(u)$ lands back in $U$, and given that $U$ is of dimension one, $T$ must send $u$ to a scalar multiple of itself. In other words, there is $\lambda \in F$ such that $T(u) = \lambda u$.
As you might have guessed by now, $\lambda$ is what we call an eigenvalue and $u$ is it’s corresponding eigenvector.
One of the central focuses of linear algebra centers around understanding the relationship between linear transformations (useful abstractions) and their matrices (computational tools). Eigenvalues and eigenvectors (loosely) allow you to write down computationally friendlier matrices corresponding to linear transformations. (The friendliest of these is known as a diagonal matrix – the only nonzero entries are those along the diagonal stretching from the top left to bottom right corners. You can write down a diagonal matrix if and only if you manage to find a basis of eigenvectors.) Eigenvalues and eigenvectors are typically not motivated particularly well. For a while, I trusted my professors that they were important and useful, but I’d really never seen a way in which they arise naturally.
They eigenvalues are presented as the roots of the characteristic polynomial of an operator. What’s the characteristic polynomial? If $A$ is the matrix corresponding to $T$ (huh?), then the characteristic polynomial is given by $p(\lambda) = det(A - I\lambda)$. The roots of $p$ are $T$’s eigenvalues. Wait, but what is that $\det$ symbol? Determinants, you say? What are those? How do I know that $det(A - I\lambda)$ is a polynomial? To know that, you’d need to know how to unpack all of those symbols, which requires understanding what they are… and before you know it, you’re down so far down so many rabbit holes that you stop thinking and just start accepting. A few days later, your professor moves on to matrix diagonalization and in a haze of all of the other things going on in your life, you’ve memorized a totally opaque technique that you’ve applied correctly just enough times to feel like you understand. This, I believe, is one of the sneakiest and most pervasive ways that beautifully intuitive math manages to pass students by.
Conclusion
Don’t fall victim! When you run up against a concept you don’t understand, keep struggling with it; don’t get lulled into indifference because you can compute a correct answer. Ask questions, there is none too small. From experience, I can tell you that there is an elegance waiting for you beyond the struggle. One so deep, fundamental and profound that you’ll be truly glad you stuck around.