Rotations and Infinitesimal Generators

In this post I’d like to talk about rotations in three-dimensional space. As you can imagine, rotations are pretty important transformations! They commonly show up in physics as an example of a symmetry transformation, and they have practical applications in fields like computer graphics.

Like any linear transformation, rotations can be represented by matrices, and this provides the best representation for actually computing with vectors, transformations, and suchlike. Various formulas for rotation matrices are well-known and can be found in untold books, papers, and websites, but how do you actually derive these formulas?

If you know a little trigonometry, you can work out the 2D rotation matrix formula by drawing a diagram like this:

The rotation takes the vector (1, 0) to (\cos \theta, \sin \theta) and the vector (0, 1) to (-\sin \theta, \cos \theta). This is just what we need, since in a matrix the first column is just the output when you put in a unit vector along the x-axis; the second column is the output for a unit vector along the y-axis, and so on. So the 2D rotation matrix is:

R_{2D}(\theta) = \left[ \begin{matrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{matrix} \right].

You can expand this 2×2 matrix into a 3×3 one and rearrange a few things to get rotations around the x, y, and z axes. That’s all well and good, but what if you want a rotation around some other axis, one that’s not lined up with x, y, or z? Trying to figure out that formula directly with trigonometry is probably hopeless!

One way to do it is to combine rotations together to achieve the desired result. We could do something like: work out the polar coordinates of the axis we want to rotate around, then apply some rotations around the y and z axes that take that axis onto the x-axis. Then do an x-rotation by your desired angle, and then revert our y and z rotations to get the axis back to where it was.

That will work, but it’s inelegant. Since this is a blog about theoretical stuff, I’m going to talk about a more elegant way to do it, using a thing called an infinitesimal generator!

The basic idea of this is to first work out the formula for an infinitesimal rotation – a rotation by a very small angle. What’s the advantage of this? Everything’s simpler in infinitesimals, because you’re allowed to simply ignore things that are “too small to matter”. Infinitesimal math lets you make fantastically crude approximations and not worry about it because eventually, you’re going to take the limit as the infinitesimal thing goes to zero, and in that limit your approximation becomes exact! In this case (and I’m giving away a bit of the story here), once we know the formula for an infinitesimal rotation, we’ll integrate it to get the formula for a finite rotation.

Let’s return to the diagram above, but now consider a rotation by an infinitesimal angle d\theta. Imagine taking the Cartesian coordinate system and just barely starting to turn it counter-clockwise. What happens to the point (1, 0)? It goes straight up! Eventually, if you keep rotating, it’ll start to curve over to the left…but since this is just an infinitesimal rotation, it doesn’t have time to do that.

Surely, you say, it doesn’t go straight up? It must go a little to the left, mustn’t it? Yes, it does…but this is one of those “too small to matter” things that you can just ignore! The smaller d\theta is, the closer to straight-up the point goes, and that justifies treating it as if it just goes straight up all the time.

An analogous thought process about the point (0, 1) reveals that it goes straight to the left, neither up nor down (within our approximation). Here’s the diagram again, modified with these observations:

As before, the effect of the rotation on these two points is all we need to get the matrix for an infinitesimal rotation:

r_{2D}(d\theta) = \left[ \begin{matrix} 1 & -d\theta \\ d\theta & 1 \end{matrix} \right].

(I’m using lowercase letters, e.g. r_{2D}(d\theta) for infinitesimal transformations, and uppercase letters like R_{2D}(\theta) for finite ones.) Anyway, this looks a lot simpler than the first matrix we found, and we didn’t need any trigonometry to work it out!

Conceptually, what we need to do next is accumulate the effect of many successive infinitesimal rotations, to get a finite rotation. Technically, this is going to be accomplished by integration. But before we can integrate anything, we need a differential equation, so we have to fiddle with the formula a bit to get it into that shape. Let’s start with a vector \vec u and apply the infinitesimal rotation to it; that’ll cause \vec u to change by some small vector d\vec u:

\begin{aligned} \vec u + d\vec u &= r_{2D}(d\theta) \vec u \\ d\vec u &= \bigl[ r_{2D}(d\theta) - I \bigr] \vec u \\ \dfrac{d\vec u}{d\theta} &= \left[ \begin{matrix} 0 & -1 \\ 1 & 0 \end{matrix} \right] \vec u \equiv G_{2D} \vec u. \end{aligned}

(In the second line there, I is the identity matrix.) Now we have a differential equation that tells us how \vec u changes with \theta for very small values of \theta. The matrix in the last line, which I’m naming G_{2D}, is the “infinitesimal generator” of 2D rotation! It’s a matrix that converts a vector into its derivative with respect to infinitesimal rotations.

The Exponential Map

At this point, we need to take a short detour into the world of differential equations. We have a differential equation in which the derivative of a vector, \vec u, is equal to a matrix multiplied by \vec u itself. This looks very similar to a famous differential equation about real numbers:

\dfrac{dy}{dx} = ky,

whose solution is the exponential function, y(x) = y_0 e^{kx} (where y_0 is a constant of integration, which equals y(0)). Working purely by analogy, we might hypothesize that the solution to our original differential equation is:

\vec u (\theta) = R_{2D}(\theta) \vec u_0 = e^{G_{2D} \theta} \vec u_0.

Now, instead of the ordinary exponential function on real numbers, we have some kind of exponential of a matrix, where e “raised to the power of” a matrix gives us back another matrix. Does this even make sense? It turns out that it does! This “matrix exponential” is a mathematical entity called the exponential map, a generalization of the exponential function to a broad range of contexts. It obeys many of the same rules as the familiar exponential function, so we can prove that it really does solve our differential equation!

Now we just need to calculate it. One way of defining the ordinary exponential function is by its Taylor series:

e^x = 1 + x + \dfrac{x^2}{2!} + \dfrac{x^3}{3!} + \cdots

The same Taylor series can be straightforwardly translated to one for the exponential map on matrices! This implies that:

R_{2D} (\theta) = I + G_{2D} \theta + G_{2D}^2 \dfrac{\theta^2}{2!} + G_{2D}^3 \dfrac{ \theta^3}{3!} + \cdots

We’ve made some progress. Instead of a differential equation, we now have an infinite series for explicitly calculating a rotation matrix for any angle! We can almost split this into the four individual matrix elements and sum up each series individually – but first we need to see what the integer powers of G_{2D} look like.

Let’s just calculate the first few powers and see what happens:

\begin{aligned} G_{2D} &= \left[ \begin{matrix} 0 & -1 \\ 1 & 0 \end{matrix} \right] & G_{2D}^2 &= \left[ \begin{matrix} -1 & 0 \\ 0 & -1 \end{matrix} \right] \\ G_{2D}^3 &= \left[ \begin{matrix} 0 & 1 \\ -1 & 0 \end{matrix} \right] & G_{2D}^4 &= \left[ \begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix} \right] \\ G_{2D}^5 &= \left[ \begin{matrix} 0 & -1 \\ 1 & 0 \end{matrix} \right] \end{aligned}

Woah! That’s interesting – the fifth power of G_{2D} is the same as G_{2D} itself. This means that if we kept going, we’d just generate the same four matrices over and over again. Note also that G_{2D}^3 = -G_{2D}, and G_{2D}^4 = -G_{2D}^2. Not only do the matrices repeat every four powers, they repeat with a sign change every two powers.

This is very similar to the behavior of a couple of other famous Taylor series!

\begin{aligned} \sin x &= x - \dfrac{x^3}{3!} + \dfrac{x^5}{5!} - \cdots \\ \cos x &= 1 - \dfrac{x^2}{2!} + \dfrac{x^4}{4!} - \cdots \end{aligned}

The Taylor series of the sine and cosine functions are the same as that of the exponential function…except that they only include every two powers, and alternate with a sign change each term, with sine taking the odd powers and cosine taking the even powers. (This is the basis of Euler’s formula.)

If we regroup the terms in our series for R_{2D}(\theta) into odd and even, and use the fact that the powers of G_{2D} repeat with sign changes, we get:

\begin{aligned} R_{2D}(\theta) &= \left(I + G_{2D}^2 \dfrac{\theta^2}{2!} + G_{2D}^4 \dfrac{\theta^4}{4!} + \cdots \right) \\ &\qquad + \left(G_{2D} \theta + G_{2D}^3 \dfrac{\theta^3}{3!} + G_{2D}^5 \dfrac{\theta^5}{5!} + \cdots \right) \\ &= -G_{2D}^2 \left(1 - \dfrac{\theta^2}{2!} + \dfrac{\theta^4}{4!} - \cdots \right) \\ &\qquad + G_{2D} \left(\theta - \dfrac{\theta^3}{3!} + \dfrac{\theta^5}{5!} - \cdots \right) \\ &= -G_{2D}^2 \cos \theta + G_{2D} \sin \theta \\ &= \left[ \begin{matrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{matrix} \right]. \end{aligned}

This final matrix is the same as in the very first equation in this article, but now we’ve gotten there through a completely different route – we first worked out the generator matrix for an infinitesimal rotation, then we calculated the exponential of that matrix (multiplied by \theta) to get the matrix for a finite rotation!

Axis-Angle Rotations in 3D

We can apply the same approach to solve the problem I posed near the top of this article: to find the matrix of a rotation about an arbitrary axis. Let’s identify this axis by \vec a, a unit vector pointing along the axis. Our goal is to find an expression for R_a(\theta), the 3×3 matrix that rotates around \vec a by an angle \theta. As before, we’ll begin by considering an infinitesimal rotation, and working out the generator G_a.

Let’s consider the action of a rotation around \vec a by an infinitesimal angle d\theta on an arbitrary 3D vector \vec u. What will happen to \vec u? In the infinitesimal approximation, it moves in a direction straight out of the plane containing \vec a and \vec u, as shown here:

We can express this using a vector operation called the cross product, which takes two vectors and gives you back a vector perpendicular to both of them:

\dfrac{d\vec u}{d\theta} = \vec a \times \vec u.

(Note that the length of the cross product vector is related to the lengths of the two vectors being crossed. I leave it as an exercise for the reader to show that the length of d\vec u is correct in the above equation!)

The cross product can also be expressed as a matrix. In this case, we need a matrix that, when multiplied by \vec u, gives us \vec a \times \vec u. This will be G_a, the infinitesimal generator for rotations around \vec a.

\vec a \times \vec u = \left[ \begin{matrix} 0 & -a_z & a_y \\ a_z & 0 & -a_x \\ -a_y & a_x & 0 \end{matrix} \right] \vec u = G_a \vec u

It follows that R_a(\theta) = e^{G_a \theta}, i.e. the rotation matrix we’re trying to find is the exponential of \theta times the matrix G_a above! If you work out the first few powers of G_a and use the fact that \vec a is a unit vector (so a_x^2 + a_y^2 + a_z^2 = 1), you’ll see that it follows the same pattern that the powers of G_{2D} did: they repeat every four powers, and repeat with sign alternation every two, so that G_a^3 = -G_a and G_a^4 = -G_a^2. We can almost use the same odd/even term grouping as before to split the series into a part proportional to \cos \theta, and a part proportional to \sin \theta. However, there is one little bug to deal with. In the 2D version of this derivation, we implicitly used the fact that G_{2D}^4 = I, the identity matrix, when factoring -G_{2D}^2 out of the even terms. Here, G_a^4 \neq I, so we have to do a little bit of a fixup.

\begin{aligned} R_a(\theta) &= \left(I + G_a^2 \dfrac{\theta^2}{2!} + G_a^4 \dfrac{\theta^4}{4!} + \cdots \right) \\ &\qquad + \left(G_a \theta + G_a^3 \dfrac{\theta^3}{3!} + G_a^5 \dfrac{\theta^5}{5!} + \cdots \right) \\ &= I - G_a^2 \left(- \dfrac{\theta^2}{2!} + \dfrac{\theta^4}{4!} - \cdots \right) \\ &\qquad + G_a \left(\theta - \dfrac{\theta^3}{3!} + \dfrac{\theta^5}{5!} - \cdots \right) \\ &= I - G_a^2 (\cos \theta - 1) + G_a \sin \theta \\ &= I + G_a^2 (1 - \cos \theta) + G_a \sin \theta \\ \end{aligned}

Finally, we can put everything together to get our final rotation matrix (it’s a wide formula, so click to zoom):

And there it is! Whew! That was a lot of work, but now we have an explicit matrix for arbitrary rotations around any axis in 3D space, and along the way we met a bunch of interesting concepts, such as infinitesimal math, the exponential map, and Taylor series.

Rotations are truly a fascinating set of transformations, and in this post I’ve only just scratched the surface of the interesting structure that they have. Rotations in any number of dimensions form a mathematical entity called a Lie group, and the fact that they can be produced by infinitesimal generators is intimately linked with the special properties of Lie groups! But that’s a topic for a future post.

About these ads

About Nathan Reed

I'm a graphics programmer, an amateur physicist, and a sci-fi nerd. I teach computers how to make pretty pictures. I'm excited by beautiful, immersive, story-driven games and interactive fiction. I enjoy messing around with esoteric ideas. I like explaining things. I currently work for NVIDIA DevTech. Previously, I worked for Sucker Punch Productions on the Infamous series of games for PS3 and PS4. reedbeta.com - developer blog, OpenGL demos, and other projects. @reedbeta on Twitter.
This entry was posted in Mathematics. Bookmark the permalink.

5 Responses to Rotations and Infinitesimal Generators

  1. Troy Winfree says:

    This is a very thorough explanation. I’d like to point out that, as long as I am not thinking about this incorrectly, it is much easier to derive the rotation matrix formula with the following three geometric steps: (1) project into the plane perpendicular to the axis of rotation; (2) perform a planar rotation; (3) “project out” of the plane perpendicular to the axis of rotation. This produces the following formula:

    Given an axis v, angle theta and position vector x the new rotated position is:

    cos(theta)(x-(v dot x)v) + sin(theta)(v cross x) + (v dot x) v

    where we are assuming that v is normal. Note that the vector (x-(v dot x)v) is x projected into the plane perpendicular to v, which together with the vector (v cross x) forms an orthogonal basis for this plane. Since v is normal, (x-(v dot x)v) and (v cross x) have the same magnitude. So the first two summands in the above formula express the projection of x into the plane perpendicular to v and rotated within that plane by theta. The final summand “projects out” by shifting the rotated position along v by the height of x above the plane of rotation.

    The columns of the rotation matrix are produced by applying the above formula to the standard basis vectors.

    If the above is correct it seems like it should be the “standard” derivation, though I can’t seem to find it anywhere on the internet.

  2. Nathan Reed says:

    Troy, I think your formula is the same or similar to the Rodrigues rotation formula. Yes, that is an easy way to derive it. :) I do like the infinitesimal-generators approach, though, as it’s readily generalizable to other sorts of transformations beyond rotations.

  3. Troy Winfree says:

    Cool. Thanks for pointing out Rodrigues’ formula. Looks like it is identical to the one I give above.

  4. iguananaut says:

    Hey, just wanted to say thanks for this detailed explanation. It really made sense of infinitesimal generators for me in a way that most textbooks, unfortunately, haven’t. In large part it helps to clarify “don’t worry that it’s not exact; we’ll fix that up later”. A lot of texts don’t clarify that and have left me scratching my head at parts over whether this is really okay. Makes sense now though.

  5. Dhiman Bhowmick says:

    very fine explanation.I read some books but I couldn’t understand.
    THANK YOU

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s