glm::rotate() changes the rotation axis, but why?-CodePudding

Easy test code

glm::mat4 m = glm::rotate(glm::mat4(1.0f), 0.78f, glm::vec3(0,1,0));

while (true) {
    glm::vec3 axis = glm::normalize(glm::vec3(m[0]));  // right vector
    PRINT_VEC("{0:.3f} {1:.3f} {2:.3f}", axis.x, axis.y, axis.z);
    m = glm::rotate(m, glm::radians(5.f), axis);  // 5 degrees each iteration
}

So, assume that I have a model matrix rotated from identity by 0.78 radians around the y axis, then every frame I'm going to rotate around the local right vector, which is the first column of the matrix (assume a right-handed system). Since the right vector is the axis around which I rotate, I expect it to be constant, but that's not true. I don't understand why glm::rotate also changes the rotation axis.

The output changes quite drastically so I don't think it's floating point precision errors.

0.657 -0.424 -0.623
0.643 -0.482 -0.595
0.626 -0.539 -0.563
0.608 -0.593 -0.527
0.588 -0.646 -0.487
0.566 -0.696 -0.442
0.543 -0.742 -0.393
0.518 -0.785 -0.339
0.491 -0.824 -0.281
0.464 -0.858 -0.219
0.435 -0.887 -0.153
0.406 -0.910 -0.084
0.377 -0.926 -0.012
0.347 -0.936 0.063
0.319 -0.937 0.140
0.292 -0.931 0.218
0.267 -0.917 0.296
0.244 -0.895 0.374
0.224 -0.864 0.450
0.208 -0.826 0.524

CodePudding user response：

m[0] isn't truly the 'local right vector'. The local right vector is vec3(1,0,0), which is what you should use to achieve the desired rotation:

glm::mat4 m = glm::rotate(glm::mat4(1.0f), 0.78f, glm::vec3(0,1,0));

while (true) {
    glm::vec3 axis = glm::normalize(glm::vec3(m[0]));  // right vector
    PRINT_VEC("{0:.3f} {1:.3f} {2:.3f}", axis.x, axis.y, axis.z);
    m = glm::rotate(m, glm::radians(5.f), glm::vec3(1,0,0));  // 5 degrees each iteration
}

Prints:

0.711 0.000 -0.703
0.711 0.000 -0.703
0.711 0.000 -0.703
...

What your code does instead, is to transform the vector vec3(1,0,0) into the world coordinate system, and then applies a local rotation around that numeric vector. Note that GLM transformation functions, like glm::rotate, apply the transformation on the right; i.e.

m = m * glm::rotate(glm::mat4(1.0f), glm::radians(5.f), axis);

Therefore the space rotation is applied (local) ends up different from the space you calculated your vector in (world). Consequently, when interpreted in the local coordinate system, axis is just some arbitrary vector.

Alternatively, you can use the world vector axis to rotate in the world space around that same vector; you do that by multiplying on the left:

m = glm::rotate(glm::mat4(1.0f), glm::radians(5.f), axis) * m;

which gives the same result as the first version above.

EDIT: Column vectors rendered using the correct vs incorrect code: