Confusion with perspective matrix.

[nikitablack]

Hi,
Hi, in my project I noticed that a matrix, created with Mat4::perspective_rh (together with Mat4::look_at_rh) produces strange results, namely, it transforms a 3d model from a right-handed to a left-handed coordinate system.

This is the model I created in Blender:

From the picture, you can see, that the arrows create a right-handed system (x - right, y - bottom, z - front (into the screen)).

But this is what happens when I draw the same model with a view and perspective matrices from glam::Mat4:

Maybe it's not clearly seen from the static image, but here x points right, y points down, z points back (from the screen), i.e. the handedness was changed.

The same model rendered in other third-party viewers:

Everything is as expected.

I tried to change the perspective matrix to the one from here:

As you can see, it's right-handed again.

Here are some tests I made with the matrix:

let p = glam::Mat4::perspective_rh(
    90.0f32.to_radians(),
    (1440.0f32) / (900.0f32),
    0.1f32,
    100.0f32,
);

let x = glam::Vec4::new(10.0, 0.0, 10.0, 1.0);
let y = glam::Vec4::new(0.0, 10.0, 10.0, 1.0);
let z = glam::Vec4::new(0.0, 0.0, 20.0, 1.0);

let x = p * x;
let y = p * y;
let z = p * z;
println!("{}", x / x.w); // [-0.625, -0, 1.011011, 1]
println!("{}", y / y.w); // [-0, -1, 1.011011, 1]
println!("{}", z / z.w); // [-0, -0, 1.006006, 1]

I'm working with Vulkan which uses a right-hand coordinate system with x coordinate pointing to the right, y pointing down, and z pointing forward (into the screen). So defining my three points x, y, and z I expect that after the projection and the perspective divide, they will stay on the same axes and keep the direction. But the first thing to note is that after the perspective divide, the homogeneous z coordinate is out of range (it's more than 1.0) despite the fact that initially, the z coordinate of all the three vectors is in the matrix range [0.1 - 100.0] and has the value of 10.0. The second thing is if we look at the NDC positions of these three vectors, we see that x is on the left half, y is on the top half, and z has a lower z-coordinate. If we pretend these would be vectors instead of points, the x is pointing to the left, y is pointing up and z is pointing back (from the screen), i.e. these 3 vectors form a left-hand triplet.

looking at the Mat4::perspective_rh source code I'm assuming that it "works" with a negative z direction as if it was positive. Indeed, if I change my test to use:

let x = glam::Vec4::new(10.0, 0.0, -10.0, 1.0);
let y = glam::Vec4::new(0.0, 10.0, -10.0, 1.0);
let z = glam::Vec4::new(0.0, 0.0, -20.0, 1.0);

(notice the negative z-coordinates) I get the correct output:

let x = p * x;
let y = p * y;
let z = p * z;
println!("{}", x / x.w); // [0.625, 0, 0.99099094, 1]
println!("{}", y / y.w); // [0, 1, 0.99099094, 1]
println!("{}", z / z.w); // [0, 0, 0.995996, 1]

Now x points to the right, y - down, z - forward (into the screen). Everything is as it should be.

But now I want to know what I should do with my models to display them correctly. Should I always scale them over the z coordinate before applying the projection?

[bitshifter]

I can't remember where I came up with that perspective projection matrices, some of them were contributed, but I suspect that they are using the D3D view coordinate system where y is up, x is right and z is forward, where as Vulkan expects y to be down. I think possibly the only differences is the y column needs to be negated, you could try p.y_axis *= -1.0 and see if that works.

Also, glam provides Mat4::project_point3(Vec3) which will perform the perspective divide for you so you don't need to do it manually.

[nikitablack]

Yes, these matrices assume that the axes are directed as you pointed out. Negating the y-column is not enough. Also, Mat4::look_at_rh builds a matrix with D3D basis (+x - right, +y - up, +z - back), which does not work with Vulkan's perspective matrix.
Anyway, if somebody is stuck with the same problem, here is the solution:

// builds a view matrix with +x pointing right, +y pointing down, and +z pointing forward
let look_at = |eye: glam::Vec3, pivot: glam::Vec3, up: glam::Vec3| -> glam::Mat4 {
    let z = pivot - eye;
    let z = z.normalize();

    let x = up.cross(z).normalize();
    let y = z.cross(x);
    glam::Mat4::from_cols(
        glam::Vec4::new(x.x, y.x, z.x, 0.0),
        glam::Vec4::new(x.y, y.y, z.y, 0.0),
        glam::Vec4::new(x.z, y.z, z.z, 0.0),
        glam::Vec4::new(-x.dot(eye), -y.dot(eye), -z.dot(eye), 1.0),
    )
};

// builds a perspective matrix for Vulkan coordinate system.
let perspective_rh = |fovy: f32, aspect: f32, z_near: f32, z_far: f32| -> glam::Mat4 {
    let y_scale = 1.0f32 / (fovy * 0.5f32).tan();
    let x_scale = y_scale / aspect;

    let k = z_far / (z_far - z_near);

    glam::Mat4::from_cols(
        glam::Vec4::new(x_scale, 0.0, 0.0, 0.0),
        glam::Vec4::new(0.0, y_scale, 0.0, 0.0),
        glam::Vec4::new(0.0, 0.0, k, 1.0),
        glam::Vec4::new(0.0, 0.0, -z_near * k, 0.0),
    )
};

[bitshifter]

I do have an issue to try and address this sort of thing and provide methods that work for any (or any common) coordinate system - #79. I don't have any solutions as yet though.