So, one of my pet projects is a game engine based on a 3D Software Renderer (that I will post about some other day). One night, while I was working on it implementing an FPS camera, I started getting strange results when moving the mouse.

My development setup for this kind of stuff consisted on a VM running Fedora XFCE Spin, where I would code using CLion, build and execute the program. All was working fine until I got to the mouse related code.

For the approach I was taking, we needed to set the mouse to the Relative Mouse mode, by using the SDL_SetRelativeMouseMode() function. According to the SDL Wiki, it works like this:

While the mouse is in relative mode, the cursor is hidden, the mouse position is constrained to the window, and SDL will report continuous relative mouse motion even if the mouse is at the edge of the window.

When used together with the SDL_GetRelativeMouseState() function, I should have the mouse deltas since the last time the input was read (once per frame). With those values, I could rotate my camera accordingly.

Ok, so it seems that this is exactly what I needed: the mouse should not exit the window and it will continue to report motion even when at the edge of it. Let’s see how I implemented that on the code.

void init_display()
{
    // Window creation code
    // ...

    if (SDL_SetRelativeMouseMode(SDL_TRUE) != 0)
    {
        // Log the error and exit the program
        return;
    }

    // ...
}

void process_input()
{
    SDL_PumpEvents();
    
    // Keyboard state reading code
    // ...
    
    int32_t x_offset, y_offset;
    SDL_GetRelativeMouseState(&x_offset, &y_offset); // Ignoring the button state for now
    // Do stuff with the relative mouse motion values set by the function
    // ...
}

But then I got into troubles. When running this code for the first time, with a little touch on the mouse, the camera went crazy. It seemed to me that the sensitivity of the mouse motion was too high; I then proceeded to multiply those two variables by a scaling factor in an attempt to correct that - it didn’t solve the issue, but made the movement slower, which allowed me to discover another problem: when the mouse cursor seemed to reach the border of the window, the movement stopped completely. Well, there was something clearly wrong.

I was not sure if the camera rotation code was correct, so I hacked a simple test using keyboard input and it worked flawlessly. Ok, then I was sure that the values I was getting from the mouse state were wrong somehow. To verify that, I put a simple debug printf on the process_input() function:

void process_input()
{
    // ...
    
    int32_t x_offset, y_offset;
    SDL_GetRelativeMouseState(&x_offset, &y_offset); // Ignoring the button state for now
    printf("Mouse motion: X: %d Y: %d\n", x_offset, y_offset);
    
    // ...
}

Which returned me values such as:

Mouse motion: X: -34 Y: 142
Mouse motion: X: -34 Y: 214
Mouse motion: X: 0 Y: 0
Mouse motion: X: 0 Y: 142
Mouse motion: X: -34 Y: 214
Mouse motion: X: 0 Y: 71
Mouse motion: X: 0 Y: 0
Mouse motion: X: -34 Y: 71
Mouse motion: X: 0 Y: 0
Mouse motion: X: 0 Y: 72
Mouse motion: X: 0 Y: 0
Mouse motion: X: 0 Y: 0
Mouse motion: X: 0 Y: 0
Mouse motion: X: 0 Y: 0

Given that the camera_rotate() function that I was using expect values in degrees, those values were really greater than what it should be for it to work properly. Moreover, those X:0 Y:0 lines where printed when the mouse was on the edge of the screen and I was still moving it.

With a quick search on Google, I managed to find an issue on the SDL GitHub that seemed related to the problems I was having. There was an statement from Sam Lantinga (the SDL creator) on it, which said:

You also can’t reliably test relative mode in a VM, which is one of the reasons I haven’t gotten to it yet. :)

Hm, so could the problem be the virtual machine? I didn’t have any Linux box on hand to try the code, so I ended up porting the source and built it on the Windows host. After solving some issues related to the porting process, I ran it. And it worked perfectly, no issues whatsoever. Here is the output of that printf() debug line:

Mouse motion: X: -2 Y: 1
Mouse motion: X: -3 Y: 2
Mouse motion: X: -3 Y: 1
Mouse motion: X: -2 Y: 1
Mouse motion: X: -3 Y: 2
Mouse motion: X: -3 Y: 2
Mouse motion: X: -2 Y: 2
Mouse motion: X: -2 Y: 2
Mouse motion: X: -2 Y: 2
Mouse motion: X: -2 Y: 4
Mouse motion: X: -2 Y: 4
Mouse motion: X: -1 Y: 2

Notice how smaller the values are. It works just fine with the camera_rotate() function that I made, and the only times I got the X:0 Y:0 results were when the mouse was completely still.

Some days later, having some time away from work, I built a Linux box with some pretty old parts I had laying around, and installed the same version of Fedora and the tools I was using, and built the source code. Just like the Windows machine, absolutely no errors; this made me pretty confident that the problem was indeed with the virtual machine.

On that GitHub issue that I talked about, there was a workaround that seemed to work for someone, but I had no luck on getting it to work. It consisted of setting an SDL Hint, like this:

void init_display()
{
    // Window creation code
    // ...
    
    SDL_SetHintWithPriority(SDL_HINT_MOUSE_RELATIVE_MODE_WARP, "1", SDL_HINT_OVERRIDE); // the workaround
    if (SDL_SetRelativeMouseMode(SDL_TRUE) != 0)
    {
        // Log the error and exit the program
        return;
    }

    // ...
}

When I tried it, it didn’t change the behavior of the program at all. For the time, I just gave up on getting this to work on the VM, and moved the development to the Windows host. Maybe someday I will try to fix this again, but, given that it works on real hardware running Linux, I’m not that concerned.

If anyone has any suggestions to this problem, I would love to know.