KeyboardHook catching keys destroys the Keyboard state-CodePudding

I want to have a keyboard hook that catches all the input from the user and translates it to unicode characters. With the following code this works, but only if the i call the next hooks. Otherwise letters get printed small all the time although pressing shift for example.

I have looked into the keyboard state and it seems that it doesnt get updated if i "eat" the keystrokes.

LRESULT CALLBACK LowLevelKeyboardProc(int nCode, WPARAM wParam, LPARAM lParam) {
    if(nCode < 0) return CallNextHookEx(hook, nCode, wParam, lParam);

    KBDLLHOOKSTRUCT* keystruct = (KBDLLHOOKSTRUCT*)lParam;

    WCHAR uc[5] = { 0 };
    BYTE kb[256];
    GetKeyboardState(kb);
    
    if (wParam == WM_SYSKEYDOWN || wParam == WM_KEYDOWN) {
        int res = ToUnicode(keystruct->vkCode, keystruct->scanCode, kb, uc, 4, 0);
        switch (res) {
            case -1:
                // code for handling dead keys has been removed for simplicity
                break;
            case  0:
                break;
            case  1:
            case  2:
            case  3:
            case  4: {
                wcout << uc;
            }
        }
    }

    // eat keystroke
    return 1;

    // dont eat keystroke
    return CallNextHookEx(hook, nCode, wParam, lParam);
}

I dont know if there is a right way to "eat" the keystrokes and still have the keyboard state update accordingly.

I even tried updating the keyboard state myself before eating the keystroke and returning 1:

#define BIT_SET(a,b) ((a) |= (1ULL<<(b)))
#define BIT_CLEAR(a,b) ((a) &= ~(1ULL<<(b)))
#define BIT_FLIP(a,b) ((a) ^= (1ULL<<(b)))
#define BIT_CHECK(a,b) (!!((a) & (1ULL<<(b))))

// if key down
if (wParam == WM_KEYDOWN || wParam == WM_SYSKEYDOWN) {
    // and the key was not down previously
    if (!BIT_CHECK(kb[keystruct->vkCode], 7)) {
        // switch the toggled state by setting the low-order bit (https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getkeystate)
        if (BIT_CHECK(kb[keystruct->vkCode], 0)) BIT_CLEAR(kb[keystruct->vkCode], 0);
        else BIT_SET(kb[keystruct->vkCode], 0);
    }
    
    // mark key as down by setting the high-order bit
    BIT_SET(kb[keystruct->vkCode], 7);
}
else {
    // else clear the bit so mark as not down
    BIT_CLEAR(kb[keystruct->vkCode], 7);
}

// update the keyboard state manually
SetKeyboardState(kb);

This still gave the same result.

Is there some way to "eat" the keystrokes and still keeping the keyboard state up to date / having ToUnicode still work as expected?

CodePudding user response：

So it turns out that setting the keyboard state and adjusting it manually works. The only problem was that ToUnicode ignores LSHIFT, RSHIFT, LCONTROL, RCONTROL and so on. So i just mapped those keys to their equivalent SHIFT, CONTROL, MENU and everything works fine now.

CodePudding user response：

According to the Doc: ToUnicodeEx function

You should provide a pointer to a 256-byte array that contains the current keyboard state (const BYTE *lpKeyState). Each element (byte) in the array contains the state of one key. If the high-order bit of a byte is set, the key is down.

To retrieve status information for an individual key, use the GetKeyState function.

As far as I'm concerned you could try to add an extra index to the buffer.

BYTE state[256] = { 0 };
GetKeyState(VK_SHIFT);
GetKeyState(VK_CONTROL);
GetKeyState(VK_MENU);
GetKeyboardState(kb);