Hooking Win API Call MessageBoxW-CodePudding

I'm trying to jump hook the Windows function MessageBoxW and get my own function to run instead. However, I'm running into an "Access violation writing location" at this line:

*(reinterpret_cast <std::uint32_t*>(reinterpret_cast<std::uint32_t>(hook_addr)   1)) = rel_addr;

Here's my program code:

#include "Windows.h"
#include <cstdint>
#include <cstring>
#include <iostream>
#include <string> 
using namespace std;

int hookedFunc(void* thisptr, void* not_edx, HWND hWnd, LPCSTR lpText, LPCWSTR lpCaption, UINT uType) {
    cout << "Hooked function called";
    return MessageBoxW(NULL, L"HOOKED BOX", L"HOOKED CAPTION", MB_YESNOCANCEL);
}

std::uint32_t tramp_hook(void* hook_addr, void* new_func, std::uint32_t instr_size)

{

    constexpr auto jmp_instr_size = 5; // minimum size required for a JMP instruction



    DWORD vp_old_prot{ 0u };

    VirtualProtect(hook_addr, instr_size, PAGE_EXECUTE_READWRITE, &vp_old_prot); // In order to overwrite code, we need proper access.

    std::memset(hook_addr, 0x90, instr_size); // setting the bytes we will overwrite to 0x90 / nop ( no operation )



    const auto rel_addr = (reinterpret_cast<std::uint32_t>(new_func) - reinterpret_cast<std::uint32_t>(hook_addr)) - jmp_instr_size; // relative address calculation. in a relative jmp / call, the result of this calculation will be used by the CPU to add to the EIP ( instruction pointer ).



    *(static_cast<std::uint8_t*>(hook_addr)) = 0xE9; // overwriting a byte to the JMP opcode ( 0xE9 )

    // problem
    *(reinterpret_cast <std::uint32_t*>(reinterpret_cast<std::uint32_t>(hook_addr)   1)) = rel_addr; // setting the result of our relative address calculation as the relative jmp operand.



    VirtualProtect(hook_addr, instr_size, vp_old_prot, nullptr); // resetting our access

    return reinterpret_cast<std::uint32_t>(hook_addr)   5; // returning the address next instruction to be executed to be used later

}

typedef int (__fastcall* MESSAGEBOXW)(void* thisptr, void* not_edx, HWND hWnd, LPCSTR lpText, LPCSTR lpCaption, UINT uType);


MESSAGEBOXW hookedBox = reinterpret_cast<MESSAGEBOXW>(&hookedFunc);

int main() {
    MessageBoxW(NULL, L"not hooked", L"this is the caption", MB_YESNO);

    HMODULE dllHandl = LoadLibrary(L"user32.dll");
    void* msgBoxAddr = GetProcAddress(dllHandl, "MessageBoxW");

    cout << hookedBox << endl;

    tramp_hook(msgBoxAddr, hookedBox, 5);

    MessageBoxW(NULL, L"not hooked 2", L"this is the caption", MB_YESNO);
}

Could someone shed some light on what's happening? I'm able to set the nop bytes and write the JMP code but the access violation happens when the relative address calculation is being set.

I have already checked the return value of the first VirtualProtect; it's not nonzero so everything is good there.

CodePudding user response：

My crystal ball says you are compiling your program as 64-bit, so when you force the pointer into a std::uint32_t it cuts off the top 32 bits of the address and now you have an invalid pointer.

When treating a pointer as a number you should use std::uintptr_t, which is an unsigned integer the same size as whatever size a pointer is.

Writing the JMP opcode works because you don't cast the pointer to std::uint32_t in that case.