Where can I find what data to pass to Windows API functions-CodePudding

From what I understand, the best way to get stuff done in x86-64 assembly is not to program stuff yourself. Instead, get windows to do it for you. Like opening a new console or printing text to a console window. But, you can't call windows system calls directly, as these are subject to change at a snap of Microsoft's fingers. So, you call the Windows API via call commands.

So, I want to be able to program full applications in x86-64 assembly on Windows 10 but am struggling to find information about the Windows API. Specifically, what information/data/numbers to pass into RCX, RDX, R8, and R9 before calling a windows API function? For example, a Hello World program I found somewhere on the internet uses both "GetStdHandle" and "WriteConsoleA". I understand what these do, but I don't know what all the numbers and register manipulation mean or how it affects the windows API. Here's the code:

extern GetStdHandle
extern WriteConsoleA

section .data
    msg: db "HelloWorld"
    msglen: equ $-msg

section .bss

section .text
start:

    mov ECX, -11
    call GetStdHandle

    mov RCX, RAX
    mov RDX, msg
    mov R8, msglen
    lea R9, [RSP-16]
    call WriteConsoleA

    mov EAX, 0

    ret

This was compiled with NASM and linked with Golink. Here's the make file:

{NasmPath} -f win64 HelloWorld.asm
{GoLinkPath} HelloWorld.obj /console kernel32.dll

For clarification, I did not run {NasmPath} or {GoLinkPath} through the console, I just redacted it because the actual path has my name in it.

TLDR: Where can I find information on what data to pass through registers RCX, RDX, R8, and R9 to Windows API Functions?

CodePudding user response：

Windows x64 calling conventions are documented on Microsoft Docs.

The call to GetStdHandle is really simple: It's passing STD_OUTPUT_HANDLE (-11) as the first parameter which uses ECX; return value is in RAX.

The call to WriteConsoleA is a bit trickier. Technically the function takes 5 parameters with the last one being reserved, but the code is only passing four.

Keep in mind that writing fully 'correct' Windows x64 assembly is pretty tricky because of the required exception unwinding information. There are some MASM macros, but don't know about NASM.