So I'm trying to understand how assembly programming works with stack frames etc.
I did some exercises and disassembled some C-code with GDB. The task now is to find out how the transfer of parameters between the 'main' and functions works. I just started learning and kinda got lost on what next example is actually doing. Any ideas or tips on where to get started?

It's a recursive program working with faculty.

The assembly code looks like this:

    1149:   f3 0f 1e fa             endbr64 
    114d:   55                      push   rbp
    114e:   48 89 e5                mov    rbp,rsp
    1151:   48 83 ec 10             sub    rsp,0x10
    1155:   89 7d fc                mov    DWORD PTR [rbp-0x4],edi
    1158:   83 7d fc 01             cmp    DWORD PTR [rbp-0x4],0x1
    115c:   76 13                   jbe    1171 <f 0x28>
    115e:   8b 45 fc                mov    eax,DWORD PTR [rbp-0x4]
    1161:   83 e8 01                sub    eax,0x1
    1164:   89 c7                   mov    edi,eax
    1166:   e8 de ff ff ff          call   1149 <f>
    116b:   0f af 45 fc             imul   eax,DWORD PTR [rbp-0x4]
    116f:   eb 05                   jmp    1176 <f 0x2d>
    1171:   b8 01 00 00 00          mov    eax,0x1
    1176:   c9                      leave
    1177:   c3                      ret
    1178:   f3 0f 1e fa             endbr64
    117c:   55                      push   rbp
    117d:   48 89 e5                mov    rbp,rsp
    1180:   48 83 ec 10             sub    rsp,0x10
    1184:   c7 45 f8 05 00 00 00    mov    DWORD PTR [rbp-0x8],0x5
    118b:   c7 45 fc 00 00 00 00    mov    DWORD PTR [rbp-0x4],0x0
    1192:   8b 45 f8                mov    eax,DWORD PTR [rbp-0x8]
    1195:   89 c7                   mov    edi,eax
    1197:   e8 ad ff ff ff          call   1149 <f>
    119c:   89 45 fc                mov    DWORD PTR [rbp-0x4],eax

CodePudding user response：

Study the calling convention for your environment.  An overview of the many calling conventions for a number of architectures: https://www.dyncall.org/docs/manual/manualse11.html

The calling convention specifies:

1. Where parameters and return values must appear at the one single point of transfer of control of the instruction stream from the caller to the callee.  For parameters being passed, that single point is after the call is made and before the first instruction of the callee (and for return values, at the point where the callee finishes and just before execution resumes in the caller).

   Many conventions combine parameter passing in CPU registers with stack memory for parameters that don't fit in CPU registers.  And even some that don't use CPU registers for parameters still use CPU registers for return values.

2. What registers a function is allowed to clobber vs. must preserve.  Call-clobbered registers can be assigned new values without concern.  Call-preserved registers can be used but must be restored to the value they had upon entry before returning to the caller.  The advantage of call-preserved registers is that since they are preserved by a call, you can use them for variables that need to survive another call.

3. The meaning & treatment of the stack pointer, regarding memory below and above the current pointer, and alignment requirements for stack allocation.

If the function allocates stack space in some manner, then memory parameters will appear to move farther away from the top of the stack (they don't actually move, of course, but become larger offsets from the current stack or frame pointer).  Compilers know this and adjust their access to stack memory accordingly.

Some compilers set up frame pointers to refer to stack memory.  A frame pointer is a copy of the stack pointer made at some point in the prologue.  Frame pointers are not always necessary but facilitate exception handling and stack unwinding, as well as dynamic stack allocation.