x86 Function Attributes in the GCC documentation says this:

On 32-bit and 64-bit x86 targets, you can use an ABI attribute to indicate which calling convention should be used for a function. The ms_abi attribute tells the compiler to use the Microsoft ABI, while the sysv_abi attribute tells the compiler to use the System V ELF ABI, which is used on GNU/Linux and other systems. The default is to use the Microsoft ABI when targeting Windows. On all other systems, the default is the System V ELF ABI.

But consider this C code:

#include <assert.h>

#ifdef _MSC_VER
#define MS_ABI
#else
#define MS_ABI __attribute__((__ms_abi__))
#endif

typedef struct {
    void *x, *y;
} foo;

static_assert(sizeof(foo) == 8, "foo must be an 8-byte structure");

foo MS_ABI f(void *x, void *y) {
    foo rv;
    rv.x = x;
    rv.y = y;
    return rv;
}

gcc -O2 -m32 compiles it to this:

f:
        mov     eax, DWORD PTR [esp 4]
        mov     edx, DWORD PTR [esp 8]
        mov     DWORD PTR [eax], edx
        mov     edx, DWORD PTR [esp 12]
        mov     DWORD PTR [eax 4], edx
        ret

But cl /O2 compiles it to this:

_x$ = 8                                       ; size = 4
_y$ = 12                                                ; size = 4
_f      PROC                                                ; COMDAT
        mov     eax, DWORD PTR _x$[esp-4]
        mov     edx, DWORD PTR _y$[esp-4]
        ret     0
_f      ENDP

Godbolt link

These are clearly using incompatible calling conventions. Argument Passing and Naming Conventions on MSDN says this:

Return values are also widened to 32 bits and returned in the EAX register, except for 8-byte structures, which are returned in the EDX:EAX register pair. Larger structures are returned in the EAX register as pointers to hidden return structures.

Which means MSVC is correct. So why is GCC using the pointer-to-hidden-return-structure approach even though the return value is an 8-byte structure? Is this a bug in GCC, or am I not allowed to use ms_abi like I think I am?

CodePudding user response：

That seems buggy to me. i386 SysV only returns int64_t in registers, not structs of the same size, but perhaps GCC forgot to take that into account for ms_abi. Same problem with gcc -mabi=ms (docs).

Even -mabi=ms doesn't in general change struct layouts in cases where that differs, or for x86-64 make long a 32-bit type. But your struct does have the same layout in both ABIs, so you'd expect it to get returned the way an MSVC caller wants. But that's not happening.

This is IIRC not the first time I've heard of bugs in __attribute__((ms_abi)) or -mabi=ms. But you are using it correctly; in 64-bit code that would make the difference to which arg-passing registers it looked in.

Clang makes the same asm as GCC, but that's not significant because it warns that the 'ms_abi' calling convention is not supported for this target. This is the asm we'd expect from i386 SysV. (And Godbolt doesn't have clang-cl.)

So thanks for reported it to GCC as bug #105932.

(Funny that when it chooses to use SSE or AVX, it loads both dword stack args separately and shuffles them together, instead of a movq load. I guess that avoids likely store-forwarding stalls if the caller didn't use a single 64-bit store to write the args, though.)