Can't write to memory requested with malloc/calloc in x64 Assembly-CodePudding

This is my first question on this platform. I'm trying to modify the pixels of an image file and to copy them to memory requested with calloc. When the code tries to dereference the pointer to the memory requested with calloc at offset 16360 to write, an "access violation writing location" exception is thrown. Sometimes the offset is slightly higher or lower. The amount of memory requested is correct. When I write equivalent code in C with calloc, it works, but not in assembly. I've also tried to request an higher amount of memory in assembly and to raise the heap and stack size in the visual studio settings but nothing works for the assembly code. I also had to set the option /LARGEADDRESSAWARE:NO before I could even build and run the program.

I know that the AVX instruction sets would be better suited for this, but the code would contain slightly more lines so I made it simpler for this question and I'm also not a pro, I did this to practice the AVX instruction set.

Many thanks in advance :)

    const uint8_t* getImagePtr(sf::Image** image, const char* imageFilename, uint64_t* imgSize) {
        sf::Image* img = new sf::Image;
        img->loadFromFile(imageFilename);
        sf::Vector2u sz = img->getSize();
        *imgSize = uint64_t((sz.x * sz.y) * 4u);
        *image = img;
        return img->getPixelsPtr();
    }

EXTRN getImagePtr:PROC
EXTRN calloc:PROC

.data
imagePixelPtr QWORD 0 ; contains address to source array of 8 bit pixels
imageSize QWORD 0 ; contains size in bytes of the image file
image QWORD 0 ; contains pointer to image object
newImageMemory QWORD 0 ; contains address to destination array
imageFilename BYTE "IMAGE.png", 0 ; name of the file

.code
mainasm PROC

sub rsp, 40
mov rcx, OFFSET image
mov rdx, OFFSET imageFilename
mov r8, OFFSET imageSize
call getImagePtr

mov imagePixelPtr, rax

mov rcx, 1
mov rdx, imageSize
call calloc
add rsp, 40

cmp rax, 0
je done

mov newImageMemory, rax

mov rcx, imageSize
xor eax, eax
mov bl, 20

SomeLoop:

mov dl, BYTE PTR [imagePixelPtr   rax]
add dl, bl
mov BYTE PTR [newImageMemory   rax], dl ; exception when dereferencing and writing to offset 16360

inc rax
loop SomeLoop

done:
ret
mainasm ENDP
END

CodePudding user response：

Let's translate this line back into C:

mov BYTE PTR [newImageMemory   rax], dl ;

In C, this is more or less equivalent to:

*((unsigned char *)&newImageMemory   rax) = dl;

Which is clearly not what you want. It's writing to an offset from the location of newImageMemory, and not to an offset from where newImageMemory points to.

You will need to keep newImageMemory in a register if you want to use it as the base address for an offset.

While we're at it, this line is also wrong, for the same reason:

mov dl, BYTE PTR [imagePixelPtr   rax]

It just happens not to crash.