Freeing a buffer in C that was created in a function-CodePudding

I'm having some problems trying to free a buffer created in a function from outside that function in C. What I'm doing is to create bufA, do some stuff with it, then call a function which creates bufB, whose length is calculated inside that function. Bytes in bufA are then copied to bufB with some additional bytes added, then the function returns. Some further processing is done with bufB back in main(), then both bufA and bufB are freed. The program runs correctly, but throws an error when free(bufB) is called.

For very much simplified code, I have something like this:

typedef unsigned char BYTE;

// -----

int main(void) {

  // -----

  BYTE bufA = NULL;
  // Specify lenBufA
  bufA = calloc(lenBufA, 1);
  // Populate bufA with some stuff.
  BYTE bufB = NULL;
  int lenBufB = myFunction(bufA, &bufB);
  // Do something with bufB.
  free(bufA);
  free(bufB); // <=== Throws an error here.
  return 0;
}

Then in myFunction() I may have something like this:

int myFunction(BYTE* inBuf, BYTE** outBuf) {
  // Initialization.
  // Calculate the length that outBuf will have and put it in outLen.
  *outBuf = (BYTE*)calloc(outLen, 1);
  // Do stuff with outBuf.
  return outLen;
}

inBuf is not specified as constant as some bytes may be changed. calloc() rather than malloc() is used in main() and in the function because I want the bytes in both buffers to be initialized to zero, as a few bytes in both are not given values.

I'm using visual Studio 2022 with Windows 10. Somehow the error caused by free(bufB) must be related to the fact that memory is allocated in the function, not in main(). How is this fixed?

CodePudding user response：

Your code is incomplete and does not compile. The main issue appears to be that the type of bufA and bufB is BYTE but should be a pointer BYTE *. Here is working code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef unsigned char BYTE;

int myFunction(const BYTE *inBuf, BYTE **outBuf) {
    int outLen = strlen(inBuf);
    *outBuf = calloc(outLen, 1);
    if(!*outBuf) {
        printf("calloc failed\n");
        exit(1);
    }
    return outLen;
}

int main(void) {
    BYTE *bufA = NULL;
    int lenBufA = 42;
    bufA = calloc(lenBufA, 1);
    if(!bufA) {
        printf("calloc failed\n");
        exit(1);
    }
    memset(bufA, !'\0', lenBufA - 1); // heh

    BYTE *bufB = NULL;
    int lenBufB = myFunction(bufA, &bufB);
    free(bufA);
    free(bufB);
    return 0;
}

It checks the return code of calloc, clarifies with const BYTE *inBuf that function doesn't change that argument.

CodePudding user response：

On Googling ntdll.dll, it appears to be a Windows file of some type, and based on some blurb, it might be corrupted.

Accordingly, I copied the source files to another Windows 10 computer that also has Visual Studio 2022 installed, created a project, then compiled the source code. On executing I get the same message with different hex codes, However the hex code for sixbit-encoding6.exe is the same. Unfortunately the copy and paste feature of the mouse doesn't work for the message on that computer, but it thus looks as if the dll is probably not corrupted, and there is a problem with my application.

CodePudding user response：

I cleaned up my code, but it still fails when I attempt to close the second buffer as free(bufB) at the end, where bufB is created in myFunction(). bufA created in main() causes no problem with free(bufA). If I comment out free(bufB) then an error is thrown at the end. However, if I have only have a buffer created in main(), then end the program without closing the buffer, no errors are produced. Of course that is very bad practice.

Getting back to the code in my initial posting after cleaning it up, at the statement free(bufB) indicated by "<==", Visual Studio shows the following:

Exception Unhandled

Unhandled exception at 0x00007FF985C8F689 (ntdll.dll) in sixbit-encoding6.exe: 0xC0000374: A heap has been corrupted (parameters: 0x00007FF985CF97F0).

Does anyone have an idea what is happening? Obviously it must have something to do with bufB being allocated in the function. I should have mentioned error message like this earlier, but it was gettinbg very late.