Can't detect huge memory usage in C program-CodePudding

I am experiencing a huge memory usage when running my C program. It uses way more memory than it is expected to use. Even the OS sometimes has to kill the process due to a lot of memory usage (And I have 20GB of RAM in my machine). I try to run valgrind to see if there are some clues about what is going on with memory usage. Among the valgrind output, I can see a warning that might be pointing to the issue:

==69541== Warning: set address range perms: large range [0x76eb040, 0x2cb27240) (undefined)

However, I do not know how to know where is this large range of memory being allocated.

Hope this is not a too generic question. I tried to provide the minimum details to solve the issue, but if needed, I can provide more information.

CodePudding user response：

You should use Valgrind with --leak-check=full option and compile code with -g option

Example:

cat my_leak.c ; gcc -g my_leak.c ; valgrind --leak-check=full a.out
#include <malloc.h>

int main() {
    char* a = malloc(1000000);
    return 0;
}
==29805== Memcheck, a memory error detector
==29805== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==29805== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==29805== Command: a.out
==29805== 
==29805== error calling PR_SET_PTRACER, vgdb might block
==29805== 
==29805== HEAP SUMMARY:
==29805==     in use at exit: 1,000,000 bytes in 1 blocks
==29805==   total heap usage: 1 allocs, 0 frees, 1,000,000 bytes allocated
==29805== 
==29805== 1,000,000 bytes in 1 blocks are definitely lost in loss record 1 of 1
==29805==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==29805==    by 0x10915E: main (my_leak.c:4)
==29805== 
==29805== LEAK SUMMARY:
==29805==    definitely lost: 1,000,000 bytes in 1 blocks
==29805==    indirectly lost: 0 bytes in 0 blocks
==29805==      possibly lost: 0 bytes in 0 blocks
==29805==    still reachable: 0 bytes in 0 blocks
==29805==         suppressed: 0 bytes in 0 blocks
==29805== 
==29805== For lists of detected and suppressed errors, rerun with: -s
==29805== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Valgrind version:

valgrind --version
valgrind-3.15.0

CodePudding user response：

The message that you copied is generated whenever Valgrind sets permissions on a new range of memory that is larger than 256Mbytes. Such large allocations are somewhat suspicious and could be an error in the calculation of the memory size.

In your case the allocation is 625197568 bytes (about 596Mbytes).

You can use the massif tool to see where this is being allocated (or a non-Valgrind tool like heaptrack).

CodePudding user response：

I was able to find the problem. I post here the approach I used and worked for me, just in case it might be useful for someone else (although probably is not the neatest one).

Using the debugger I spotted the place were the huge allocation was taking place (it was taking a lot of time). Then I realized that I was creating a hash-table object, which its size is passed as parameter. I was not passing the parameter, so the program was taking a garbage (big) number as the size.

Thanks to all of you that shared your thoughts about my question :)