I had set the xms to 32g and xmx to 32g
The program started, but when around 25GB data was loaded into memory, the process was killed by Linux, giving the reason of memory issues.
If 32g was already assigned to the process due to xms being 32g, why did it went out of memory? Doesn't xms mean allocate this memory in beginning and if you cannot please don't start the process ?
Can someone please explain, why programs fails ?
CodePudding user response：
"Allocating memory" really means "allocating virtual address space". Modern operating systems separate the address space used by a process from physical memory.
So, with -Xms32G, you've got 32G of address space.
Actual memory is allocated in pages and on demand, which generally means that something has to actually 'touch' a page before memory is 'committed' to the page.
Thus in reality, memory is only being committed as needed. And if the OS decides it is under real-memory pressure at the time, you're likely to get killed.
You can force the JVM to touch each page at startup by using the JVM option -XX: AlwaysPreTouch. The likely effect of this will be that the JVM process starts and gets killed during its initialization, before your program is entered at main(). You still can't get the memory, you just find out sooner.
The OOM killer being what it is, it is also possible that the pretouch will go ok, your code will run, but at some later time due to other system activity, the kernel will decide it's critically low on available resources, and since you're probably the largest process around, there's a target on your back.
CodePudding user response：
First of all -Xms and -Xmx control the heap of your java process. A java process need memory for other things too, like off heap allocation, GC structures, etc. What this means is that when you set your application to a heap of 1GB, your java process is going to need more than that.
Second point is that -Xms controls the committed memory in the virtual space. Virtual space for a 64 bit CPU is huge. Committed memory becomes resident in a lazy fashion (resident is actual in RAM, for example). So when you set -Xmx and -Xms, you will get a portion of virtual memory allocated as needed.
You should carefully read the logs on this part:
giving the reason of memory issues
you probably are killed by the OOM Killer, which kills the process, which you already know (from the above) that needs more than the heap.
If you want to allocate memory and make it resident, add the -XX: AlwaysPreTouch flag. This will make your process start a lot slower though and in general, is not a great idea. It has its benefits in various applications.