LTP test the fork system call, the first thing to cycle mmap 16 terabytes of memory space, every time the mmap 1 gb, but after more than 1 TB mmap return ENOMEM error, finally found the problem in the kernel code mmap. Unmapped_area_topdown function in the c file:

.
Vma=rb_entry (mm - & gt; Mm_rb. The rb_node, struct vm_area_struct vm_rb);
If (vma - & gt; Rb_subtree_gap & lt; Length)
The return - ENOMEM;
.

According to the code to see because the red-black tree node cannot allocate the length of the length of memory, involves the red-black tree, no details about before, I don't know can expand, such as the virtual address space for 16 TB is sure enough, the first post, hope to have a big answer, thank you!

CodePudding user response:

Debugging found TASK_SIZE only 1 TB, check found is to support high address the kernel compilation options open, but after open the compilation and submitted to the "Micro - assembler field overflow", try to increase as the TLB exception handler reserved space, but the same mistake, or a newspaper bug should not here, have no idea,