Anonymous mmap with near-unlimited growth-CodePudding

I'd like to declare an array that can start small and grow to a terabyte or so without bumping into anything else or wasting physical memory. There are several of these "miniheaps" with different sized structs in them and I want them to spread themselves out through virtual memory. I can do this if I back each miniheap with a file and that's my main intention, but for testing purposes I'd like a forgetful variant. It certainly isn't allowed to zero the whole terabyte.

The code below works when fd is a file but says "Can't make file mapping" when fd==-1. What should I write instead?

(I'm using linux and have no interest in porting it to anything else.)

   void * reserve; 
   // 'lim*PAGE' is 1TB ...                                                                                             
   reserve = mmap(0, lim*PAGE, PROT_NONE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
   if (reserve == (void*)-1) { fprintf(stderr, "Can't make reserve mapping\n"); quit(1);  }
                                  
   void * filemap; 
   // 'fd' is an open file or -1 ...                                                                                             
   if (fd!=-1) {                                                                                                                                                                                          
     filemap = mmap(reserve, fileSize(fd), PROT_READ|PROT_WRITE, MAP_SHARED_VALIDATE|MAP_FIXED, fd, 0);          
   } else {              
     // stp is 1 or 2 and PAGE is 4k ...                                                                                                                                                                                 
     filemap = mmap(reserve, PAGE*stp, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_FIXED, fd, 0);                    
   }                                                                                                             
                         
   if (filemap == (void*)-1) { fprintf(stderr, "Can't make file mapping\n"); quit(1);  } 

   // fileSize and quit are my own utilities for the obvious purposes.

CodePudding user response：

Problem is MAP_FIXED. It forces mmap to.place a mapping at a fixed address. But address reserve is already mapped to some anonymous private pages. Thus mmap fails. Does it work if MAP_FIXED is removed?

CodePudding user response：

I just added MAP_PRIVATE to the second mmap and it worked. Reading malloc.c is what gave it away.