In my application I have a huge chunk of memory allocated using mmap. Depending on runtime configuration options this chunk can be allocated using huge pages (i.e. with the MAP_HUGETLB flag). This memory chunk is managed by application in smaller chunks (e.g. 64 KiB).
Now I want to use the allocated memory for custom allocator. But there is a problem if code requests continuous memory larger than the small chunk size. With the default page size (4 KiB) we can simply use mremap to assemble free pages from the huge chunk into a continuous virtual memory region. But if the huge chunk was allocated using huge pages, mremap does not work since we try to remap memory smaller than page size.
Is it possible to map memory allocated by huge pages using smaller chunks without freeing huge pages first? I think in theory it should be possible, since OS has to simply create virtual address translation which points to physical memory "owned" by a huge page, but I couldn't find API which would allow my to do it.
I understand that using this approach I will lose the advantages of using huge pages. But the custom allocator will use only a small part of the huge chunk, so it does not matter that much.
As a workaround, I could allocate two separate chunks with huge and default pages, but such solution introduces risks of exhausting the default pages region and makes the application more complex and a bit less efficient.
