How can I reserve a block of memory from the Linux kernel?

Question

I have a device that needs a block of memory that is reserved solely for it, without the OS intervening. Is there any way to tell BIOS or the OS that a block of memory is reserved, and it must not use it?

I am using this device on an openSUSE machine.

Aaron D. Marasco · Answer 1 · 2016-09-26T13:41:43.220

26

If you want the OS to totally ignore it, you need to make a memory hole using "memmap." See this reference. For example, if you want 512M at the 2GB barrier, you can put "memmap=512M$2G" on your kernel command line.

You will need to check your dmesg to find a contiguous hole to steal so you don't stomp on any devices; that is specific to your motherboard+cards.

This is not the recommended way to do things - see Warren Young's answer for how to properly do it (kernel drivers + DMA). I'm answering the exact question you asked. If you plan on making this for end users, they will hate you if you do this to them... trust me, that's the only reason I knew this answer.

Edit: If you are using grub2 w/ grubby (e.g. CentOS 7), you need to make sure to escape the $. There should be a single \ before $. Example:

$ sudo -v
$ sudo grubby --update-kernel=ALL --args=memmap='128M\\$0x57EF0000'
$ sudo grubby --info $(sudo grubby --default-kernel) | grep memmap
args="ro crashkernel=auto ... memmap=128M\$0x57EF0000"

edited Sep 26 '16 at 13:41

answered May 01 '12 at 23:10

Aaron D. Marasco

5,708
24
29

Having created such a memory hole, can you then access it with `mmap(MAP_ANONYMOUS)`? If not, how? I ask here instead of posting a separate question because I assume it's a necessary part of what the OP is asking. – Warren Young May 02 '12 at 00:12
This is not recommended for end users. This is for a probe I'm building for a research project for my MSc. – Nathan Fellman May 02 '12 at 04:14
1

While your answer does answer my question directly, Warren's answer seems to be the better way to go about it. :-) – Nathan Fellman May 02 '12 at 06:32
1

@WarrenYoung I have *no* idea what you are supposed to do with it. I would guess "`uint8_t *ptr = 0x8000000`" with my example? Or that may segfault... Huh, I really don't know. Again, I knew the answer because I was an end user of a poorly designed PCI card where I had to manually allocate a buffer below the 4G mark and then tell a driver where that space was; it may not be possible from userland. – Aaron D. Marasco May 02 '12 at 10:14
@NathanFellman Yes and yes, I was sure to note that. ;) I even upvoted his correct answer. – Aaron D. Marasco May 02 '12 at 10:16
@AaronD.Marasco: that was meant by way of explanation of why I accepted his answer even though you answered my question directly – Nathan Fellman May 02 '12 at 12:06
2

Just for grins, I looked a little deeper into this, and it appears you need `MMAP_FIXED | MMAP_ANON`. Without a custom DMA device here to play with, I can't say if it actually does what the OP wanted, but my CentOS box happily gave me an 8 MB block at 512 MB when I said `memmap=8M$512M` in GRUB. It doesn't even require root access, as I had feared it might. But even if this does do the right thing, I still think you'll probably need a driver to handle interrupts and such. – Warren Young May 02 '12 at 13:05
Well, `grub` can be configured to have a password, and it doesn't need to match root's. So you used `mmap` to peek at the memory, and it didn't segfault? Cool. I guess it would have in it whatever was there from the previous reboot... – Aaron D. Marasco May 03 '12 at 00:47
3

@AaronD.Marasco: No, `mmap()`'d pages are zeroed before userland code gets to see them. It's done for security, so that data doesn't leak from one process to the next. Another reason to use a driver, since you might need to preserve the contents of the DMA buffer at driver load time. Oh and by the way, the arbitrarily-located `mmap()` call succeeds even without the `memmap` kernel boot option, at least as long as no one is already using memory located where you asked. The boot option doubtless increases the chances of success, but it's not strictly necessary. – Warren Young May 03 '12 at 17:42
I have written a driver to expose this memory to the user space: http://stackoverflow.com/q/12790382/143897 : explains the issue i am seeing, comments welcome ! Thanks – Jay D Oct 08 '12 at 22:49
we have to use `numactl` to see the stolen chunks or use `dmesg | grep -i numa` – Jay D Oct 08 '12 at 23:03
@JayD You can do this without any coding via standard drivers like phram. – vitaly.v.ch Jul 31 '13 at 11:44
this answer is a little misleading. the `memmap` boot arg isn't really an *alternative* to writing a DMA driver. `get_free_pages` can only return a contiguous block of up to 128KB, so if your DMA driver needs a larger contiguous block, `memmap` is a completely legitimate method of achieving that. it's not a hack. – Woodrow Barlow Mar 20 '17 at 18:14
@WoodrowBarlow The CMA [disagrees with you](https://lwn.net/Articles/486301/). Although that was not available when I wrote this answer nearly five years ago. – Aaron D. Marasco Mar 21 '17 at 09:36
1

@AaronD.Marasco hmm, okay. interesting. thank you for that link, it's a good read. – Woodrow Barlow Mar 21 '17 at 14:10

Warren Young · Accepted Answer · 2012-05-02T00:14:01.600

23

What you're asking for is called DMA. You need to write a driver to reserve this memory.

Yes, I realize you said you didn't want the OS to intervene, and a driver becomes part of the OS, but in absence of a driver's reservation, the kernel believes all memory belongs to it. (Unless you tell the kernel to ignore the memory block, per Aaron's answer, that is.)

Chapter 15 (PDF) of "Linux Device Drivers, 3/e" by Rubini, Corbet and Kroah-Hartmann covers DMA and related topics.

If you want an HTML version of this, I found the second-edition version of the chapter elsewhere online. Beware that the 2nd edition is over a decade old now, having come out when kernel 2.4 was new. There's been a lot of work on the memory management subsystem of the kernel since those days, so it may not apply very well any more.

edited May 02 '12 at 00:14

answered May 01 '12 at 19:28

Warren Young

71,107
16
178
168

With DMA can I choose what physical addresses to use? Will the kernel give me a contiguous block of memory? Is it guaranteed to be always available? – Nathan Fellman May 02 '12 at 04:15
1

Your questions are answered starting on page 442 of the PDF I pointed you to. – Warren Young May 02 '12 at 04:34

score 6 · Answer 3 · answered Dec 02 '16 at 08:37

To reserve a block of memory from the kernel in ARM based Linux, you can also use a reserved-memory node in your device tree (dts) file. In kernel documentation (see here), there is an example:

memory {
    reg = <0x40000000 0x40000000>;
};

reserved-memory {
    #address-cells = <1>;
    #size-cells = <1>;
    ranges;

    /* global autoconfigured region for contiguous allocations */
    linux,cma {
        compatible = "shared-dma-pool";
        reusable;
        size = <0x4000000>;
        alignment = <0x2000>;
        linux,cma-default;
    };

    display_reserved: framebuffer@78000000 {
        reg = <0x78000000 0x800000>;
    };

    multimedia_reserved: multimedia@77000000 {
        compatible = "acme,multimedia-memory";
        reg = <0x77000000 0x4000000>;
    };
};

score 0 · Answer 4 · edited Jan 24 '13 at 16:17

0

First enter this command, to check your current setting:

sysctl vm.min_free_kbytes

To change the set value, edit /etc/sysctl.conf. Look for the line:

vm.min_free_kbytes=12888

If it doesn't exist, create it (along with your desired value). The following values are acceptable:

8M is extremely conservative; it can sit at 16M comfortably. Once you change the value, run this and no reboot is needed:

sudo sysctl -p

edited Jan 24 '13 at 16:17

Michael Mrozek

91,316
38
238
232

answered Jan 24 '13 at 08:07

2

Please don't blank your posts; either delete it or flag it for a moderator to remove for you. – jasonwryan Jan 24 '13 at 08:29

How can I reserve a block of memory from the Linux kernel?

4 Answers4

Linked

Related