Unix and Linux Stack Exchange
Unix and Linux Stack Exchange

Questions tagged [mmap]

All about using memory mapped files. Questions on programming should be asked on Stack Overflow SE.

mmap() is a system call that allows for very fast and convenient file I/O. A file that is mapped into memory can be accessed like any memory region.

mmap() can also be used for inter-process communication: a process that creates an anonymous mapping that is not backed by any file can share this region with any child process since memory mappings are preserved across fork().

Note that pipes and socket cannot be mapped.

75 questions
24
votes
2 answers

How does memory mapping a file have significant performance increases over the standard I/O system calls?

Operating System Concepts says Consider a sequential read of a file on disk using the standard system calls open(), read(), and write(). Each file access requires a system call and disk access. Alternatively, we can use the virtual memory techniques…
Tim
  • 98,580
  • 191
  • 570
  • 977
20
votes
5 answers

what is the purpose of memory overcommitment on Linux?

I know about memory overcommitment and I profoundly dislike it and usually disable it. I am not thinking of setuid-based system processes (like those running sudo or postfix) but of an ordinary Linux process started on some command line by some user…
Basile Starynkevitch
  • 10,411
  • 1
  • 32
  • 52
16
votes
1 answer

Understanding mmap

I was going through documentation regarding mmap here and tried to implement it using this video. I have a few questions regarding its implementation. Does mmap provide a mapping of a file and return a pointer of that location in physical memory or…
john
  • 283
  • 1
  • 2
  • 9
9
votes
3 answers

Memory size for kernel mmap operation

I'm interested in the way Linux mmaps files into the main memory (in my context its for executing, but I guess the mmap process is the same for writing and reading as well) and which size it uses. So I know Linux uses paging with usually 4kB…
user5767
8
votes
1 answer

If I mmap a file from tmpfs, will it double the memory usage?

I am working on a system where we lock files in memory using mmap and MAP_LOCKED and MAP_POPULATE for performance. If we do this with a file that is in tmpfs, will it use the existing tmpfs memory area or will it make a copy for the mmap?
ElefEnt
  • 203
  • 2
  • 6
7
votes
1 answer

ld.so.cache and libc.so.6 memory-mapped for every call?

Playing with strace, it appears to me that ld.so.cache and libc.so.6 are opened and mapped to memory for almost every process. At least those processes that I experimented with. Doesn't this mean that these processes are mapped into process memory…
smolloy
  • 173
  • 4
7
votes
1 answer

How is the page cache mapped in the kernel on 64-bit x86 architectures?

On a modern 64-bit x86 Linux, how is the mapping between virtual and physical pages set up, kernel side? On the user side, you can mmap in pages from the page cache, and this will map 4K pages in directly into user space - but I am interesting in…
BeeOnRope
  • 538
  • 3
  • 13
7
votes
1 answer

OOM from iterating over very large memory map

I'm writing a simulator that relies on generating a (potentially) very large problem domain. As the data can't fit in RAM, I use 4 four memory mapped files to hold it. This is a 64-bit application running on 64-bit Linux with 8GB of RAM. My…
cmannett85
  • 171
  • 5
7
votes
1 answer

Behavior of mmap'd memory on memory pressure

I have a large tar file (60GB) containing image files. I'm using mmap() on this entire file to read in these images, which are accessed randomly. I'm using mmap() for the following reasons: Thread safety -- I cannot seek an ifstream from multiple…
Devin Lane
  • 173
  • 1
  • 4
7
votes
3 answers

How does mmap'ing /dev/mem work despite being from unprivileged mode?

As far as my understanding goes, User space programs run in the unprivileged mode, and thus do not have direct access to memory or I/O. Then how exactly can we directly access memory or I/O locations when we mmap /dev/mem in user space programs? For…
Stark07
  • 552
  • 2
  • 8
  • 20
7
votes
1 answer

File backed, locked shared memory and disk interaction

Varnish, a HTTP accelerator, uses a ~80MB file backed SHM log that is mlock()ed into memory. The Varnish docs recommend to store the file on tmpfs to avoid unnecessary disk access. However if the entire file is locked into memory, does the Linux…
user195311
  • 73
  • 4
6
votes
1 answer

Is it possible to disable unnecessary disk writes to mmap files on linux?

I would like to know if there is a way to prevent Linux from periodically syncing mmap'd files to disk, while still allowing the OS to write back when physical memory gets tight. I am writing applications which process large images, so large that…
Peter Fletcher
  • 61
  • 3
5
votes
2 answers

How to use dd if=/dev/mem in place of devmem ?

It seems like root@testbox:~# dd if=/dev/mem bs=1 count=4 skip=2149646336 | hd should be nearly equivalent to root@testbox:~# devmem 2149646336 32 But, while devmem works great, that dd invocation gives me a segfault. For many other bs,skip…
Gaylon
  • 51
  • 1
  • 1
  • 3
4
votes
1 answer

Hung system call

So I'm working with a custom kernel module that I'm writing a python front end for. The kernel module works, and it adds a framebuffer device file to /dev/fb1. I can read and write to it fine. I've been using python's mmap module to map the device…
Falmarri
  • 12,897
  • 17
  • 58
  • 71
4
votes
1 answer

mmap: effect of other processes writing to a file previously mapped read-only

I am trying to understand what happens when a file, which has been mapped into memory by the mmap system call, is subsequently written to by other processes. I have mmaped memory with PROT_READ protection in "process A". If I close the underlying…
user001
  • 3,598
  • 5
  • 39
  • 54
1
2 3 4 5