1

I'd like to run a Linux process in a fake-write environment where all file writes (with the write(2) system call) are redirected to an in-memory cache, and subsequent reads (of the same region only) will be served from the cache. The cache can be discarded after the process finishes.

The files the process will be modifying are huge (several terabytes), but the total amount of written data is small (a few megabytes). So overlayfs is not an option, because I don't have several terabytes of free space.

Please note that I don't care about fake device nodes or fake permissions, thus fakeroot(1) doesn't help me.

This should be possible to implement with LD_PRELOAD, ptrace(2), fuse(8) or dm snapshot. Is there an existing tool which will do it for me, like this:

$ fakewrites ./myprog myfile.dat
pts
  • 1,041
  • 13
  • 22
  • 2
    [This answer](https://unix.stackexchange.com/a/67681/86440) explains how to do this on top of a read-only device, using snapshots. (The only sensible way to do this *is* on top of a read-only device, because you can’t keep multiple writes from different sources synchronised. If the base device is writable, and is updated while there’s an overlay...) – Stephen Kitt Nov 22 '19 at 17:23
  • So [GNU/Linux: overlay block device / stackable block device](https://unix.stackexchange.com/q/67678/86440) is possibly a duplicate but I don’t want to suggest it because I have a hammer here (so no voting). – Stephen Kitt Nov 22 '19 at 17:24
  • @StephenKitt: Thank you for finding a solution using *dm snapshot*! This can work as root even if `myfile.dat` is a file (then we need an extra loop device). Feel free to copy your finding to an answer. – pts Nov 23 '19 at 09:58
  • The way we deal with such situations is to indicate that your question (which remains good and valid) already has an answer elsewhere, so I’ll do that. – Stephen Kitt Nov 23 '19 at 10:19
  • It's not an exact duplicate, beacuse a solution based on LD_PRELOAD, *ptrace(2)* or *fuse(8)* would work without root access, and it could also work if *./myprog* opens multiple files for writing. If I need such a solution, I'll ask another question with the details. – pts Nov 23 '19 at 10:33

1 Answers1

0

https://launchpad.net/libeatmydata

LD_PRELOAD library that disables all forms of writing data safely to disk. fsync() becomes a NO-OP, O_SYNC is removed etc. The idea is to use in testing to get faster test runs where real durability is not required.

Ipor Sircer
  • 14,376
  • 1
  • 27
  • 34
  • 3
    Thank you for trying to help! Unfortunately libeatmydata isn't able to help me, because it doesn't override any read(...) or write(...) calls. (I was looking at its source: https://github.com/stewartsmith/libeatmydata/blob/master/libeatmydata/libeatmydata.c) – pts Nov 22 '19 at 13:22