Overwriting a running executable or .so

Question

I have a question about overwriting a running executable, or overwriting a shared library (.so) file that's in use by one or more running programs.

Back in the day, for the obvious reasons, overwriting a running executable didn't work. There's even a specific errno value, ETXTBSY, that covers this case.

But for quite a while now, I've noticed that when I accidentally try to overwrite a running executable (for example, by firing off a build whose last step is cc -o exefile on an exefile that happens to be running), it works!

So my questions are, how does this work, is it documented anywhere, and is it safe to depend on it?

It looks like someone may have tweaked ld to unlink its output file and create a new one, just to eliminate errors in this case. I can't quite tell if it's doing this all the time, or only if it needs to (that is, perhaps after it tries to overwrite the existing file, and encounters ETXTBSY). And I don't see any mention of this on ld's man page. (And I wonder why people aren't complaining that ld may now be breaking their hard links, or changing file ownership, and like that.)

---

Addendum: The question wasn't specifically about cc/ld (although that does end up being a big part of the answer); the question was really just "How come I never see ETXTBSY any more? Is it still an error?" And the answer is, yes, it is still an error, just a rare one in practice. (See also the clarifying answer I just posted to my own question.)

Answer 1

It depends on the kernel, and on some kernels it might depend on the type of executable, but I think all modern systems return ETXTBSY (”text file busy“) if you try to open a running executable for writing or to execute a file that's open for writing. Documentation suggests that it's always been the case on BSD, but it wasn't the case on early Solaris (later versions did implement this protection), which matches my memory. It's been the case on Linux since forever, or at least 1.0.

What goes for executables may or may not go as well for dynamic libraries. Overwriting a dynamic library causes exactly the same problem that overwriting an executable does: instructions will suddenly be loaded from the same old address in the new file, which probably has something completely different. But this is in fact not the case everywhere. In particular, on Linux, programs call the open system call to open a dynamic library under the hood, with the same flags as any data file, and Linux happily allows you to rewrite the library file even though a running process might load code from it at any time.

Most kernels allow removing and renaming files while they're being executed, just like they allow removing and renaming files while they're open for reading or writing. Just like an open file, a file that's removed while it's being executed will not be actually removed from the storage medium as long as it is in use, i.e. until the last instance of the executable exits. Linux and *BSD allow it, but Solaris and HP-UX don't.

Removing a file and writing a new file by the same name is perfectly safe: the association between the code to load and the open (or being-executed) file that contains the code goes by the file descriptor, not the file name. It has the additional benefit that it can be done atomically, by writing to a temporary file then moving that file into place (the rename system call atomically replaces an existing destination file by the source file). It's much better than remove-then-open-write since it doesn't temporarily put an invalid, partially-written executable in place

Whether cc and ld overwrite their output file, or remove it and create a new one, depends on the implementation. GCC (at least modern versions) and Clang do this, in both cases by calling unlink on the target if it exists then open to create a new file. (I wonder why they don't do write-to-temp-then-rename.)

I don't recommend depending on this behavior except as a safeguard since it doesn't work on every system (it may work on every modern systems for executables, but not for shared libraries), and common toolchains don't do things in the best way. In your build scripts, always generate files under a temporary file, then move them into place, unless you know the underlying tool does this.

Answer 2

What generally happens if a file is removed while it has an open file handle is that it is marked for deletion as soon as the last file handle is closed. The file will at that time no longer appear in directory listings (for instance), but will show up in e. g. lsof output, marked as a deleted-but-in-use file.

The outout of lsof below is trimmed for brevity and clarity:

$ cat - >> foo &
[1] 30779
$ lsof | grep 30779
cat       30779                  ghoti    1w      REG      252,0        0    262155 /home/ghoti/foo

[1]+  Stopped                 cat - >> foo
$ rm foo
$ ls foo
ls: cannot access 'foo': No such file or directory
$ lsof | grep 30779
cat       30779                  ghoti    1w      REG      252,0        0    262155 /home/ghoti/foo (deleted)

If I fg, I can still write to the (deleted) foo with the still-running cat (and indeed I can recover the file from /proc/30779/fd/1 should I need to, so long as I do so while cat still has the file open).

Answer 3

Answering my own question (or really, clarifying the question somewhat, and then explaining the answer to the clarified question):

My question was sort of, "I never see ETXTBSY any more. Is it still even an error? Or are modern kernels letting you overwrite running executables without complaint, and (somehow) without breaking the running executables?"

I was beginning to seriously suspect that modern kernels were implementing some kind of fancy copy-on-write semantics when writing to running executables.

But that wasn't it. ETXTBSY is still definitely an error.

The answer to my confusion is simply that writes to running executables really hardly ever come up in practice. If you move a new executable into place (and the old one is still running), you're almost never actually overwriting it; you're always removing and replacing it. If you're using mv, you're removing and replacing it. If you're using install, or dpkg -i, or anything like that, you're removing and replacing it. Only if for some reason you tried to use cp would you be trying to overwrite it, and running the risk of getting ETXTBSY if the old one were still running.

And then, due to the quiet change to ld, trying to cc -o on top of a running executable is now in the category of "removing and replacing", also.

Answer 4

Your guess is correct: ld writes a new file, which you can check using ls -li. The -i option shows the inode number, which changes after each compile. I don't think very many care about breaking hard links; programs are not usually compiled so that the executable is written to its final destination by ld.