64

I'm trying to restart services after a yum update on RHEL 7.4. I could restart every service using systemctl, but needs-restarting from yum utils tells me that I should also restart systemd itself:

# needs-restarting
1 : /usr/lib/systemd/systemd --system --deserialize 21

Can I restart systemd without rebooting the server, and how?

I found a few mentions of systemctl daemon-reload, but this doesn't make it disappear from the needs-restarting list.

Martin Schröder
  • 939
  • 1
  • 10
  • 35
BenMorel
  • 4,447
  • 8
  • 36
  • 46

2 Answers2

72

To restart the daemon, run

systemctl daemon-reexec

This is documented in the systemctl manpage:

Reexecute the systemd manager. This will serialize the manager state, reexecute the process and deserialize the state again. This command is of little use except for debugging and package upgrades. Sometimes, it might be helpful as a heavy-weight daemon-reload. While the daemon is being reexecuted, all sockets systemd listening on behalf of user configuration will stay accessible.

Unfortunately needs-restarting can’t determine that systemd has actually restarted. systemd execs itself to restart, which doesn’t reset the process’s start time; but needs-restarting compares the executable’s modification time with the process’s start time to determine whether a process needs to be restarted (among other things), and as a result it always considers that systemd needs to be restarted... To determine whether systemd really needs to be restarted, you can check the output of lsof -p1 | grep deleted: systemd uses a library, libsystemd-shared, which is shipped in the same package and is thus upgraded along with the daemon, so if systemd needs to be restarted you’ll see it using a deleted version of the library. If lsof shows no deleted files, systemd doesn’t need to be restarted. (Thanks to Jeff Schaller for the hint!)

Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164
  • Since the process will most likely be running with pid 1, the reboot will need to be executed. – Raman Sailopal Jan 24 '18 at 14:56
  • 1
    @Raman, `daemon-reexec` is supposed to work even with `systemd` as pid 1. – Stephen Kitt Jan 24 '18 at 15:03
  • 3
    The crux of needs-restarting boils down to https://github.com/rpm-software-management/yum/blob/master/utils.py#L132 where it queries the PID's "start_time"; if the daemon-reexec doesn't update that, needs-restarting will remain "confused". – Jeff Schaller Jan 24 '18 at 15:21
  • 1
    Do not assume that any codepath involved is well-tested, especially on non-RedHat systems. It's technically possible to run daemon-reexec, but it's safer to reboot. – Harald Jan 29 '18 at 19:28
  • 2
    @Harald it’s used any time anyone upgrades `systemd` on Debian and derivatives, so it is well-tested. It’s also reasonably straightforward (look for [`do_reexecute`](https://github.com/systemd/systemd/blob/master/src/core/main.c)). – Stephen Kitt Jan 29 '18 at 19:49
  • 1
    @StephenKitt - When i attempt to run `lsof -p1 | grep deleted` the following output is generated `lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs Output information may be incomplete`. In reading (https://unix.stackexchange.com/questions/171519/lsof-warning-cant-stat-fuse-gvfsd-fuse-file-system), it appears that even root is unable to access it. What would the alternative be to `lsof -p1 | grep deleted`? – Motivated Feb 01 '19 at 16:52
  • 1
    @Motivated nothing, you can ignore that in the context of this answer. – Stephen Kitt Feb 01 '19 at 17:31
  • 1
    @StephenKitt - Nothing as there as there no other option and that there the warning output should be ignored? – Motivated Feb 01 '19 at 17:32
  • 1
    @Motivated yes. – Stephen Kitt Feb 01 '19 at 17:33
  • @StephenKitt - If i am ignoring the warning, it seems that i am unable to execute the command. Do you mean to say that since there is no other option, i should simply restart the device? – Motivated Feb 01 '19 at 17:52
  • @Motivated no, I mean you should ignore the warning. Why would it mean that you can’t execute the command? – Stephen Kitt Feb 01 '19 at 19:13
  • @StephenKitt - I meant to say that if i ignore the warning and don't resolve the issue, the command won't execute since it doesn't at the moment. In searching online, it suggests it can't be resolved (at least that's my understanding). – Motivated Feb 02 '19 at 16:26
  • 1
    @Motivated it’s a warning, not an error, it doesn’t stop the command from executing. All it means is that `lsof` won’t provide information about files under `/run/user/1000/gvfs`. Since the point here is to check whether `systemd` is using a deleted version of a library, and neither systemd nor the libraries it uses live under `/run`, the fact that `lsof` can’t analyse anything under `/run/user/1000/gvfs` doesn’t alter the usefulness of the command for this purpose. – Stephen Kitt Feb 02 '19 at 18:56
  • @StephenKitt - Thanks. That clarified my understanding. I have assumed that if it returns to the shell prompt, no deleted files were found. To clarify, what does -p1 refer to? – Motivated Feb 03 '19 at 03:18
  • @StephenKitt - According to (https://serverfault.com/questions/805745/reboot-or-systemctl-daemon-reload-for-changes-to-etc-systemd-system-conf), it suggests that there is no requirement to run the command `systemctl daemon-reexec` as there is a scriplet included in package when `systemd` is upgraded. – Motivated Feb 03 '19 at 03:37
  • @Motivated yes, and I alluded to that in the fifth comment under this answer. Some distributions restart `systemd` on upgrade, others don’t, my answer explains how to restart `systemd` and how to determine if it’s necessary in all cases. – Stephen Kitt Feb 03 '19 at 08:30
  • @StephenKitt - Thanks Stephen. What does `-p1` refer to? – Motivated Feb 03 '19 at 16:11
  • If you really are @Motivated, you should be able to figure that out from `man lsof`. – Stephen Kitt Feb 03 '19 at 16:20
  • @StephenKitt - I did and the description in the man page isn't clear to me `excludes or selects the listing of files for the processes whose optional process IDentification (PID) numbers are in the comma-separated set s - e.g., ``123'' or ``123,^456''. (There should be no spaces in the set.)`. What does mean optional process whose identification numbers are comma-separated? – Motivated Feb 03 '19 at 16:27
  • 1
    @Motivated hmm, I can see why it wouldn’t be all that clear... `-p` tells `lsof` to match processes whose pid corresponds to `-p`’s arguments, so 1 for `-p1` (and pid 1 is always the system’s init process, `systemd` when you use systemd), or 123 for `-p123`; the match can be negated using `^`, so `^456` matches all processes except pid 456, and multiple criteria can be specified by separating them with commas. – Stephen Kitt Feb 03 '19 at 16:49
  • @StephenKitt - Thanks heaps. That's much clearer to me now. – Motivated Feb 03 '19 at 16:51
7

In my case, I had just upgraded systemd and any systemctl command was failing:

# systemctl daemon-reexec
Failed to reload daemon: Access denied
# systemctl status
Failed to read server status: Access denied

However according to the init manpage, you can do the same thing by sending SIGTERM to the daemon running as PID 1, which worked:

kill -TERM 1

This reloaded the daemon, after which all the systemctl commands started working again.

Malvineous
  • 6,524
  • 5
  • 52
  • 76