Why is there no apparent clone or fork in simple bash command and how it's done?

Question

Consider the following (with sh being /bin/dash):

$ strace -e trace=process sh -c 'grep "^Pid:" /proc/self/status /proc/$$/status'
execve("/bin/sh", ["sh", "-c", "grep \"^Pid:\" /proc/self/status /"...], [/* 47 vars */]) = 0
arch_prctl(ARCH_SET_FS, 0x7fcc8b661540) = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fcc8b661810) = 24865
wait4(-1, /proc/self/status:Pid:    24865
/proc/24864/status:Pid: 24864
[{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 24865
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=24865, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
exit_group(0)                           = ?
+++ exited with 0 +++

There's nothing unusual, grep replaced a forked process (here done via clone()) from main shell process. So far so good.

Now with bash 4.4:

$ strace -e trace=process bash -c 'grep "^Pid:" /proc/self/status /proc/$$/status'
execve("/bin/bash", ["bash", "-c", "grep \"^Pid:\" /proc/self/status /"...], [/* 47 vars */]) = 0
arch_prctl(ARCH_SET_FS, 0x7f8416b88740) = 0
execve("/bin/grep", ["grep", "^Pid:", "/proc/self/status", "/proc/25798/status"], [/* 47 vars */]) = 0
arch_prctl(ARCH_SET_FS, 0x7f8113358b80) = 0
/proc/self/status:Pid:  25798
/proc/25798/status:Pid: 25798
exit_group(0)                           = ?
+++ exited with 0 +++

Here what's apparent is that grep assumes pid of the shell process and no apparent fork() or clone() call. Question is, then, how does bash achieve such acrobatics without either of the calls ?

Note, however, that clone() syscalls appears if the command contains shell redirection, such as df > /dev/null

Answer 1

The sh -c 'command line' are typically used by things like system("command line"), ssh host 'command line', vi's !, cron, and more generally anything that is used to interpret a command line, so it's pretty important to make it as efficient as possible.

Forking is expensive, in CPU time, memory, allocated file descriptors... Having a shell process lying about just waiting for another process before exiting is just a waste of resources. Also, it makes it difficult to correctly report the exit status of the separate process that would execute the command (for instance, when the process is killed).

Many shells will generally try to minimize the number of forks as an optimisation. Even non-optimised shells like bash do it in the sh -c cmd or (cmd in subshell) cases. Contrary to ksh or zsh, it doesn't do it in bash -c 'cmd > redir' or bash -c 'cmd1; cmd2' (same in subshells). ksh93 is the shell that goes furthest in avoiding forks.

There are cases where that optimisation cannot be done, like when doing:

sh < file

Where sh can't skip the fork for the last command, because more text could be appended to the script whilst that command is running. And for non-seekable files, it can't detect the end-of-file as that could mean reading too much too early from the file.

Or:

sh -c 'trap "echo Ouch" INT; cmd'

Where the shell may have to run more commands after the "last" command has been executed.

Answer 2

By digging through bash source code, I was able to figure out that bash in fact will ignore forking if there's no pipes or redirections. From line 1601 in execute_cmd.c:

  /* If this is a simple command, tell execute_disk_command that it
     might be able to get away without forking and simply exec.
     This means things like ( sleep 10 ) will only cause one fork.
     If we're timing the command or inverting its return value, however,
     we cannot do this optimization. */
  if ((user_subshell || user_coproc) && (tcom->type == cm_simple || tcom->type == cm_subshell) &&
      ((tcom->flags & CMD_TIME_PIPELINE) == 0) &&
      ((tcom->flags & CMD_INVERT_RETURN) == 0))
    {
      tcom->flags |= CMD_NO_FORK;
      if (tcom->type == cm_simple)
    tcom->value.Simple->flags |= CMD_NO_FORK;
    }

Later those flags go to execute_disk_command() function, which sets up nofork integer variable, which then later is checked before attempting forking. The actual command itself would be run by execve() wrapper function shell_execve() from either forked or parent process, and in this case it's the actual parent.

The reason for such mechanic is well explained in Stephane's answer.

---

Side note outside the scope of this question: should be noted that apparently it matters whether the shell is interactive or running via -c. Prior to executing the command there will be a fork. This is evident from running strace on interactive shell (strace -e trace=process -f -o test.trace bash) and checking the output file:

19607 clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_t
idptr=0x7f2d35e93a10) = 19628
19607 wait4(-1,  <unfinished ...>
19628 execve("/bin/true", ["/bin/true"], [/* 47 vars */]) = 0

See also Why bash does not spawn a subshell for simple commands?