How can I kill and wait for background processes to finish in a shell script when I Ctrl+C it?

Question

I'm trying to set up a shell script so that it runs background processes, and when I Ctrlc the shell script, it kills the children, then exits.

The best that I've managed to come up with is this. It appears that the kill 0 -INT also kills the script before the wait happens, so the shell script dies before the children complete.

Any ideas on how I can make this shell script wait for the children to die after sending INT?

#!/bin/bash
trap 'killall' INT

killall() {
    echo "**** Shutting down... ****"
    kill 0 -INT
    wait # Why doesn't this wait??
    echo DONE
}

process1 &
process2 &
process3 &

cat # wait forever

Answer 1

Your kill command is backwards.

Like many UNIX commands, options that start with a minus must come first, before other arguments.

If you write

kill -INT 0

it sees the -INT as an option, and sends SIGINT to 0 (0 is a special number meaning all processes in the current process group).

But if you write

kill 0 -INT

it sees the 0, decides there's no more options, so uses SIGTERM by default. And sends that to the current process group, the same as if you did

kill -TERM 0 -INT    

(it would also try sending SIGTERM to -INT, which would cause a syntax error, but it sends SIGTERM to 0 first, and never gets that far.)

So your main script is getting a SIGTERM before it gets to run the wait and echo DONE.

Add

trap 'echo got SIGTERM' TERM

at the top, just after

trap 'killall' INT

and run it again to prove this.

As Stephane Chazelas points out, your backgrounded children (process1, etc.) will ignore SIGINT by default.

In any case, I think sending SIGTERM would make more sense.

Finally, I'm not sure whether kill -process group is guaranteed to go to the children first. Ignoring signals while shutting down might be a good idea.

So try this:

#!/bin/bash
trap 'killall' INT

killall() {
    trap '' INT TERM     # ignore INT and TERM while shutting down
    echo "**** Shutting down... ****"     # added double quotes
    kill -TERM 0         # fixed order, send TERM not INT
    wait
    echo DONE
}

./process1 &
./process2 &
./process3 &

cat # wait forever

Answer 2

Unfortunately, commands started in background are set by the shell to ignore SIGINT, and worse, they can't un-ignore it with trap. Otherwise, all you'd have to do is

(trap - INT; exec process1) &
(trap - INT; exec process2) &
trap '' INT
wait

Because process1 and process2 would get the SIGINT when you press Ctrl-C since they're part of the same process group which is the foreground process group of the terminal.

The code above will work with pdksh and zsh which in that regard are not POSIX conformant.

With other shells, you would have to use something else to restore the default handler for SIGINT like:

perl -e '$SIG{INT}=DEFAULT; exec "process1"' &

or use a different signal like SIGTERM.

Answer 3

For those that just want to kill a process and wait for it to die, but not indefinitely:

It waits max 60 seconds per signal type.

Warning: This answer is in no way related to traping a kill signal and dispatching it.

# close_app_sub GREP_STATEMENT SIGNAL DURATION_SEC
# GREP_STATEMENT must not match itself!
close_app_sub() {
    APP_PID=$(ps -x | grep "$1" | grep -oP '^\s*\K[0-9]+' --color=never)
    if [ ! -z "$APP_PID" ]; then
        echo "App is open. Trying to close app (SIGNAL $2). Max $3sec."
        kill $2 "$APP_PID"
        WAIT_LOOP=0
        while ps -p "$APP_PID" > /dev/null 2>&1; do
            sleep 1
            WAIT_LOOP=$((WAIT_LOOP+1))
            if [ "$WAIT_LOOP" = "$3" ]; then
                break
            fi
        done
    fi
    APP_PID=$(ps -x | grep "$1" | grep -oP '^\s*\K[0-9]+' --color=never)
    if [ -z "$APP_PID" ]; then return 0; else return "$APP_PID"; fi
}

close_app() {
    close_app_sub "$1" "-HUP" "60"
    close_app_sub "$1" "-TERM" "60"
    close_app_sub "$1" "-SIGINT" "60"
    close_app_sub "$1" "-KILL" "60"
    return $?
}

close_app "[f]irefox"

It selects the app to kill by name or arguments. Keep the brackets for the first letter of the app name to avoid matching grep itself.

With some changes, you can directly use the PID or a simpler pidof process_name instead of the ps statement.

Code details: Final grep is to get the PID without the trailing spaces.

Answer 4

If what you want is to manage some background processes, why not to use bash job control features?

$ gedit &
[1] 2581
$ emacs &
[2] 2594
$ jobs
[1]-  Running                 gedit &
[2]+  Running                 emacs &
$ jobs -p
2581
2594
$ kill -2 `jobs -p`
$ jobs
[1]-  Interrupt               gedit
[2]+  Done                    emacs

Answer 5

So I tinkered around with this as well. In Bash, you can define functions and do fancy things with those. My coworkers and I use terminator, so for batch executions we normally spawn a whole bunch of terminator windows if we want to view the output (less elegant than tmux tabs, but you can use a more GUI-like interface haha).

Here's the setup that I came up with, as well as an example of stuff you can run:

#!/bin/bash
custom()
{
    terun 'something cool' 'yes'
    run 'xclock'
    terun 'echoes' 'echo Hello; echo Goodbye; read'
    func()
    {
        local i=0
        while :
        do
            echo "Iter $i"
            let i+=1
            sleep 0.25
        done
    }
    export -f func
    terun 'custom func' 'func'
}

# [ Setup ]
run()
{
    "$@" &
    # Give process some time to start up so windows are sequential
    sleep 0.05
}
terun()
{
    run terminator -T "$1" -e "$2"
}
finish()
{
    procs="$(jobs -p)"
    echo "Kill: $procs"
    # Ignore process that are already dead
    kill $procs 2> /dev/null
}
trap 'finish' 2

custom

echo 'Press <Ctrl+C> to kill...'
wait

Running it

$ ./program_spawner.sh 
Press <Ctrl+C> to kill...
^CKill:  9835
9840
9844
9850
$ 

Edit @Maxim, just saw your suggestion, that makes it a ton simpler! Thanks!