How can I alert on completion of a long task over ssh?

Question

My desktop is Ubuntu, which has the handy program notify-send, which pops up an alert on the desktop. It also has the following handy alias built in:

$ type alert
alert is aliased to `notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e 's/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//')"'

This means that you can run very-long-running-command; alert, leave the terminal running in the background, and receive a notification when the task is complete.

When you’re on a CentOS server through ssh, things are a little trickier. Here’s one way to do it:

localhost$ ssh [email protected]; alert
example.net$ very-long-running-command; exit

This works, after a fashion, but exiting after every command is not generally what I want to do.

My computer doesn’t have speakers. It can beep.

Answer 1

The simplest thing I can imagine here is to use a second SSH session to forward a named FIFO. I'm assuming you have a tmp/ in your home dir; feel free to keep the FIFO wherever you like.

local$ ssh me@remotehost 'mkfifo ~/tmp/alert_fifo ; while cat ~/tmp/alert_fifo ; do : ; done' | \
    while read icon_name text ; do
        notify_send --urgency=low -i "$icon_name" "$text"
    done &

Then you can let this run in the background while you open a second SSH session to do your real work:

local$ ssh me@remotehost
remote$ alias remote_alert='echo ... >~/tmp/alert_fifo'
remote$ long_running_command; remote_alert

...where remote_alert is a modified alert alias with notify_send --urgency=low -i replaced with echoing things to the FIFO.

This will work with a minimal set of tools on the remote machine: only ssh and standard POSIX utilities. However, with a FIFO, if you forget to run your reader (or your reader dies), then your writer will hang. So if you happen to have socat available, you can make this a little more forgiving:

local$ ssh me@remotehost 'socat UNIX-RECV:~/tmp/alert_socket -' | \
    while read ...
        ...
    done &

local$ ssh me@remotehost
remote$ alias remote_alert='echo ... | socat - UNIX-SEND:~/tmp/alert_socket'

Answer 2

You can install the terminal emulator terminator. Then right click on your terminal and select "watch for activity" or "watch for silence". This will notify you as soon as output output is produced in the terminal or no output is produced for some time, respectively. If your command does not produce output, you can just append something like ; echo "{!1} Done!" to the end of your command. I had built an extension for terminator that notified me everytime a specific keyword in the output occurred. Though I lost the code. The programming is fairly straight forward if you built on the existing extensions and are versed in Python. The added benefit is that it works the same even if you tunnel through a chain of ssh connections.

Answer 3

This is a partial answer. See D-Bus authentication and authorization for points that I'm stuck on.

notify-send sends a message over D-Bus. Modern desktop environments launch a D-Bus bus per session and arrange for the programs in the session to find the right bus, generally by setting the environment variable DBUS_SESSION_BUS_ADDRESS. D-Bus supports several ways of connecting to the daemon that centralizes and dispatches messages, including abstract sockets, unix sockets and TCP.

OpenSSH ≥6.7 can forward Unix sockets, but as I write, very few systems are running such a recent version.

You can run a D-Bus daemon that listens over TCP (you can make it listen on multiple addresses, e.g. both TCP and a Unix socket). That may be difficult to arrange if dbus-daemon is launched by your session startup scripts in a way you can't influence.

You can forward the D-Bus abstract socket or named Unix socket over TCP with a tool such as netcat or socat. Here's a proof-of-concept script that sets up forwarding on TCP port 8004.

#!/bin/sh
case $DBUS_SESSION_BUS_ADDRESS in
  '') echo 1>&2 "No local D-Bus instance";;
  unix:abstract=*,guid=*)
    guid=${DBUS_SESSION_BUS_ADDRESS##*[:,]guid=}
    guid=${guid%%,*}
    socket=${DBUS_SESSION_BUS_ADDRESS##*[:,]abstract=}
    socket=ABSTRACT-CONNECT:${socket%%,*}
    ;;
  unix:path=*,guid=*)
    guid=${DBUS_SESSION_BUS_ADDRESS##*[:,]guid=}
    guid=${guid%%,*}
    socket=${DBUS_SESSION_BUS_ADDRESS##*[:,]path=}
    socket=UNIX-CONNECT:${socket%%,*}
    ;;
  *) echo 1>&2 "Unsupported DBUS_SESSION_BUS_ADDRESS";;
esac
socat "TCP-LISTEN:8004,reuseaddr,fork,range=127.0.0.1/32" "$socket"

You can now forward this TCP connection over SSH.

ssh -R 8004:localhost:8004 [email protected]
[email protected]$ export DBUS_SESSION_BUS_ADDRESS=tcp:host=127.0.0.1,port=8004

You need to do one more thing: copy the authorization cookie to the remote machine.

rsync -a .dbus-keyrings/org_freedesktop_general [email protected]:.dbus-keyrings/org_freedesktop_general

Beware that unlike the X11 cookie, the D-Bus cookie can change during the lifetime of the server. In fact, in my experiments, it seemed to change at irregular intervals, sometimes after only a few minutes.

Only recent versions of Gnome libraries read the cookie file, as far as I can tell. notify-send does read it on FreeBSD 10.1 with Gnome 3.14 but not on Debian wheezy with Gnome 3.4.