Not sure if cron job is run

Question

I have the following cron job in /etc/cron.d/backup:

*/1 * * * * backupbot /home/backupbot/bin/backup-script.sh

Basically, I want the backup-script.sh to run every minute (and the user backupbot should be executing the job).

The /home/backupbot/bin/backup-script.sh file is owned by backupbot (and he has "x" permission on it). The file looks as follows:

#!/bin/bash
set -e

{

BACKUP_DIR=/var/app/backups
STORAGE_ACCOUNT_URL=https://myserver/backups

BACKUP_FILE=$(ls $BACKUP_DIR -t | head -1)

if [ -z "$BACKUP_FILE" ]; then
    echo "There are no backups to synchronize"
    exit 0
fi

azcopy login --identity
azcopy copy $BACKUP_DIR/$BACKUP_FILE $STORAGE_ACCOUNT_URL/$BACKUP_FILE

} >/tmp/cron.backup-script.$$ 2>&1

Normally, any output should be logged into /tmp/cron.backup-script.xxxx. Such a file is never created.

The only evidence that the job is being noticed by Cron is the following output of systemctl status cron.service:

● cron.service - Regular background program processing daemon
   Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2021-04-13 09:59:08 UTC; 6h ago
     Docs: man:cron(8)
 Main PID: 1086 (cron)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/cron.service
           └─1086 /usr/sbin/cron -f

Apr 13 16:00:01 my-vm CRON[17201]: pam_unix(cron:session): session closed for user root
Apr 13 16:00:01 my-vm CRON[17198]: pam_unix(cron:session): session closed for user root
Apr 13 16:00:01 my-vm CRON[17199]: pam_unix(cron:session): session closed for user root
Apr 13 16:01:01 my-vm CRON[17402]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 13 16:01:01 my-vm CRON[17403]: (root) CMD ([ -f /etc/krb5.keytab ] && [ \( ! -f /etc/opt/omi/creds/omi.keytab \) -o \( /etc/krb5.keytab -nt /etc/opt/omi/creds/omi.keytab \) ] &&
Apr 13 16:01:01 my-vm CRON[17401]: pam_unix(cron:session): session opened for user backupbot by (uid=0)
Apr 13 16:01:01 my-vm CRON[17404]: (backupbot) CMD (/home/backupbot/bin/backup-script.sh)
Apr 13 16:01:01 my-vm CRON[17402]: pam_unix(cron:session): session closed for user root
Apr 13 16:01:01 my-vm CRON[17401]: (CRON) info (No MTA installed, discarding output)
Apr 13 16:01:01 my-vm CRON[17401]: pam_unix(cron:session): session closed for user backupbot

It mentions something about sessions for backupbot. How can I investigate further?

One additional thing that is weird: when I run this script (backup-script.sh) on my local computer, where the `/var/app/backups` directory doesn't exist, I get the following log: `ls: cannot access '/var/app/backups': No such file or directory There are no backups to synchronize`. Shouldn't the script exit on the `ls` error already? I have the `set -e`... — mnj, Apr 13 '21 at 16:11
The subshell that runs `ls` does not have `set -e` enabled, and if it had, exiting the subshell would not terminate the script. The exit status of the command substitution is the exit status of `head`, and that runs successfully. — Kusalananda, Apr 13 '21 at 16:31
Since you say that the output file isn't created, it means that the script never executes to the `{ ...; }`. I.e., it never even starts (or terminates straight after `set -e`, which is terribly unlikely). Could you show the permissions and ownership of the file `/home/backupbot/bin/backup-script.sh`? — Kusalananda, Apr 13 '21 at 16:35
@Kusalananda Here's the metadata about `/home/backupbot/bin/backup-script.sh`: `---x------ 1 backupbot marcin 428 Apr 13 15:45 backup-script.sh*` — mnj, Apr 14 '21 at 06:27
So, you can't actually _read_ the script. Does it work if you make it readable by the owner? — Kusalananda, Apr 14 '21 at 07:14
Oh, I wouldn't think about it. I changed the permissions to "500". And I see the log in `/tmp`! Thanks! I wonder, isn't there any bultin logging mechanism that would tell me that corn can't even run the script? — mnj, Apr 14 '21 at 08:05
Normally, the cron daemon would send an email message to the owner of the job if the job produced errors or any other output. If you haven't set up local delivery of mail on your system, such emails would be discarded. I don't know how this is normally organized on Linux (I'm not a Linux user). — Kusalananda, Apr 14 '21 at 08:26
`(CRON) info (No MTA installed, discarding output)` -- having a working email system, even within the same host would help a lot in debugging cron, as you'd get any errors in the emails. What options there are to do that easily (and without exposing the SMTP server to the internet...) may depend on the distribution, though. Anyway, lacking that, an alternative would be to put redirections directly the crontab command, i.e. `*/1 * * * * backupbot /home/backupbot/bin/backup-script.sh > /tmp/crontest.out 2>/tmp/crontest.error`, or `> file 2>&1` to put both in the same file, or something like that — ilkkachu, Apr 14 '21 at 09:38

Kusalananda · Accepted Answer · 2021-04-14T08:49:29.550

According to comments, the script is only executable by the owner. I.e., it is not readable. This stops the owner from executing the script.

Example:

$ chmod 500 script
$ ls -l script
-r-x------ 1 myself myself 24 Apr 14 09:21 script
$ ./script
hello

$ chmod 100 script
$ ls -l script
---x------ 1 myself myself 24 Apr 14 09:21 script
$ ./script
/bin/bash: ./script: Permission denied

If the script file is not readable, it can't be read by the bash shell interpreter (which is running as the current user).

Make the script both executable and readable.

Apart from the above, consider using proper double quoting of variable expansions, and don't parse the output from ls (it's only really useful to look at).

Your script, modified:

#!/bin/bash

backup_dir=/var/app/backups
storage_account_url=https://myserver/backups

{
    # Find most recently modified file in "$backup_dir".
    # Assumes that there are only files there, no subdirectories.

    set -- "$backup_dir"/*
    backup_file_path=$1
    shift

    for pathname do
        if [ "$pathname" -nt "$backup_file_path" ]; then
            backup_file_path=$pathname
        fi
    done
    
    if [ ! -e "$backup_file_path" ]; then
        echo 'There are no backups to synchronize'
        exit 1
    fi
    
    # Perform backup.

    azcopy login --identity || exit 1
    azcopy copy "$backup_file_path" "$storage_account_url/$(basename "$backup_file_path")"

} >/tmp/cron.backup-script.$$ 2>&1

Great advices. Thank you very much! – mnj Apr 14 '21 at 08:06 — mnj, Apr 14 '21 at 08:06

score 1 · Answer 2 · answered Apr 13 '21 at 16:18

Jobs run through cron, or at, or batch, aren't run in the same runtime environment that you have on your desktop. None of your PATH changes, or other environment variable settings are automatically propagated to your cron job. For example, there's no $DISPLAY, so GUI programs need special treatment (read man xhost).

One can set environment variables for all one's cron jobs in the crontab file Read man 5 crontab.

Look at the results of echo "=== set ===";set;echo "=== env ===";env | sort;echo "=== alias ===";alias in each of your environments.

Since the command part of the crontab line is, by default, interpreted by /bin/sh, which has a simpler syntax than /bin/bash, I recommend having command be a call to a bash script (executable, mounted, starts with #!/bin/bash) which sets up the environment, then calls the desired program.

Always use absolute paths (paths starting at /, the root directory) in scripts run through cron, at, batch.

The script that I run through `cron` already starts with `#!/bin/bash`. Even if some command is unrecognized, all is inside of `{}`, and it is redirected to a file (with absolute path - `/tmp/cron.backup-script.$$`). So I suppose that any problem would be logged into this file, right? But it's not.. — mnj, Apr 14 '21 at 06:33

Not sure if cron job is run

2 Answers2