0

If on a debian 7 server (wheezy) running in a hosted VM and an up-to-date cron daemon there is a bug so cron stopped running without obvious reason. Since this only happened once ever since, it is hard to debug.

How can you ensure cron automatically restarts if it should crash again and sends an email alert?

rubo77
  • 27,777
  • 43
  • 130
  • 199
  • 4
    Is this Debian wheezy or jessie (using systemd), or an earlier release (SysV) ? – Jeff Schaller May 17 '16 at 10:53
  • Technically, you could use `at` if the command you schedule runs another `at` itself, but you really really want to fix your `cron` problem instead. – Ulrich Schwarz May 17 '16 at 11:12
  • How can I fix the cron Crash problem if there is no sign, why and how often it crashes? Only solution for me would be a fresh install – rubo77 May 17 '16 at 12:19
  • Couldn't i run something like daemontool to monitor and restart the process? And send a mail to get informed so i can search for the reason? – rubo77 May 17 '16 at 12:21
  • 1
    If something as stable and reliable as crond started crashing "without reason", I'd suspect: 1. hardware fault(s), 2. the kernel's out-of-memory killer (do you have any processes or cron jobs that use an enormous amount of RAM?), 3. some rogue process killing crond. – cas May 18 '16 at 01:35
  • What version of debian? and what version of cron (and the libraries it depends on)? don't post your reply in a comment, edit your question and add the info there, using the `{}` icon to highlight code, commands and command output. – cas May 18 '16 at 01:38

4 Answers4

1

Couldn't I run something like daemontools to monitor and restart the process?

Yes, indeed; and on some machines I do exactly that. The "something like daemontools" is actually the service manager from the nosh package but other members of the daemontools family are more than capable of supervising GNU cron. (Vixie cron is another matter, but you did say Debian.)

GNU cron is one of the simplest services to run under daemontools family service management. Gerrit Pape's run script collection has it, as does the service bundle collection that accompanies the nosh toolset.

That said, I cannot recall ever having needed to restart GNU cron because it crashed.

On the gripping hand, service management is not just about auto-restarting. It is also about logging and resource control, both of which are relevant to the task of diagnosing why GNU cron is crashing on you.

Diagnosing the problem would involve things such as:

  • Editing the run program to use softlimit to enable core dumps.
  • Editing the restart script (or equivalent) …
    • … to check whether the dæmon exited or terminated with a signal.
    • … to collect core dumps.
    • … to raise alerts and to mail notifications. (I once set up a restart script to mail me the last few lines of log/main/current on a crash/abort.)
    • … to tune restart rate-limiting.
  • Reading GNU cron's individual log, and the service manager's own log, to determine when and how often the dæmon has restarted, and what (if any) error messages it has output.

Further reading

Community
  • 1
JdeBP
  • 66,967
  • 12
  • 159
  • 343
  • Which implementation are you referring to as GNU cron? [Mcron](http://www.gnu.org/software/mcron/)? The cron in Debian is a (heavily-patched) [Vixie cron](https://tracker.debian.org/media/packages/c/cron/copyright-3.0pl1-128). – Stephen Kitt May 19 '16 at 05:19
1

You can always check out the monit project.
You can restart services and keep them up with it.
If there is no way for you to fix your cron as comments suggested.

Hristo Mohamed
  • 133
  • 1
  • 8
-1

Since one cannot monitor crond via crond i'd make it this way:
echo "while true; do if ! (ps aux |grep crond |grep -v grep); then /etc/init.d/crond start; fi && sleep 5; done &" >> /etc/rc.local

user1700494
  • 2,172
  • 8
  • 13
-1

No need to install daemontools, runit, supervise, etc. As useful as these tools are, they cover use-cases you generally are not in need of for only cron. What you simply need can be handled at ease with init. Add to /etc/inittab:

cron:2345:respawn:/usr/sbin/crond -n

Make sure crond supports the -n option first. This tells it not to fork and to remain in the foreground. ** Be sure to disable crond from the rc scripts **.

/usr/lib/lsb/remove_initd /etc/init.d/crond

If, for some reason, crond outputs to stdout or stderr, you will need to write a wrapper script to deal with that output, and spawn the wrapper script. Keep that script simple:

#!/bin/sh
#crond-wrapper.sh 
exec crond -n &>>/var/log/crond

Alternatively, you can modify the existing packaged init.d/crond script with one that invokes crond -n within a while loop. But in that case, you have to be clever about saving the pid for later use by this script.

Otheus
  • 5,945
  • 1
  • 22
  • 53
  • Will this be update safe? – rubo77 May 18 '16 at 11:54
  • Favouring `/etc/inittab` over proper service management is rather foolish in 2016. [`/etc/inittab` is a thing of the past](http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/inittab-is-history.html); and, for the reasons that rubo77 alludes to, using it won't even survive an upgrade to the current Debian 8. The ever-growing logfile is equally as bad an idea in 2016. We've long-since learned better than that, too. – JdeBP May 21 '16 at 10:45
  • @rubo77 I claim no knowledge of Debian's update path, but as long as your system is running sysvinit, it is update-safe. If you upgrade versions of Debian, chances are it will come packaged with a more advanced process management system, such as `upstart` or `systemd`. At least with systemd, this problem will solve itself -- it will make sure crond stays running for you (at least, that's a capability systemd offers). – Otheus May 24 '16 at 11:24
  • `JdeBP` Regardless of what year it is, a working system that works is valuable when the alternatives offer very little benefit and require an additional learning curve on the part of the user. In this case, for only ensuring `crond`, using the existing sysvinit is more useful than installing a more complex system which the user has no experience with, or a script which itself may be buggy and turn into a resource hog. An upgrade to a non-sysvinit based system will surely solve his `crond` problem, but then create numerous other transition challenges. – Otheus May 24 '16 at 11:37