Friday, June 21, 2013

@reboot jobs will be run at computer's startup.

"@reboot jobs will be run at computer's startup."
What on earth does that mean?

These days RedHat use cronie as the system cron daemon. Described as 'based on the original cron' I think it is a fork of vixi-cron which was used until EL5.

For some time, both of these cron daemons have had an @reboot syntax which allows you to run scripts at (more or less) boot time (when the cron daemon is started). This allows users to start long running processes without the sysadmin having to write an initscript.

It also happens that from time to time, the cron daemon crashes. This is not ideal because cron is the very tool which can be used to periodically confirm that a daemon has not crashed. For now I have added a check to puppet to ensure that the cron daemon is running.

When the cron daemon is started, it logs some messages, one of which is the cryptic:
@reboot jobs will be run at computer's startup.
message. I understand that it is trying to tell me something but it fals short of conveying the message. The internet did not have much to say on the topic either so I had to resort to the source code.

The source of the message is from within the run_reboot_jobs function. The function first checks the existence of a (so called) lock file. The file is
/var/run/cron.reboot
If this file is present, the message is printed out and none of the @reboot jobs are run. If the file is absent, it is created and then the @reboot jobs are queued up to be run.

Perhaps the message should read:
Lock file /var/run/cron.reboot present. @reboot jobs have already been run. skipping.

So that is the mystery almost solved. The remaining details are that during boot, the rc.sysinit script removed a number of stale lock files, including the contents of /var/run. This ensures that at boot time, the cron daemon runs the @reboot jobs.

If you wanted to re-run the @reboot jobs without rebooting your server, you can easily trick it with:
rm /var/run/cron.reboot
service crond restart

Perhaps that could also be added to the init script so you run
service crond restart-boot