Periodic scripts which use a common resource can end up being configured to run at the same time for multiple hosts.

The result is that tasks like puppet runs or backups can take longer since they are trying to run all at once instead of being staggered.

One way to do this is to give each cron task unique times manually.

Another way is to add a random sleep before the task runs.

My preferred way is to use the fqdn_rand function in puppet. Here’s how it works.

The fqdn_rand function creates a random number between 0 and whatever you pass in based on the fqdn of the host. This means that for the same supplied number with the same hostname you get the same number back.

For a puppet cron job for instance I could do

cron { "puppet":
  ensure  => present,
  command => "/usr/bin/puppet agent --onetime --no-daemonize --logdest syslog > /dev/null 2>&1",
  user    => 'root',
  minute  => [fqdn_rand(30), fqdn_rand(30) + 30]

Which would give me a cron job which runs twice an hour but would reduce the chance of 2 running at the same time.

The advantage to doing it this way is that you can view the crontab and see when the cron job is actually scheduled to run.

# Puppet Name: puppet
18,48 * * * * /usr/bin/puppet agent --onetime --no-daemonize --logdest syslog > /dev/null 2>&1

But the number will be slightly different between hosts so you wont have them all try to run at once.