Friday, April 14, 2006

Linux Job Scheduling

How I learned to stop worrying and love the Cron.

Today, in our ongoing series on learning to live with Linux's ``inner dæmons'', we are going to look at two dæmons that schedule job execution on Linux. These dæmons are more or less exactly like those found on virtually every UNIX out there. (Linux has separate dæmons for at and cron. Old versions of Linux used a program called ``atrun'', which was run in root's crontab once a minute to execute at requests. Some other Unix operating systems have atd functionality directly in crond. This qualifier brought to you by the bureau of auctorial honesty. This article will cover atd and crond as they are distributed with most currently sold distributions, including Debian 2.1, Red Hat, SuSE and Corel, among others.) My test cases were all carried out on a Red Hat 6.1 installation using version 3.1.7 of at. Debian and SuSE versions I currently have are at 3.1.8.
Read More


As for cron, most Linux distributions use ``Vixie cron'' which was originally written, as you might guess, by Paul Vixie. The distributions have each done their own fixes to address a security hole discovered in August 1999. Check your distribution's update page for the most recent version of cron, and make sure you have it installed.

What you think about at and cron will largely depend on what your background is. If you are familiar with only the DOS and Windows world, you should be fairly impressed with what atd and crond offer, even if you have made use of the System Agent, which has certain similarities to crond. If you are an old hand from the world of MIS where you had JCL and various batch environment control systems, you will probably find atd and crond lacking in some essential features. Even so, I hope you will come away from this introduction with a healthy appreciation for what these tools do offer, and perhaps a few ideas about how, even with their limitations, they significantly enhance Linux's capabilities.

People with a mainframe background are very familiar with the concept of job scheduling. They usually use this term interchangably with batch processing. Alas, job scheduling is not batch processing. Batch processing, to my mind at least, includes the concepts of job dependencies, batch process monitoring, checkpoint/restart and recoverability. Neither atd nor crond provides these facilities. If you come from the world of big iron, you may be feeling some disappointment. Don't. As you will see, atd and crond fit in well with the overall UNIX philosophy of simple tools that do one thing well.

If you are coming from a Windows/DOS perspective, you should be pleased by the multi-user nature of atd and crond. Unlike System Agent, you do not have to be logged in for your jobs to be carried out.

If you have a UNIX background, well, you are amongst old friends here.

For those totally unfamiliar with these concepts, what we are talking about is running programs. So what, you say? I log in and type commands and click on little icons. I run programs all day. What's the big deal?

What about having programs run at a certain time of the day, whether you are there or not? What about compiling the latest version of WINE on a busy Linux server when it won't slow down the branch office Intranet? What about that annoying log file the on-line order application spits out that is about to eat up all the free disk space on /usr/prod/orders?

This is where job scheduling comes into play.

There are two kinds of scheduled jobs. You can think of them as ``one shot'' and ``repeating''. One-shot jobs are single executions of programs you want to have take place at some future time, whether or not you are logged in. Repeating jobs are programs you want to have run at certain times or dates, over and over again.

The command you use to schedule one-shot jobs is called ``at''. The way to schedule repeating jobs is through a ``crontab'' (which is a portmanteau word made from CRON TABle, similar to INITtialization TABle and other *nix-y portmanteau words). Oddly enough, the command used to view, edit and store crontabs is called ``crontab''.

Unlike some of the other dæmons we have covered in this series, these two have interactive user programs that control them. Because of this, we will cover the basics of using these two dæmons as a non-privileged user (I hope you aren't logging in to your Linux system as root!), then we will go over the dæmons and how they work, then we will cover some fine points of ``non-user'' or system-scheduled jobs, and finally some of the little ``gotchas'' that sometimes cause commands to behave differently than you expect when you run them through a scheduler.
Using at

The at command is used to schedule one or more programs for a single execution at some later time. There are actually four client commands:

1.

at: Runs commands at specified time
2.

atq: Lists pending commands
3.

atrm: Cancels pending jobs
4.

batch: Runs commands when system load permits

The Linux at command accepts a number of time specifications, considerably extending the POSIX.2 standard. These include:

HH:MM

Run at this hour and minute. If this is already passed, the next day is assumed. A 24-hour time is assumed, unless you suffix the time with ``am'' or ``pm''.

now noon midnight teatime

You read that right. You can type ``at teatime'', and Linux's at is civilized enough to know that this is 4 p.m. local time. The ``noon'' and ``midnight'' keywords have their normal meaning. The ``now'' keyword means what it says. It might seem like a dumb thing to have, since if you wanted to run something now, you would type it without the at command, but it has an application in ``relative time'' invocations. We'll see those after the date modifiers described below.

Date Modifiers

These time specifications may be optionally followed by a date specification. Date specifications come in a number of forms, including:

today tomorrow

These mean what you would expect. ``at teatime tomorrow'' will run the commands at 4 p.m. the following day. Note that if you specify a time already passed (as in ``at noon today'' when it is 3 p.m.), the job will be run at once. You do not get an error. At first you might think this a bad thing, but look at it this way. What if the system had been down since 10 a.m. and was only being restarted now at 3 p.m.? Would you want a critical job skipped, or would you want it to run as soon as possible? The at system takes the conservative view and assumes you will want the job run.

[]

where month_name is ``jan'' or ``feb'', etc., and day is a day number. The year is optional, and should be a four-digit year, of course.

MM/DD/YYYY YYYY-MM-DD

Don't listen to what the ``man at'' page tells you! At least in Red Hat 6.1, it is wrong! I suspect it is wrong in certain other releases as well, and I'm willing to bet this is because the documentation has not caught up with Y2K fixes to this subsystem. The at shipped with Red Hat 6.1 handles dates in the two formats above. It appears to handle 2-digit years correctly, turning values less than 50 into 20xx and those greater than 50 into 19xx. I did not test to find the exact pivot point, and I do not recommend that you bother to, either. If you use two-digit years at this point, be prepared to pay a price! Depending on your version of at to treat two-digit years a certain way is foolish. Use four-digit years. Haven't we learned our lesson? (If you worked with computers from 1995 to 1999, you felt the pain as work came to an almost complete halt while we pored over every system with microscopes, looking for date flaws in the designs of our systems. Don't make a Y2.1K problem! PLEASE!!!)

Relative Time

Another way you can modify a time specification is to apply a relative time to it. The format of a relative time specification is +

No comments: