Major Tom, This is Job Control
By: Jay Fink
Thursday, July 27, 2000 10:03:08 AM EST
URL: http://www.linuxplanet.com/linuxplanet/tutorials/2116/1/
Acting Nice with the Linux System
Job control and scheduling is quickly becoming less and less of an issue for
most Linux users and administrators. Virtually all scheduling is done
(justifiably so) via cron. However, I have come across many
occasions of late where using job control and processing techniques is quite
handy. This article will look at two aspects of job control: first, priority
scheduling and altering priorities with nice and
renice, respectively; second, batch processing and scheduling
using batch, at, and cron, respectively.
Nice and Renice
Many UNIX users are very familiar with nice and
renice, and as they pertain to job control they deserve mention
within this context. The nice command is used to alter an initial
job priority. On Linux systems this is fairly simple: the lower the
nice command the higher the priority. The range on a Linux system
is -20 (being the highest) to 19 (the lowest). Using nice is
pretty simple. Let's say we want to make sure that a compile and install for
fetchmail has a pretty high priority. We might do the following:
nice -n 5 make
We have lowered the nice number and raised the job priority initially for this task.
The renice command is used to alter the nice value
of a job after it has been started. It is important to note that only
root may alter the nice value of jobs it does not own, and
non-root users may only alter their nice values between 0 and 20
(the former is obviously so users may not tamper with other users while the
latter protects the privileged processes of the system). An example of using
renice on a single process might look like so:
renice 5 -p 10023
where we lower the nice value to 5 of PID 10023. The
renice command can also affect entire group of processes as well.
For instance, let us say we wanted all of jdoe user's processes to have a
nice value of 12. The syntax on Linux would be:
renice 12 -u jdoe
Processing and Scheduling Commands
Many times a user will want to run a large job but not wish to lock up a
terminal waiting for it to finish. The fastest way to do this is to simply put
it in the background. This will only work well for programs that do not print
to STDOUT. For example, if I wanted to move a bunch of directories separately
and in the background, I could do this:
mv -f dirname_one new_dirname_one & mv -f dirname_two new_dirname_two &
The & is used to put the job in the background so my
terminal is free for me to keep working. The system will inform me when the
jobs are done. It is important not to logout while
background jobs are running. On most systems you will see a warning message
stating that there are running jobs. If this is ignored, the jobs
will be terminated. To see the current jobs running, just type
jobs and a list is printed out to the terminal. This output varies
from shell to shell.
There are instances when putting jobs in the background alone is not enough. You may wish to logout after the job is started. This is where batch processing or scheduling comes in handy.
It is important to note at this point that at and
batch are related on Linux systems and similarly implemented.
Batch Processing and Scheduling
Batch processing on Linux means to submit a job that will be run when the system
load permits. This is normally when /proc/loadavg is below 1.5 or
a value specified by atrun. An example usage of batch is as
follows where exec_file.sh is a script we want to run:
batch 1400 -f exec_file.sh
To check the status of a job the atq utility shows the status
of jobs using the at & batch facilities.
Using at is pretty easy but offers a lot of great options. A
simple example using the very same exec_file.sh would look like
this:
at -f exec_file.sh 1504
There are some interesting options that go with batch and
at. For instance, they both have a default queue priority. The
priority scheme is letters from a-z and A-Z. Queues with higher letters run
with higher priority. Capitalized queues are treated as if they were submitted
at that time. So, to raise the queue priority of our most recent submission, we
could do this:
at -f exec_file.sh -q g 1523
Finally, we can also have the facility mail us with the -m
option.
Again, to look at current jobs in the queue use atq. To remove
a job the atrm command is used. The job number must be specified.
For example, if we wanted to remove job 3:
atrm 3
Last and definitely not least is cron. The cron
facility is primarily for repetitive daily work such as backing up systems,
trimming logfiles, running certain applications or other such tasks. The
cron facility on many systems vary and have various ways of being
implemented. All of them, however, do follow one convention: if you are a
cron user, you can simply use the crontab -e command
to directly edit a file that will be submitted to crond.
The cron facility reads out the time information from a series
of fields on each line of the cronfile. If any of these fields are omitted, it
simply truncates from the right. The fields are as follows:
minutes_after_the_hour hours days_of_the_month months_of_the_year weekdays
In other words, to schedule exec_file.sh to run every Monday morning at 3AM our entry would look like so:
00 3 * * 1 /home/mydir/exec_file.sh
If there is any possibility of messages being output by a job, it is usually a good idea to redirect it somewhere:
00 3 * * 1 /home/mydir/exec_file.sh 2>&1
The weekdays start from 0; however, years and months do not.
While this article skimmed over a lot, it is not particularly difficult to get the hang of basic job handling and prioritizing. Just write a practice script and try out all of the scheduling commands.
Combining Commands
We all know Linux is about small, clean utilities and tools that work well
together. The aforementioned tools, commands and utilities are no different. As
an example, lets say I have two jobs that run overnight. The first job is a
tar job that makes a raw backup of my home filesystem. The second
is a cleaner script for the system. Additionally, for the sake of argument, we
will say during the night the system is doing downloads. Now we obviously want
the backup to have a pretty high priority. So our crontab might look like this:
0 2 * * * /usr/bin/nice -n 0 cd /home; tar -cf /dev/hdb myhome 2>&1 0 0 * * * /bin/rm -rf /var/tmp/* 2>&1
So we ensure that the tar job has a priority of 0. (The default priorities
are: c for at and E for batch.)
Here are some further examples of these commands in action.
nice
To start a job off with a nice priority of 3 under Linux:
nice 3 job_name
renice
To lower a job's nice number and hence raising its priority to 3
with a PID of 1253:
renice -n 3 -p 1253
simple background job submission
To enter commands into the background simply use the ampersand symbol:
mv -f dirname_one new_dirname_one & mv -f dirname_two new_dirname_two &
To check the number of jobs and their status use:
jobs
batch
To submit a script named exec_file.sh to the queues at 1400:
batch 1400 -f exec_file.sh
at
to submit a script to at named exec_file.sh at 1504:
at -f exec_file.sh 1504
cron
The cron fields are as follows from left to right (the top of the
list being the leftmost field):
- minutes_after_the_hour - hours - days_of_the_month - months_of_the_year - weekdays
To submit the file /home/myscripts/exec_file.sh to run at 3
a.m. every Monday:
00 3 * * 1 /home/myscripts/exec_file.sh 2>&1
Okay, Genius, When Does This Come in Handy?
The usage of cron is pretty obvious: Repetitive automation. I
have found batch processing to really be helpful when you have a
job to run repetitively--say, a hardware information gathering script, for
example. I have used at extensively to rerun cronjobs that may
have had a glitch or (as I mentioned earlier) simply to keep my terminal freed
up. If performance is not an issue and you wish to keep the terminal free,
just put the job(s) in the background.
With a little time and practice, using simple job control and batch processing can provide more productive time on your system.