...
Here is a very simple job script which just does some variable substitutions, and then exits.<pre>
| Code Block |
|---|
#!/bin/bash |
...
#$ -S /bin/bash |
...
#$ -N job |
...
#$ -m bea |
...
echo this is a line of output # except for this |
...
#echo this is another line of output > output 2> error |
...
#echo this is yet another command > command-2 2>& error-2 |
...
# Create Working Directory |
...
WDIR=/state/partition1/$USER/$JOB_NAME-$JOB_ID |
...
echo $WDIR |
</pre>To make this task aware we use the qsub option -t which takes an argument form '''n-m:s'''. Here '''n''' is the starting index number which should be at least 1, '''m''' is the ending index, and '''s''' is a step value. For example,<pre>
| Code Block |
|---|
-t 3 |
...
-t 1-3 |
...
-t 1-10:5 |
...
|
are all legitmate uses of -t. Like all SGE options this can be permanently embedded into the script using the '''#$''' syntax. In response to the -t option SGE adds the variables '''$SGE_TASK_FIRST''', '''$SGE_TASK_LAST''', '''$SGE_TASK_STEP''', and '''$ SGE_TASK_ID'''.
Here is a job script that uses these variables<pre>
| Code Block |
|---|
#!/bin/bash |
...
#$ -S /bin/bash |
...
#$ -N job |
...
#$ -t 3 |
...
#$ -e err-$JOB_ID-$TASK_ID |
...
#$ -o out-$JOB_ID-$TASK_ID |
...
#$ -m bea |
...
echo this is a line of output # except for this |
...
#echo this is another line of output > output 2> error |
...
#echo this is yet another command > command-2 2>& error-2 |
...
# Create Working Directory |
...
WDIR=/state/partition1/$USER/$JOB_NAME-$JOB_ID-$SGE_TASK_ID |
...
echo $WDIR |
...
echo \$SGE_TASK_FIRST $SGE_TASK_FIRST |
...
echo \$SGE_TASK_LAST $SGE_TASK_LAST |
...
echo \$SGE_TASK_STEP $SGE_TASK_STEP |
...
|
Notice that some of these variables will be set to the value '''undefined''' if they are not set. '''$SGE_TASK_STEP''' is empty unless it is used. None of these have to be used, but as the example script shows you should use at least '''$SGE_TASK_ID''' to make your working directory into a task and job specific name. Other uses of these variables might be to select what a particular task does.
...
Here is a example showing the strength of array jobs, with a fragment of a script that 8 calculations on the same dataset:<pre>
| Code Block |
|---|
WDIR=/state/partition1/$USER/$JOB_NAME-$JOB_ID |
...
mkdir -p $WDIR |
...
if [ ! -d $WDIR ] |
...
then |
...
echo $WDIR not created |
...
exit |
...
fi |
...
cd $WDIR |
...
# Copy Data and Config Files |
...
cp $HOME/Data/Project/phylip.dat . |
...
cp $HOME/Data/Project/garli.tre . |
...
cp $HOME/Data/Project/garli_constraints . |
...
#Put your Science related commands here |
...
i=0 |
...
while [ $i -lt 8 ] |
...
do |
...
echo starting run number $i |
...
/share/apps/Garli0.96b8 garli_constraints |
...
tar cf run_$i.tar * |
...
i=$((i+1)) |
...
done # Copy Results Back to Home Directory RDIR=$HOME/Data/Project/Results/$JOB_NAME-$JOB_ID |
...
mkdir -p $RDIR |
...
cp * $RDIR |
...
This script loops 8 times running a garli job each time. This means this one job will take longer, and it's also much more complex. If you wanted to update the number of loops you have to update the script, and no provision is made for the fact that you might have runs 1-8 already done, but you need a second set of 8. The second version of this script handles these issues and is much cleaner:<pre>
| Code Block |
|---|
# Create Working Directory |
...
WDIR=/state/partition1/$USER/$JOB_NAME-$JOB_ID-$SGE_TASK_ID |
...
mkdir -p $WDIR |
...
if [ ! -d $WDIR ] |
...
then |
...
echo $WDIR not created |
...
exit |
...
fi |
...
cd $WDIR |
...
# Copy Data and Config Files |
...
cp $HOME/Data/Project/phylip.dat . |
...
cp $HOME/Data/Project/constraint.tre . |
...
cp $HOME/Data/Project/garli_constrain.com . |
...
# Put your Science related commands here |
...
/share/apps/Garli0.96b8 garli_constrain.com |
...
# Copy Results Back to Home Directory |
...
RDIR=$HOME/Data/Project/Results/$JOB_NAME-$JOB_ID/Run-$SGE_TASK_ID |
...
mkdir -p $RDIR |
...
cp * $RDIR |
...
The first thing to note is that this script did not require very much to modify from the single run version to the task array version. Really there are just a few additions to make sure a task specific working directory is created. Also, task specific results directories are created. Besides ensuring that different tasks don't overwrite each others files, this provides a mechanism for adding the number of runs. For example, if you had 10 runs, then it's easy to add 10 more, using -t 11:20 as the command line. The output will then nicely be stored into 10 new output files.
Of course, you could use the variable '''$SGE_TASK_ID''' to select multiple data sets rather than running over the same data set. One way to do this is to just have the multiple files each of which has a index number such as phylip-1.dat, phylip-2.dat, .... Then it is easy enough to use '''$SGE_TASK_ID''' to select what file the task should process. However, if your files are not labeled this way, it might be a burden to go renaming them. An alternative is to use a file like so<pre>
| Code Block |
|---|
cd ~/Data/Project |
...
ls * > INPUTS |
...
|
Replace the '''*''' on with some other file glob that matches the input names you wish to use. The job script should then copy the INPUTS file to the local working directory. Each task will then process the file on line '''$SGE_TASK_ID'''. This can be found using the following bit of shell magic<pre>
| Code Block |
|---|
file=`awk 'NR==n' n=$SGE_TASK_ID INPUTS` |
...
|
'''$file''' can now be used in your script in any location where you would have put a data set name. Note the use of the back tick ` here. The back ticks are a shell quoting mechanism that indicates you wish the shell to replace the back ticked quantity with the output of the command contained between the tick marks.
...