Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The use of task arrays is one way to have a single job script which processes several data sets. Of course, you could just submit the job script several times, but this puts a burden on the head node. Also, you might have to edit the script to make it data set specific. You could also write your job script to execute in a loop, although again if the loop value changes, then you would again have to edit the job each time you submit it. Task arrays make this much nicer as 1 job is submitted with a certain number of tasks specified. SGE then takes care of partitioning the threads out to individual nodes. If you have to remove the tasks, there is just the one job idea which has to be removed. Additionally, SGE puts a new variable name in to the environment of each job, so that they the script can be made more generic.

...