...
If a program does not sanity check its operations and data, it can sometimes proceed for many subsequent steps until finally either a bad (or empty) result is generated, or some called program that does sanity check its data, notices the problem, and terminates execution. It is then challenging to backtrack to where the original, causative error occurred – assuming there are even log files available to examine!
...
There are mechanisms in bash that provide some of this functionality (see especially set -e and set -x described in https://www.turnkeylinux.org/blog/shell-error-handling and http://linuxcommand.org/lc3_wss0150.php, and trap-ing signals in http://linuxcommand.org/lc3_wss0150.php).
...
| Code Block | ||
|---|---|---|
| ||
# Sets up auto-logging to a log file in the current directory
# using the specified logFileTag (arg 1) in the log file name.
auto_log() {
local logFileTag="$1"
local logFilePath="./autoLog_${logFileTag}.log"
check_arg_not_empty "$logFileTag" 'logFileTag'
exec 1> >(tee "$logFilePath") 2>&1
check_res $? "Logging to '$logFilePath'"
} |
Some pre-built useful functions
In writing a number of bioinformatics support scripts, both command-processing scripts and other pipeline scripts, I've developed a set of useful functions in a file called cbb_common_functions.sh. It contains error checking functions like those above, as well as a number of other useful general-purpose functions.
If you have access to a BRCF pod, such as the GSAF pod we've been using, this file is available from any compute server as: /mnt/bioi/script/cbb_common_functions.sh.
If you work on one of the TACC compute clusters, such as ls6, frontera or stampede3, the file is available in the shared Work file system as: /work/projects/BioITeam/common/script/cbb_common_functions.sh.
And here's a copy if you want to download it: cbb_common_functions.sh.
That ../script/ directory in both places also has a number of useful bioinformatics pipeline scripts I've written, that will run both on BRCF pods and at TACC, including:
- align_bwa_illumina.sh - full NGS alignment pipeline using BWA
- align_bowtie2_illumina.sh - full NGS alignment pipeline using Bowtie2
- run_macs2.sh - pipeline to run the MACS2 peak-calling program
All these scripts use another file of functions in the ../script/ directory, common_functions.sh. It contains all the functions in cbb_common_functions.sh, as well as functions to perform work for the pipeline scripts; for example, to perform a BWA alignment. While there are parts of cbb_common_functions.sh that are specific to the TACC or BRCF pod environment, you might find it useful for building your own scripts.