...

idev  -m 240 -r CCBB_Bio_Summer_School_2016_Day1 -A UT-2015-05-18 -N 12 -n 8
 
# waitThis should forreturn yourthe commandfollowing:
prompt# to comeWe backfound cd $SCRATCH/BDIB_breseq_tutorial_1
breseq -j 24 -r lambda.gbk lambda_mixed_population.fastq > log.txt &

A bunch of progress messages will stream by during the breseq run which would clutter the screen if not for the redirection to the log.txt file. The & at the end of the line tells the system to run the previous command in the background which will enable you to still type and execute other commands while breseq runs. The output text details several steps in a pipeline that combines the steps of mapping (using SSAHA2), variant calling, annotating mutations, etc. You can examine them by peeking in the log.txt file as your job runs using tail -f log.txt. The -f option means to "follow" the file and keep giving you output from it as it gets bigger. To stop the tailing command hit ctrl-c which is the keyboard interrupt signal. While breseq is running lets look at what the different parts of the command are actually doing:

Looking at breseq predictions

breseq will produce a lot of directories beginning 01_sequence_conversion, 02_reference_alignment, ... Each of these contains intermediate files that can be deleted when the run completes, or explored if you are interested in the inner guts of what is going on. More importantly, breseq will also produce two directories called: data and output which contain files used to create .html output files and .html output files respectively. The most interesting files are the .html files which can't be viewed directly on lonestar. Therefore we first need to copy the output directory back to your desktop computer.

tar -czvf output.tar.gz output  # the czvf options in order mean Create, Zip, Verbose, Force
pwd

scp -r <username>@ls5.tacc.utexas.edu:<the_directory_returned_by_pwd>/output.tar.gz .
tar -xvzf output.tar.gz  # the new "x" option at the front means eXtract 

Navigate to the output directory in the finder and open the a file called index.html. This will open the results in a web browser window that you can click through different mutations and other information and see the evidence supporting it. The summary page provides useful information about the percent of reads mapping to the genome as well as the overall coverage of the genome. The Mutation Predictions page is where most of the analysis time is spent in determining which mutations are important (and more rarely inaccurate).

Feel free to click around through the different mutations and examine their evidence when you have time, but first start the next breseq run so that it can finish running while you look at the data. We will go over the different types of mutations and the evidence for them as a group towards the end of class today, but additional information on analyzing the output can be found at the following reference:

• Deatherage, D.E., Barrick, J.E.. (2014) Identification of mutations in laboratory-evolved microbes from next-generation sequencing data using breseq. Methods Mol. Biol. 1151:165-188. «PubMed»

Example 2: E. coli data sets

Now we'll try running breseq on some Escherichia coli genomes from an evolution experiment. These files are larger. 

Data

The data files for this example are in the following path. Go ahead and copy them to a new folder in your $SCRATCH directory called BDIB_breseq_tutorial_2:

$WORK/GVA2016/ecoli_clones/data

mkdir $SCRATCH/BDIB_breseq_tutorial_2
cp $WORK/GVA2016/ecoli_clones/data/* $SCRATCH/BDIB_breseq_tutorial_2
cd $SCRATCH/BDIB_breseq_tutorial_2

if everything worked correctly, you should see the following files. We've provided a bit more context to what those files actually are:

Running breseq on TACC

breseq will take a little longer to run on these sequences, so this give us an opportunity to run several commands at the same time making use of the multiple cores on a single processor. You'll want each command (line) in the commands file to look something like this:

breseq -j 3 -r NC_012967.1.gbk -o output_<XX>K SRR030252_1.fastq SRR030252_2.fastq &> <XX>K.log.txt

Notice we've added some additional options:

breseq -j 3 -r NC_012967.1.gbk -o output_00K SRR030252_1.fastq SRR030252_2.fastq &> 00K.log.txt &
breseq -j 3 -r NC_012967.1.gbk -o output_02K SRR030253_1.fastq SRR030253_2.fastq &> 02K.log.txt &
breseq -j 3 -r NC_012967.1.gbk -o output_05K SRR030254_1.fastq SRR030254_2.fastq &> 05K.log.txt &
breseq -j 3 -r NC_012967.1.gbk -o output_10K SRR030255_1.fastq SRR030255_2.fastq &> 10K.log.txt &
breseq -j 3 -r NC_012967.1.gbk -o output_15K SRR030256_1.fastq SRR030256_2.fastq &> 15K.log.txt &
breseq -j 3 -r NC_012967.1.gbk -o output_20K SRR030257_1.fastq SRR030257_2.fastq &> 20K.log.txt &
breseq -j 3 -r NC_012967.1.gbk -o output_40K SRR030258_1.fastq SRR030258_2.fastq &> 40K.log.txt &

chmod +x commands  # makes the commands file executable
./commands  # executes the commands file

an ACTIVE reservation request for you, named CCBB_Bio_Summer_School_2016_Day1.
#  Do you want to use it for your interactive session?
#  Enter y/n [default y]: 


# If for any reason you don't see the above message let us know by raising your hand.
 
# Your answer should be y, which should return the following:
#  Reservation      : --reservation=CCBB_Bio_Summer_School_2016_Day1 (ACTIVE)
 
# Some of you may see a new prompt stating something like the following:
# We need a project to charge for interactive use.
# We will be using a dummy job submission to determine your project(s).
# We will store your (selected) project $HOME/.idevrc file.
# Please select the NUMBER of the project you want to charge.\n
# 1 OTHER_PROJECTS
# 2 UT-2015-05-18
# Please type the NUMBER(default=1) and hit return:
 
# If you see this message, again let us know.
 
# You will then see something similar to the following:
# job status:  PD
# job status:  R
# --> Job is now running on masternode= nid00032...OK
# --> Sleeping for 7 seconds...OK
# --> Checking to make sure your job has initialized an env for you....OK
# --> Creating interactive terminal session (login) on master node nid00032.
 
# If this takes more than 1 minute get our attention, otherwise continue with the following
cd $SCRATCH/BDIB_breseq_tutorial_1
module load intel
module load Rstats
breseq -j 48 -r lambda.gbk lambda_mixed_population.fastq &> log.txt &

A bunch of progress messages will stream by during the breseq run which would clutter the screen if not for the redirection to the log.txt file. The & at the end of the line tells the system to run the previous command in the background which will enable you to still type and execute other commands while breseq runs. The output text details several steps in a pipeline that combines the steps of mapping (using SSAHA2), variant calling, annotating mutations, etc. You can examine them by peeking in the log.txt file as your job runs using tail -f log.txt. The -f option means to "follow" the file and keep giving you output from it as it gets bigger. To stop the tailing command hit ctrl-c which is the keyboard interrupt signal. While breseq is running lets look at what the different parts of the command are actually doing:

This will finish very quickly (likely before you begin reading this) with a final line of "Creating index HTML table...". Before we start looking at what mutations were actually identified, we are going to start the second example so that it can be running while we investigate it making better use of our time.

Example 2: E. coli data sets

Now we'll try running breseq on some Escherichia coli genomes from an evolution experiment. These files are larger. 

Data

The data files for this example are in the following path. Go ahead and copy them to a new folder in your $SCRATCH directory called BDIB_breseq_tutorial_2:

$WORK/GVA2016/ecoli_clones/data

mkdir $SCRATCH/BDIB_breseq_tutorial_2
cp $WORK/GVA2016/ecoli_clones/data/* $SCRATCH/BDIB_breseq_tutorial_2
cd $SCRATCH/BDIB_breseq_tutorial_2

if everything worked correctly, you should see the following files. We've provided a bit more context to what those files actually are:

Running breseq on TACC

breseq will take a little longer to run on these sequences, so this give us an opportunity to run several commands at the same time making use of the multiple cores on a single processor. You'll want each command (line) in the commands file to look something like this:

breseq -j 12 -r NC_012967.1.gbk -o output_<XX>K SRR030252_1.fastq SRR030252_2.fastq &> <XX>K.log.txt

Notice we've added some additional options:

breseq -j 12 -r NC_012967.1.gbk -o output_00K SRR030252_1.fastq SRR030252_2.fastq &> 00K.log.txt &
breseq -j 12 -r NC_012967.1.gbk -o output_02K SRR030253_1.fastq SRR030253_2.fastq &> 02K.log.txt &
breseq -j 12 -r NC_012967.1.gbk -o output_05K SRR030254_1.fastq SRR030254_2.fastq &> 05K.log.txt &
breseq -j 12 -r NC_012967.1.gbk -o output_10K SRR030255_1.fastq SRR030255_2.fastq &> 10K.log.txt &
breseq -j 12 -r NC_012967.1.gbk -o output_15K SRR030256_1.fastq SRR030256_2.fastq &> 15K.log.txt &
breseq -j 12 -r NC_012967.1.gbk -o output_20K SRR030257_1.fastq SRR030257_2.fastq &> 20K.log.txt &
breseq -j 12 -r NC_012967.1.gbk -o output_40K SRR030258_1.fastq SRR030258_2.fastq &> 40K.log.txt &

chmod +x commands  # makes the commands file executable
./commands  # executes the commands file

Looking at breseq predictions

breseq produced a lot of directories beginning 01_sequence_conversion, 02_reference_alignment, ... Each of these contains intermediate files that can be deleted when the run completes, or explored if you are interested in the inner guts of what is going on. More importantly, breseq will also produce two directories called: data and output which contain files used to create .html output files and .html output files respectively. The most interesting files are the .html files which can't be viewed directly on lonestar. Therefore we first need to copy the output directory back to your desktop computer. Go back to the first tutorial (BDIB_breseq_tutorial_1) and transfer the contents of the output directory back to your local computer.

cd $SCRATCH/BDIB_breseq_tutorial_1
tar -czvf output.tar.gz output  # the czvf options in order mean Create, Zip, Verbose, Force
pwd

scp -r <username>@ls5.tacc.utexas.edu:<the_directory_returned_by_pwd>/output.tar.gz .
 
# Enter your password and wait for the file transfer to complete
 
tar -xvzf output.tar.gz  # the new "x" option at the front means eXtract 

Navigate to the output directory in the finder and open the a file called index.html. This will open the results in a web browser window that you can click through different mutations and other information and see the evidence supporting it. The summary page provides useful information about the percent of reads mapping to the genome as well as the overall coverage of the genome. The Mutation Predictions page is where most of the analysis time is spent in determining which mutations are important (and more rarely inaccurate).

Click around through the different mutations and examine their evidence to see what kinds of mutations you can identify. Interact with your instructors, and show us what different types of mutations you are able to identify, or ask us what mutations you don't understand. Periodically check the 2nd example and when it is done, transfer that data back and begin to look at the data as well. Additional information on analyzing the output can be found at the following reference:

• Deatherage, D.E., Barrick, J.E.. (2014) Identification of mutations in laboratory-evolved microbes from next-generation sequencing data using breseq. Methods Mol. Biol. 1151:165-188. «PubMed»

Examining breseq results

...