Page Comparison

A healthy taste of resources available, specifically for this course - not a comprehensive catalog.

Linux/TACC

• Linux fundamentals on this wiki
• Cloud recording of CBRS's Intro to Unix short course, taught by Benni Goetz:
  • https://utexas.zoom.us/rec/play/5co1OfvhN5650hOccTNzANod0hU7h-b8cFvCos8Y1Q_LTO3_PYfMfiQ6PjxaeYSJIwk4_8Sxag33qbMt.9qt5joWzzGbgRTSy?continueMode=true
• Cloud recording of CBRS's Intro to TACC short course, taught by Benni Goetz:
  • https://utexas.zoom.us/rec/share/x1zCXQg-nB1HZYzgnXb8VK44J0bSkxHQr1evth08tJEeqtfeGk366fVMtrV9NEVv.zoUqgtYhGTy2-FEv

• Online tutorials:

  • Ryan's Linux Tutorial: http://ryanstutorials.net/linuxtutorial/
  • Unix and Perl for Biologists: bootcamp for biologists: http://korflab.ucdavis.edu/bootcamp.html
  • Unix primer (longer version) for biologists:
    • http://korflab.ucdavis.edu/
    unix
    • Unix_and_Perl/unix_and_perl_v2.3.4.pdf

Community Resources

• SEQAnwers forum - many NGS sequencing questions answered here
  • A funny SEQAnwers post about biologists starting to analyze NGS data: http://seqanswers.com/forums/showthread.php?t=4589
• UCSC Genome Browser - visualize and download NGS data (see more below)Galaxy website for online sequencing data analysis
• Broad Institute Integrated Genomcs Viewer (IGV)
  
  • especially good for visualizing BAM file details
• Introduction to Sequence analysis in the Amazon EC2 cloud
  
  • where you can "rent" Linux machines (useful if you don't have access to TACC)
• Galaxy website for online sequencing data analysis

Sequencing Technologies

• Overviews
  
  • Wikipedia overview of NGS technologies
  • Broad Center GA Boot Camp
  • Early paper comparing NGS technologies (Liu et al., 2012)

• Technology intros

  • Illumina (Solexa) – most common "short" (< 300 bp) read sequencing
    

    • Short Illumina video

    • Longer Illumina video
    • 2016 Sequencing Technologies talk
  • Newer single molecule sequencing
    • Oxford Nanopore
    • PacBio SMRT system
      • PCR-free protocol
  • Single cell sequencing
    • 10x Genomics platform
  • Older technologies (less common now)
    

    • Life Technologies SOLiD (short reads in "colorspace")

    • Roche/454 – long (multi-Kb) reads often used in assemblies

...

• Wikipedia FASTQ format page
• Illumina library construction on GSAF user wiki - useful for contaminant detection or adapter removal
• FastQC from Babraham Bioinformatics – http://www.bioinformatics.babraham.ac.uk/projects/fastqc/
  
  • produces nice quality report for FASTQ files
• MultiQC – http://multiqc.info/
  
  • A great tool for consolidating QC multiple QC reports into one HTML page
  • Anna's Byte Club tutorial on using MultiQC – https://wikis.utexas.edu/display/bioiteam/Using+MultiQC
  • Available on ls5 at /work/projects/BioITeam/ls5/opt/multiqc-1.0/multiqc
    • also needs this $PYTHONPATH modification:
      export PYTHONPATH="/work/projects/BioITeam/ls5/lib/python2.7/annab-packages:$PYTHONPATH"
• cutadapt – https://cutadapt.readthedocs.io/en/stable/
  • An excellent command line tool for adapter sequence removal
  • Good support for trimming paired-end datasets
  • Available on ls5 at /work/projects/BioITeam/ls5/opt/cutadapt-1.10/cutadapt
    • also needs this $PYTHONPATH modification:
      export PYTHONPATH="/work/projects/BioITeam/ls5/lib/python2.7/site-packages:$PYTHONPATH"
  • Script that handles the details of paired-end read trimming
    
    • /workwork2/projects/BioITeam/common/script/trim_adapters.sh
• trimmomatic – http://www.usadellab.org/cms/?page=trimmomatic
  • Supports trimming paired-end datasets.
• fastx toolkit – http://hannonlab.cshl.edu/fastx_toolkit/
  • Suite of command line tools for FASTQ and FASTA analysis and manipulation
  • Good for hard clipping. Available at TACC.
  • Documentation at: http://hannonlab.cshl.edu/fastx_toolkit/commandline.html
• seqtk – https://github.com/lh3/seqtk
  • Suite of command line tools for FASTQ and FASTA analysis and manipulation

...

• Gencode – https://www.gencodegenes.org/
  
  • reference genomes, transcriptomes and high-quality annotations for human and mousehttps://www.gencodegenes.org/releases/current.html
• UCSC downloads – http://hgdownload.cse.ucsc.edu/downloads.html
  • reference genomes, transcriptomes and high-quality annotations for many eukaryotes
• Ensembl downloads – ftphttp://ftp.ensembl.org/pub/
  
  • reference genomes, transcriptomes and high-quality annotations for many eukaryotes
• NCBI
  
  • RefSeq – https://www.ncbi.nlm.nih.gov/refseq/
    • well curated genome, transcriptome sequences
  • GenBank – https://www.ncbi.nlm.nih.gov/genbank/
    • public repository for sequence data, especially for prokaryotic genomes
    • not curated
• Reference genome vocabulary – https://software.broadinstitute.org/gatk/documentation/article?id=7857
  • excellent introduction to the types of genome references and the vocabulary used to describe them
    • aimed at higher eukaryotes but vocabulary useful nonethelenonetheless
• GATK blog describing ALT contigs in GRCh38 – https://software.broadinstitute.org/gatk/blog?id=8180
• Support for mapping to ALT contigs containing variants
  
  • bwa mem + bwakit by Heng-Li – https://github.com/lh3/bwa/blob/master/README-alt.md

...

• Comparison of different aligners
  • by Heng Li, developer of bwa, samtools, and many other bioinformatics tools
• File formats
  • input: FASTQ format
  • output: the SAM (Sequence Alignment Map) format specification (SAM1.pdf)
    • SAM1.pdf – header fields, body fields, flag definitions
    • https://github.com/samtools/hts-specs/blob/master/SAMtags.pdf – tag fields
• Aligners
  • bwa (Burrows-Wheeler Aligner) by Heng Li – http://bio-bwa.sourceforge.net/
    • fast, sensitive and easy to use
  • bowtie2 – http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml
    • fast, sensitive and extremely configurable
• The BioITeam has some TACC-aware alignment scripts you might find useful:
  • bwa alignment
    • /workwork2/projects/BioITeam/common/script/align_bwa_illumina.sh
  • bowtie2 alignment
    • /workwork2/projects/BioITeam/common/script/align_bowtie2_illumina.sh
  • merging sorted BAM files (read-group aware)
    
    • /workwork2/projects/BioITeam/common/script/merge_sorted_bams.sh
  • also available on many BRCF pods under /mnt/bioi/script.
  • email or come talk to Anna if you have questions or problems

...

• SAM (Sequence Alignment Map) format specification (SAM1.pdf)
  
  • Translate SAM file flags web calculator: http://broadinstitute.github.io/picard/explain-flags.html
    
    • type in a decimal number to see which flags are set
• samtools – by Heng Li
  • SAM/BAM conversion, flag filtering, sorting, indexing, duplicate filtering
  • older 0.1.xx versions: http://samtools.sourceforge.net/
  • newer 1.3+ versions: http://www.htslib.org/
• Picard toolkit – http://broadinstitute.github.io/picard/
  • SAM/BAM utilities that are read-group aware
  • especially MarkDuplicates for flagging duplicate alignments
    • http://broadinstitute.github.io/picard/command-line-overview.html#MarkDuplicates
• bedtools – http://bedtools.readthedocs.org/en/latest/
  
  • All sub-commands: http://bedtools.readthedocs.io/en/latest/content/bedtools-suite.html
  • Swiss army knife for all manner of common BED, BAM, VCF, GFF/GTF file manipulation.
  • See BEDTools Overview for some common use cases.
  • Available in the TACC module system
• SAMStat - http://samstat.sourceforge.net/
  
  • produces detailed graphical statistics for SAM/BAM files.

File formats and conversion

• SAM format specification – http://samtools.github.io/hts-specs/SAMv1.pdf
  • crucial for performing format conversions, of which ChIP-seq analysis can have many
• Genome browser file formats – http://genome.ucsc.edu/FAQ/FAQformat.html
  
  • BED, bedGraph, narrowPeak and many more
• SRA (Sequence Read Archive) from NCBI
  • overview on this wiki
  • SRA search home page
  • SRA Toolkit
    • NCBI documentation
    • SRA toolkit downloads
• UCSC file format conversion scripts - useful for getting to/from WIG and BED to corresponding binary formats.
  • Make sure you download the correct scripts for your operating system!
  • Directories containing these tools can be found at TACC:
    • /work/projects/BioITeam/common/opt/UCSC_utils.2019_09
    • /work/projects/BioITeam/common/opt/UCSC_utils.2017_07
      

...