Evaluating your raw data

Before you start the alignment and analysis processes, it is useful to perform some initial quality checks on your raw data. Here we will assume you have data from GSAF's Illumina HiSeq sequencer.

idev -r RNAseq -m 60 -q normal -A OTH21164

Illumina sequence data format (FASTQ)

GSAF gives you paired end sequencing data in two matching fastq format files, contining reads for each end sequenced -- for example Sample_ABC_L005_R1.cat.fastq and Sample_ABC_L005_R2.cat.fastq. Each read end sequenced is representd by a 4-line entry in the fastq file.

A 4-line fastq file entry looks like this:

@HWI-ST1097:104:D13TNACXX:4:1101:1715:2142 1:N:0:CGATGT
GCGTTGGTGGCATAGTGGTGAGCATAGCTGCCTTCCAAGCAGTTATGGGAG
+
=<@BDDD=A;+2C9F<CB?;CGGA<<ACEE*1?C:D>DE=FC*0BAG?DB6

1. Line 1 is the read identifier, which describes the machine, flowcell, cluster, grid coordinate, end and barcode for the read. Except for the barcode information, read identifiers will be identical for corresponding entries in the R1 and R2 fastq files.
2. Line 2 is the sequence reported by the machine.
3. Line 3 is always '+' from GSAF (it can optionally include a sequence description)
4. Line 4 is a string of Ascii-encoded base quality scores, one character per base in the sequence. For each base, an integer quality score = -10 log(probabilty base is wrong) is calculated, then added to 33 to make a number in the Ascii printable character range.

See the Wikipedia FASTQ format page for more information.

Get your data

To try out exercises, we've provided some sample data on lonestar6.

So, go get it!

cds
cd my_rnaseq_course
cd day_1_partA/fastqc_exercise

ls 
ls data

Exercise: Examine the 2nd sequence in a FASTQ file

What is the 2nd sequence in the file Sample1_R1.fastq?

head data/Sample1_R1.fastq

Exercise: Get the first 10 read IDs in the fastq file

grep '^@HWI' data/Sample1_R1.fastq |head

Counting sequences

One of the first thing to check is that your fastq files are the same length, and that length is evenly divisible by four. The wc command (word count) using the -l switch to tell it to count lines, not words, is perfect for this:

wc -l data/Sample1_R1.fastq 

Exercise: Counting FASTQ file lines

How many sequences are in the FASTQ file above?

grep '^@HWI' data/Sample1_R1.fastq |wc -l

Lets move on to assessing the quality of this data...