Environment Variables
Linux has lots of variables in your default environment. Variables in Linux start with a '$' and, by convention, they frequently use capital letters (but that is not required). You can also create variables in Linux to use later, which is a convenient trick that we will use. Try the following commands:
List all the variables currently in your environment:
env
Print the "HOME" variable to see what it contains:
echo $HOME
Notice that you have to use a $ for an existing variable!
What do you think the WORK variable contains?
Try this:
cd $WORK pwd cd $HOME pwd echo $PWD
HOME and WORK are "convenience variables" because they are short, easy to remember, and faster than typing out the full directory structure. The command "pwd" prints the working directory, but there is also a variable called $PWD that is set to your present working directory too.
I put some example data in a folder for you, but you probably don't want to type in that in every time. Let's create a new environment variable.
export TEST_DATA=/scratch/01114/jfonner/training ls $TEST_DATA
Notice that '$' is a prefix only to existing variables. When creating a new variable, we don't need the '$'. When referencing a variable we already created, we need the '$' first.
One Line Scripts
Life scientists often get stuck trying to do simple searching, sorting, and filtering of data if there isn't a tool that already does exactly what is needed. Let's explore some built in tools in Linux:
head $TEST_DATA/250k_reads.fastq grep -A 1 '^@M00' $TEST_DATA/250k_reads.fastq | head -20 head -100000 $TEST_DATA/250k_reads.fastq | grep -A 1 '^@M00' | head head -100000 $TEST_DATA/250k_reads.fastq | grep -A 1 '^@M00' | grep -v '^@M00' | head head -100000 $TEST_DATA/250k_reads.fastq | grep -A 1 '^@M00' | grep -v '^@M00' | grep -v '^--$' | head head -100000 $TEST_DATA/250k_reads.fastq | grep -A 1 '^@M00' | grep -v '^@M00' | grep -v '^--$' | sort | head head -100000 $TEST_DATA/250k_reads.fastq | grep -A 1 '^@M00' | grep -v '^@M00' | grep -v '^--$' | sort | uniq -c | head head -100000 $TEST_DATA/250k_reads.fastq | grep -A 1 '^@M00' | grep -v '^@M00' | grep -v '^--$' | sort | uniq -c | sort -n -r | head
If you don't know what a command did, try using the "man" command, for example "man uniq"
The above example isn't particularly efficient, but it gets the job done. In the above examples, we heavily abuse the Linux "pipe," which is the symbol '|' (probably shares the '\' key on your keyboard above the Enter key). Pipe takes the "standard output" that would normally go to the screen and sends it to the next command.
Further reading and examples: Scott's list of linux one-liners