...
| Expand |
|---|
| title | Expand here for detailed descriptions of the troubleshooting that took place for this last year. |
|---|
|
| Info |
|---|
The assumption last year was that the correct command would be: conda install -c bioconda samtools as it is what was listed at https://anaconda.org/bioconda/samtools. Instead the correct command ended up being: conda install -c bioconda samtools bcftools openssl=1.0 There are 2 different things going on in this command. - Forcing the installation of a specific version of openssl. In this case, a lower version than would normally be installed if samtools were installed by itself. According to https://github.com/bioconda/bioconda-recipes/issues/12100 my understanding is that when the conda package was put together there is an error wherein samtools specifically says to get version 1.1 of openssl, but the samtools program specifically requires version 1.0 to be present.
- We are installing both samtools and bcftools at the same time. This can clean up some installation problems when there are conflicts between individual packages and you want to use them in a single environment. An alternative would be to have a samtools environment and a bcftools environment, but that creates unnecessary steps of having to change environments in the middle of your analysis.
| Expand |
|---|
| title | Click here to expand and see what the outcome of the assumed installation command is, what the problem is, and steps you could take to fix it. |
|---|
|
| Warning |
|---|
| title | This box contains example commands and outputs showing you something that does NOT work for educational and diagnostic purposes. If you use the code listed in this box, be sure you use ALL the code or you may run into downstream problems with this tutorial. |
|---|
|
| Code Block |
|---|
| conda install -c bioconda samtools
samtools --version |
The above command appeared to install correctly as other conda installations did, but the second command which you would expect to show the version of samtools instead returns the following error: | No Format |
|---|
samtools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory |
Googling the entire error the top results clearly mentioned conda and several pages listed problems associated "fixes" with different conda installation commands: - One of the suggested fixes was to add access to the conda-forge channel (as we have done this year).
- Additionally, last year the entire course was taught with the hope of sequentially adding new programs to a single growing environment. As mentioned yesterday, such an approach is not always optimal/easy, and hence why this year we are creating a number of additional environments. Working from the assumption that you wanted to keep a single environment, one fix that appeared to be working well based on community feedback was
conda install -c bioconda samtools=1.9 --force-reinstall, but at the expense of altering existing packages in the environment. When running the command, the number of programs that would be downgraded was nearly 2 full screens long. - Rather than jumping to the "force-reinstall" solution it was suggested to copy the existing conda environment to a new "test" environment (conda create --name GVA2021-samtools-test --clone GVA2021). Once in the new environment, using the "force-reinstall" command above would have given access to samtools, but would then also require testing other programs in the environment (such as bowtie2, cutadapt, fastqc). Assuming all programs still (seemed) to work you could then rename the environment (
conda create --name GVA2021-V2 --clone GVA2021-samtools-test; conda env remove --name GVA2021-samtools-test). Obviously the more programs that are added, the more likely running into these kinds of conflicts (where different versions of the same dependency are required), and the more programs you would have to check to see if the "test" envvironment broke any previously installed programs. - As there was information that suggested the issues were specific to version 1.12 (including: https://github.com/bioconda/bioconda-recipes/issues/13958), another solution was simply to install an older version of samtools deliberately from the start. In order to do this, I first had to remove the existing (incorrect) samtools version (
conda remove samtools) and then specify the older version we wanted to use (conda install -c bioconda samtools==1.11). This then allowed the samtools --version command to give expected output of version 1.11 Unfortunately, during testing it quickly became obvious that the next program to install (bcftools) was going to create an entirely new set of installation problems meaning that samtools again hat to be uninstalled and samtools bcftools and the downgraded openssl version installed together. conda remove samtools ; conda install -c bioconda samtools bcftools openssl=1.0.
|
|
|
|
...
Assuming you have the above output for samtools --version and bcftools --version (both 1.15.91), first, you need to index the reference file. (This isn't the same as indexing it for read mapping. It's indexing it so that SAMtools can quickly jump to a certain base in the reference.)
...
| Expand |
|---|
| title | view sort and index are all subcommands to the samtools program, and --help can be invoked to get more information about each of the subcommands. Can you figure out what the purpose of each of the above options is? |
|---|
|
| subcommand | flag | value | purpose |
|---|
| view/sort | --threads | 64 | use 64 additional threads | | view | -b | has no value | is a toggle to output in bam format | | view | -S | has no value | was a toggle to declare the input format as sam. help now tells you that this is depreciated as the program auto detects input format | | view | -o | SRR030257.bam | write the output of the command to the SRR030257.bam file | | view |
| SRR030257.sam | unflagged option/keyword. in this case, the top line of the help output lists:
Usage: samtools view [options] <in.bam>|<in.sam>|<in.cram> [region ...]the <>|<>|<> section states that you can give a bam, sam, or cram file as input | | sort |
| SRR030257.bam | unflagged option/keyword. in this case, the top line of the help output lists: Usage: samtools sort [options...] [in.bam] the in.bam section states that the input must be in bam format.
Note that the input to the sort subcommand is the output of the view command. | | sort | -o | SRR030257.sorted.bam | write output to file SRR030257.sorted.bam | | index | -@ | 64 | use 64 additional threads. note that in both the previous help outputs, they listed -@, --threads because both are recognized as the same flag. in the first 2 cases, we used the --threads but could have used -@ but for the index subcommand, only -@ was allowed. | | index |
| SRR030257.sorted.bam | unflagged option/keyword. in this case, the top line of the help output lists:
Usage: samtools index [-bc] [-m INT] <in.bam> [out.index] in this case we are again using the previous commands output file as an input file.
Note that we have not listed an output file here. Keep that in mind as you read further on. |
| Info |
|---|
| title | Why was they -S option included in the view command above if its use is depreciated? |
|---|
| This is included to highlight 2 things: - sometimes you find or are given commands that work, but you dont necessarily understand why they work.
- sometimes program version updates change options in ways that dont require you to update your commands
|
|
...