Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand
titleCatch up

To catch up, just use stage Anna's pre-made files:

Code Block
languagebash
mkdir -p $SCRATCH/byteclub/multiqc/01_fastq
cd $SCRATCH/byteclub/multiqc/01_fastq
lnrsync -savrP -f /work/01063/abattenh/projects/byteclub/multiqc/fastqc
cp /work/01063/abattenh/projects/byteclub/multiqc/conf/02_simple_custom.yaml multiqc_config.yaml .01_fastq/ 01_fastq/

After saving this file, remove the previous MultiQC outputs and re-run the program:

...

Expand
titleCatch up

To catch up, just use Anna's pre-made files:

Code Block
languagebash
mkdir -p $SCRATCH/byteclub/multiqc/02_bowtie
cd $SCRATCH/byteclub/multiqc/02_bowtie
lnrsync -savrP -f /work/01063/abattenh/projects/byteclub/multiqc/03_bowtie/fastqc
rsync -avrP /work/01063/abattenh/projects/byteclub/multiqc/bowtie2/ bowtie2/
cp /work/01063/abattenh/projects/byteclub/multiqc/conf/02_simple_custom.yaml multiqc_config.yaml . 02_bowtie/

Now run multiqc again:

Code Block
languagebash
cd $SCRATCH/byteclub/multiqc/02_bowtie
rm -rf mqc_report*
multiqc .

...

Expand
titleCatch up

To catch up, just use Anna's pre-made files:

Code Block
languagebash
mkdir -p $SCRATCH/byteclub/multiqc/01_fastq
cd $SCRATCH/byteclub/multiqc/02_bowtie
lnrsync -savrP -f /work/01063/abattenh/projects/byteclub/multiqc/04_picard_fixed/fastqc
rsync -avrP /work/01063/abattenh/projects/byteclub/multiqc/bowtie2/ bowtie2/
cp /work/01063/abattenh/projects/byteclub/multiqc/conf/04_fix_dupinfo.yaml multiqc_config.yaml
mkdir for_multiqc
cp /work/01063/abattenh/projects/byteclub/multiqc/for_multiqc/brain*.dupmetrics.txt for_multiqc/ 02_bowtie/

After making this config file modification, you can now run multiqc again:

...

Code Block
titlemultiqc_config.yaml
# Titles to use for the report.
title: "ATAC-Seq QC Reports"
subtitle: null
intro_text: "MultiQC reports for Igor's ATAC-Seq proof-of-concept project."
report_header_info:
    - Sequenced by: 'GSAF'
    - Job: 'JA17277'
    - Run: 'SA17121'
    - Setup: '2x150'

# Change the output filenames
output_fn_name: mqc_report.html
data_dir_name: mqc_report_data

# Ignore these files / directories / paths when searching for reports
fn_ignore_files:
    - '*.dupinfo.txt'

# Modules that should come at the top of the report
top_modules:
    - 'generalstats'
    - 'fastqc'
    - 'samtools'
    - 'picard'

...

Expand
titleCatch up

To catch up, just use Anna's pre-made files:

Code Block
languagebash
mkdir -p $SCRATCH/byteclub/multiqc/02_bowtie
cd $SCRATCH/byteclub/multiqc/02_bowtie
ln -s -f /work/01063/abattenh/projects/byteclub/multiqc/fastqc
rsync -avrP /work/01063/abattenh/projects/byteclub/multiqc/bowtie2/ bowtie2/
cp /work/01063/abattenh/projects/byteclub/multiqc/conf/05_section_order.yaml multiqc_config.yaml
mkdir for_multiqc
cp mkdir -p $SCRATCH/byteclub/multiqc
cd $SCRATCH/byteclub/multiqc
rsync -avrP /work/01063/abattenh/projects/byteclub/multiqc/for05_section_multiqc/brain*.dupmetrics.txt for_multiqcorder/ 02_bowtie/

After making this config file modification, you can now run multiqc again:

...

Producing a report like this:    .  ftphttp://gapdhweb.corral.icmbtacc.utexas.edu/misciyer/byteclub/multiqc/05_section_order.mqc_report.changed_section_order.htmlhtml, with a section order that more closely follows workflow processing steps. 

About MultiQC custom data

When MultiQC does not know about data produced by a program it doesn't know about, it has a mechanisms for adding custom report sections. The simple way to do this is declaratively, (i.e., via configuation parameters) as described below. (You can also write a Python module for very fine-grained control, but that is a lot more work.)

To add a section for custom data:

  1. Format the data appropriately
    • MultiQC supports a number of data file formats (yaml, comma-separated values, etc.)
      • I recommend using simple tab-delimited text files, with MultiQC's preferred .tsv extension.
    • data can be provided as one file per sample (where the sample name is part of the file name)
      • or as a single table-like file containing data for all samples
  2. Add two required custom data section entries in the multiqc_config.yaml configuration file
    • a sp (search path) section for finding report data
      • specifying a wildcard pattern, if data is supplied as one file per sample
      • or a single file name for a consolidated data file
    • each report has a user-named section under a single custom_data section.
      • the required id attribute must be unique, and ties the custom_data, sp and custom_content sections
      • other important attributes include description, file_format, and plot_type.
      • a pconfig sub-section contains plot configuration options
  3. Specify the ordering of the custom report section (optional)
    • add a custom_content section order list entry

...

Code Block
titlemultiqc_config.yaml
# Titles to use for the report.
title: "ATAC-Seq QC Reports"
subtitle: null
intro_text: "MultiQC reports for Igor's ATAC-Seq proof-of-concept project."
report_header_info:
    - Sequenced by: 'GSAF'
    - Job: 'JA17277'
    - Run: 'SA17121'
    - Setup: '2x150'

# Change the output filenames
output_fn_name: mqc_report.html
data_dir_name: mqc_report_data

# Ignore these files / directories / paths when searching for reports
fn_ignore_files:
    - '*.dupinfo.txt'

# Modules that should come at the top of the report
top_modules:
    - 'generalstats'
    - 'fastqc'
    - 'samtools'
    - 'picard'

# --------------------------------
# Custom data
# --------------------------------
custom_content:
  order:
    - bowtie2_isize_section

custom_data:
    bowtie2_isize:
        id: 'bowtie2_isize_section'
        section_name: 'Bowtie2 insert size'
        description: 'distribution for alignments (bowtie2 --local -X2000 --no-mixed --no-discordant)'
        file_format: 'tsv'
        plot_type: 'linegraph'
        pconfig:
            id: 'bowtie2_isize_plot'
            title: 'Insert sizes for proper pairs'
            xlab: 'Insert size'
            ylab: 'Count'

sp:
    bowtie2_isize_section:
        fn: '*.bowtie2_isizes.tsv'ylab: 'Count'

sp:
    bowtie2_isize_section:
        fn: '*.bowtie2_isizes.tsv'

x

Expand
titleCatch up

To catch up, just use Anna's pre-made files:

Code Block
languagebash
mkdir -p $SCRATCH/byteclub/multiqc
cd $SCRATCH/byteclub/multiqc
rsync -avrP /work/01063/abattenh/projects/byteclub/multiqc/05_section_order/ 02_bowtie/

Then the usual...

Code Block
languagebash
cd ~/playtime/multiqc/atacseq; rm -rf mqc_report*; multiqc .

Resulting in a report that includes our inset size distribution data the custom data section we configured: xxx. : ftp://gapdh.icmb.utexas.edu/misc/multiqc/06.mqc_report.custom_linegraph.html

...