Resolwe-bio

===================

Added
-----
- Add bcftools version 1.14 to ``resolwebio/common`` Docker image

Changed
-------
- **BACKWARD INCOMPATIBLE:** Rewrite processes
``workflow-bbduk-star-featurecounts-qc-single`` and
``workflow-bbduk-star-featurecounts-qc-paired`` to Python
- **BACKWARD INCOMPATIBLE:** Rewrite workflows
``workflow-bbduk-star-fc-quant-single``,
``workflow-bbduk-star-fc-quant-paired``,
``workflow-cutadapt-star-fc-quant-single`` and
``workflow-cutadapt-star-fc-quant-wo-depletion-single`` to Python
- **BACKWARD INCOMPATIBLE:** Rewrite workflows
``workflow-bbduk-salmon-qc-single`` and
``workflow-bbduk-salmon-qc-paired`` to Python
- Changes to ``resolwebio/common:3.0.0`` Docker image include pinning
of R version and corresponding packages, fixed Python to 3.8,
updated picard-tools to version 2.26.10, updated samtools to
version 1.14 and updated MultiQC to version 1.11
- Normalize processes that use ``resolwebio/common`` Docker image to
use the latest version 3.0.0 and modify tests as necessary
- Bump GATK version to 4.2.4.1 and pin R package versions in
``resolwebio/dnaseq:6.3.0`` Docker image
- **BACKWARD INCOMPATIBLE:** Bump Django requirement to version 3.2

Fixed
-----
- Fix file import and process progress updates in ``upload-fasta-nucl``
- Fix Ensembl-VEP installation in ``resolwebio/dnaseq:6.3.1`` Docker
image


===================

===================

Added
-----
- Add an action for resolving pasted genes on Feature endpoint
- Make Knowledge base squashed migration reversable

Changed
-------
- **BACKWARD INCOMPATIBLE:** Run ``ensembl-vep`` process offline and add
mandatory reference sequence input
- **BACKWARD INCOMPATIBLE:** Remove Diagenode CATS RNA-seq pipeline and
related tools (``rsem``, ``index-fasta-nucl``,
``workflow-custom-cutadapt-star-htseq-single``,
``workflow-custom-cutadapt-star-htseq-paired``,
``workflow-custom-cutadapt-star-rsem-single``,
``workflow-custom-cutadapt-star-rsem-paired``,
``cutadapt-custom-single``, ``cutadapt-custom-paired``).
- **BACKWARD INCOMPATIBLE:** Remove HTSeq-count tool and related
workflows (``htseq-count``, ``htseq-count-raw``,
``workflow-rnaseq-single``, ``workflow-rnaseq-paired``,
``workflow-bbduk-star-htseq``).
- **BACKWARD INCOMPATIBLE:** Remove redundant ``wgs-preprocess`` process
- **BACKWARD INCOMPATIBLE:** Unify Feature autocomplete and search
endpoints into a single endpoint
- Rewrite ``goenrichment`` process to Python
- Rewrite process ``basespace-file-import`` to Python
- Change Ensembl-VEP version check in ``ensembl-vep`` process
- Rename ``featureCounts`` class to ``FeatureCounts``
- Add scatter-gather approach for ``BaseRecalibrator`` and ``ApplyBQSR``
in ``wgs-preprocess-bwa2`` process

Fixed
-----
- Fix build mismatch error message in ``differentialexpression-deseq2``
- Fix how ``self.progress`` is called in ``FeatureCounts``


===================

===================

Added
-----
- Add ``gatk-select-variants`` process

Changed
-------
- **BACKWARD INCOMPATIBLE:** Rewrite ``alignment-star`` and
``alignment-star-index`` processes to Python
- **BACKWARD INCOMPATIBLE:** Rewrite processes ``upload-expression``
and ``upload-expression-cuffnorm`` to Python
- Rewrite processes ``seqtk-sample-single`` and ``seqtk-sample-paired``
to Python
- Rewrite ``bbduk-single`` and ``bbduk-paired`` processes to Python
- Rewrite processes ``upload-fastq-single``, ``upload-fastq-paired``,
``files-to-fastq-single`` and ``files-to-fastq-paired`` to Python
- Rewrite processes ``clustering-hierarchical-samples`` and
``clustering-hierarchical-genes`` to Python
- Add java memory setting and remove unused inputs in
``gatk-genotype-gvcfs``
- Change the ``independent`` field to True by default in process
``differentialexpression-deseq2`` to match the behaviour of the R
script
- Add ``--fork`` parameter in ``ensembl-vep`` process


===================

===================

Added
-----
- Add ``gatk-merge-vcfs`` process

Changed
-------
- **BACKWARD INCOMPATIBLE:** Require Resolwe 29.x
- **BACKWARD INCOMPATIBLE:** Update ``workflow-wgs-gvcf`` to include
BWA-MEM2-based preprocessing step. Support triggering the pipeline
using a pre-aligned BAM input file
- Add ``qin`` and ``ignorebadquality`` BBDuk options to workflows
``workflow-bbduk-salmon-qc-single``,
``workflow-bbduk-salmon-qc-paired``,
``workflow-bbduk-star-featurecounts-qc-single``,
``workflow-bbduk-star-featurecounts-qc-paired``,
``workflow-bbduk-star-fc-quant-single``,
``workflow-bbduk-star-fc-quant-paired``,
``workflow-bbduk-star-htseq``,
``workflow-bbduk-star-htseq-paired``
- Replace ReSDK interface with Python API calls when accesing the
gene KB in processes ``salmon-quant``, ``alleyoop-collapse``,
``slam-count`` and ``mapped-microarray-expression``
- Use downsampled alignment for running QoRTs in 3' mRNA-Seq workflows
``workflow-bbduk-star-fc-quant-single``,
``workflow-bbduk-star-fc-quant-paired``,
``workflow-cutadapt-star-fc-quant-single``,
``workflow-cutadapt-star-fc-quant-wo-depletion-single``
- Replace ``ANONYMOUS_USER_ID`` with ``ANONYMOUS_USER_NAME`` in
settings.py
- Add java memory settings in ``gatk-genomicsdb-import``

Fixed
-----
- Fix data name and advanced options in ``variants-to-table`` process


===================

===================

Added
-----
- Add ``gatk-refine-variants`` process
- Add ensembl-vep tool to the ``resolwebio/dnaseq:6.2.0`` Docker image
- Add ``upload-vep-cache`` process
- Add ``ensembl-vep`` process
- Add ``variants-to-table`` process

Changed
-------
- **BACKWARD INCOMPATIBLE:** Update ``merge-fastq-single`` and
``merge-fastq-paired`` processes to use sample relations for merging
FASTQ files
- **BACKWARD INCOMPATIBLE:** Rewrite ``feature_counts`` process to
Python
- **BACKWARD INCOMPATIBLE:** Create a separate process
``gatk-genomicsdb-import`` for importing GVCFs into the database and
use it as an input in the parallelised ``gatk-genotype-gvcfs`` process
- Support ChIP-Seq and ATAC-Seq data sets in ``geo-import`` process
- Replace the stdout/stdin file interface in SortSam / SetNmMdAndUqTags
stage of the ``wgs-preprocess-bwa2`` with two distinct analysis steps


===================

===================

Added
-----
- Add BWA-mem2 to the ``resolwebio/common`` Docker image
- Add ``bwamem2-index`` process
- Add ``bwamem2`` process
- Add ``wgs-preprocess-bwa2`` process
- Add ``upload-bwamem2-index`` process


Changed
-------
- Use ``resolwebio/common:2.9.0`` Docker image version in
``resolwebio/dnaseq`` Docker image
- Optimize CPU usage in process ``gatk-haplotypecaller-gvcf``
- Make the read trimming step (trimmomatic) optional in the
``workflow-wgs-gvcf`` workflow
- Add aligned reads (BAM format) as an alternative input option in the
``wgs-preprocess`` process
- Set the requirements for number of cores from 20 to 4 and memory from
16 GB to 32 GB in ``alignment-bwa-mem`` process

Fixed
-----
- Fix an edge case in ``methylation-array-sesame`` process where calling
the ``sesame.R`` script using Plumbum was failing for some compressed
IDAT inputs due to the file encoding issues


===================