Resolwe

===================

Changed
-------
- Rename ``transfer_rec`` to ``transfer_objects`` and change its signature to
accept dictionary objects with information about name, size and hashes of
objects to transfer
- Move part of ``Data`` object validation to listener
- Improve loading time of collection endpoint

Added
-----
- Add ``move_to_collection`` method to ``Data`` viewset
- Report registration failure in ``ProcessTestCase``
- Add a pseudo Python process to serve as a template
- Add ``validate_urls`` method to storage ``BaseConnector`` class
- Validate storage connector settings on registraton
- Add ``transfer_data`` method to ``StorageLocation`` class
- Remove data when ``StorageLocation`` object is deleted
- Store file hashes inside ``ReferencedPath`` model and connect it to
``StorageLocation`` model
- Add ``get_hashes`` method to storage connectors
- Add ``open_stream`` method to storage connectors
- Add ``compute_hashes`` function to ``storage.connectors.hasher`` module
- Use threads when transfering files with ``AwsS3Connector``
- Add ``duplicate`` method to storage connectors
- Add pre-processing and post-processing hooks to storage connectors
- Use multiple threads for file transfer

Fixed
-----
- Add missing decorator ``validate_url`` to ``AwsS3Connector``
- Always import exceptions from ``requests`` library
- Fix bug that sometimes caused objects inside workflow to fail with
``Failed to transfer data.``
- Fix dependency handling bug in listener: when checking for missing data
listener must only consider depencies with kind ``KIND_IO`` instead of all
depencies.
- Raise exception when data transfer failed.


===================

===================

Changed
-------
- **BACKWARD INCOMPATIBLE:** Move purge code inside worker, remove old purge
code
- Various code fixes to make code work with the new storage model
- Use storage connectors in workers to download data not available locally

Added
-----
- Add ``resolwe.storage`` application, a framework for storage management
- Add storage connectors for Google Cloud Storage, Amazon Simple Storage
service and local filesystem.
- Add migrations to move from old storage model to the new one
- Add storage manager
- Add management command to start storage manager
- Add cleanup manager for removing unreferenced data
- Add ``isnull`` related lookup filter
- Add ``entity_count`` to the ``Collection`` serializer
- Add ``inherit_collection`` to ``Data`` viewset
- Add entity_always_create in ``Process`` serializer


===================

===================

Added
-----
- Add support for the ``allow_custom_choice`` field property in Python
processes
- Add ordering by contributor's first and last name to Collection and Data
viewsets
- Add ``data_count`` and ``status`` fields to the ``Collection`` serializer

Fixed
-----
- Enable all top-level class definitions in Python processes
- Make filtering via foreign key more 'Django like': when foreign key does
not exist return empty set instead of raising validation exception.
Also when filtering using list of foreign keys do not raise validation
exception if some foreign keys in the list do not exist.
- Reduce number of database queries in API viewsets by prefetching all
required data


===================

===================

Changed
-------
- Bump Cutadapt to v2.4 and use ``resolwebio/common:1.2.0`` as a base
image in ``resolwebio/rnaseq:4.6.0``

Added
-----
- Add pigz package to ``resolwebio/common:1.2.0`` Docker image
- Add StringTie and UMI-tools to ``resolwebio/rnaseq:4.7.0`` Docker
image

Fixed
-----
- Fix ``spikeins-qc`` process to correctly handle the case where all
expressions are without spikeins
- Fix an error in ``macs2-callpeak`` process that prevented correct
reporting of build/species mismatch between inputs
- Support UCSC annotations in ``feature_counts`` process by assigning
empty string gene_ids to the "unknown" gene


===================

===================

Changed
-------
- **BACKWARD INCOMPATIBLE:** Use Postgres filtering instead of Elasticsearch
on API endpoints
- **BACKWARD INCOMPATIBLE:** Remove filtering by ``year``, ``month``, ``day``,
``hour``, ``minute`` and ``second`` on API endpoints
- Migrate docker images to Fedora 31
- Use ``DictRelatedField`` for ``collection`` field in ``RelationSerializer``.
In practice this fixes inconsistency comparing with how other serializers
handle collections field.

Added
-----
- Add a custom database connector to optimize queries and enable them to use
database indexes
- Add database indexes to improve search performance
- Add database fields and triggers for full-text search in Postgres
- Add support for annotating entities in processes
- Add support for Python 3.8


===================

===================

Added
-----
- Support workflows as inputs to Python processes
- Support retrieval of ``Data.name`` in Python process
- Add ``name_contains``, ``contributor_name``, and ``owners_name`` collection
and data filtering fields on API
- Add ``username`` to ``current_user_permissions`` field of objects on API
- Add ``delete_chunked`` method to Collection, Entity and Storage managers

Fixed
-----
- Delete orphane Storage objects in chunks in purge command to prevent running
out of memory


===================