Umi-tools

A long overdue release covering some minor functionality updates and bugfixes:

Additional functionality:
- Write out reads failing regex matching with `extract`/`whitelist` (see options `--filtered-out`, `--filtered-out2`). See 328 for motivation
- Ignore template length with paired-end `dedup`/`group` (see option `--ignore-tlen`). See 357 for motivation. Thanks skitcattCRUKMI
- Ignore read pair suffixes with `extract`/`whitelist` e.g `/1` or `/2`. (see option `--ignore-read-pair-suffixes`). See 325, 391, 418, PierreBSC/Viral-Track9 for motivation

Performance
- Sped up error correction mapping for cell barcodes in `whitelist` by using BKTree. Thanks redst4r. Note that this adds a new python dependency (`pybktree`) which is available via `pip` and `conda-forge`.
- Very slight reduction in memory usage for `dedup`/`group` via bugfix to reduce the amount of reads being retained in the buffer. Thanks to mitrinh1 for spotting this (428). The bug was equivalent to hardcoding the option `-buffer-whole-contig` on, which ensures all reads with the same start position are grouped together for deduplication, but at the cost of not yielding reads until the end of each contig, thus increasing memory usage. As such, the bug was not detrimental to results output.


Bugfixes:
- Unmapped mates were not properly discarded with `dedup` and `group`. Thanks Daniel-Liu-c0deb0t for rectifying this.

Debug for `KeyError` when some reads are missing a cell barode tag and stats output required from `umi_tools dedup`. See comments from ZHUwj0 in 281

This release is intended to be a stable release with no plans for significant updates to UMI-tools functionality in the near future. As part of this release, much of the code base has been refactored. It is possible this may have introduced bugs which have not been picked up by the regression testing. If so, please raise an issue and we'll try and rectify with a minor release update ASAP.

**Documentation**

UMI-tools documentation is now available online: https://umi-tools.readthedocs.io/en/latest/index.html

Along with the previous documentation, the readthedocs pages also include new pages:

- FAQ
- Making use of our Alogrithmns: The API

**New knee method for whitelist**

- The method to detect the "knee" in `whitelist` has been updated (317). This method should always identify a threshold and is now set as the default method. Note that this knee method appears to be slightly more conservative (fewer cells above threshold) but having identified the knee, one can always re-run `whitelist` and use `--set-cell-number` to expand the whitelist if desired
- The old method is still available via `--knee-method=density`
- In addition, to run the old knee method but allow whitelist to exit without error even if a suitable knee point isn't identified, use the new `--allow-threshold-error` option (249)
- Putative errors in CBs above the knee can be detected using `--ed-above-threshold` (309)

**Explicit options for handling chimeric & inproper read pairs** (312)

The behaviour for chimeric read pairs, inproper read pairs and unmapped reads can now be explictly set with the `--chimeric-pairs`, `--unpaired-reads` and `--unmapped-reads`.

**New options**

- `--temp-dir`: Set the directory for temporary files (254)
- `--either-read` & `--either-read-resolve`: Extract the UMI from either read (175)

**Misc**

- Updates python testing version to 3.6.7 and drops python 2 testing
- Replace deprecated imp import (318)
- Debug error with `pysam <0.14` (319)
- Refactor module files
- Moves documentation into dedicated module

Mainly minor debugs and improved detection of incorrect command line options. Minor updates to documentation.

- Resolves issues correctly skipping reads which have not been assigned (191 & 273). This involves the addition of the `--assigned-status-tag` option

Testing for OSX has been dropped due to unresolved issues with travis. We hope to resurrect this in the future!

In line with major python packages (e.g https://www.numpy.org/neps/nep-0014-dropping-python2.7-proposal.html), support for python 2 will be dropped from January 1st 2019.

- The defualt value for `--skip_regex` was incorrectly formatted. Thanks to ekernf01 for spotting
(231/256)

Debugs wide-format output for `count` (227). Thanks kevin199011