Autouri

Added two methods to all URI classes.
- `find_all_files()`: Find all files (not sub-directories) on a directory.
- `rmdir()`: Delete all outputs on a directory.
- local path: `rm -rf`-like deletion on the directory itself.
- `gs://`, `s3://`: Find all files prefixed with the directory and delete them all.

Added CLI sub-command for each new method.
- See details with `autouri --help`.
- Sub-command `find` to wrap `AutoURI(uri_dir).find_all_files()`.
- Sub-command `rmdir` to wrap `AutoURI(uri_dir).rmdir()`.

Added
- `pre-commit` for better linting with Python Black style.

Unlocking locked files on Google Cloud Storage buckets.
If you see any failures related to timeout of a lock file (`.lock`) on `gs://`, then you can use the following command lines to unlock/delete it. Deleting a lock is generally safe if you don't have multiple processes trying to write on it.
bash
$ gsutil retention temp release [LOCK_FILE]
$ gsutil rm -f [LOCK_FILE]

Bug fix for URL contents decoding

Traceback (most recent call last):
File "/usr/local/bin/caper", line 13, in <module>
main()
File "/usr/local/lib/python3.6/dist-packages/caper/cli.py", line 58, in main
c.submit()
File "/usr/local/lib/python3.6/dist-packages/caper/caper.py", line 384, in submit
imports_file = self.__create_imports_zip_file_from_wdl(tmp_dir)
File "/usr/local/lib/python3.6/dist-packages/caper/caper.py", line 847, in __create_imports_zip_file_from_wdl
if CaperWDLParser(self._wdl).zip_subworkflows(zip_file):
File "/usr/local/lib/python3.6/dist-packages/caper/caper_wdl_parser.py", line 75, in zip_subworkflows
root_zip_dir=tmp_d)
File "/usr/local/lib/python3.6/dist-packages/caper/caper_wdl_parser.py", line 189, in __recurse_subworkflow
depth=depth + 1)
File "/usr/local/lib/python3.6/dist-packages/caper/caper_wdl_parser.py", line 146, in __recurse_subworkflow
imports = self.find_imports()
File "/usr/local/lib/python3.6/dist-packages/caper/caper_wdl_parser.py", line 39, in find_imports
CaperWDLParser.RE_PATTERN_WDL_IMPORT)
File "/usr/local/lib/python3.6/dist-packages/caper/caper_wdl_parser.py", line 85, in __find_val
for line in u.read().split('\n'):
File "/usr/local/lib/python3.6/dist-packages/autouri/httpurl.py", line 113, in read
return b.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Removed `--verbose` since it already shows INFO level logs.
Refactored code structure and added individual logger to each module.

Added parameters
- `--debug` and `--verbose`: better logging.
- Better directory parsing in CLI (e.g. `.` and `..`)

Bug fixes
- Required python version: `3.5` -> `3.6` (due to variable annotation).
- Fix for dependency problem during `pip install`.

Important notice
- To allow direct transfer between S3 and GCS, Autouri uses `gsutil` CLI. For this, `GCSURI.USE_GSUTIL_FOR_S3` flag must be on. Otherwise Autouri will stream file transfer through a local machine. Old `gsutil` had a py3 compatibility [issue](https://github.com/GoogleCloudPlatform/gsutil/issues/935). Users need to update `gsutil` to >= 4.47.

Added circleci (integrated testing)

Added CLI
- See `autouri --help` for details

Improved locking to prevent race condition
- Tested up to 10 (50 for local file) threads competing to write on the same file.

Others
- Soft-linking for local paths (`AbsPath`)
- Converting relative path into abs path (`AbsPath.get_abspath_if_exists()`)
- `AbsPath` only allows absolute path as its name says. so added a helper function for such conversion so that apps using Autouri can use it.
- Detailed logging (INFO, DEBUG)
- Can hide flag for `AutoURI.cp()` and `AutoURI.localize()` (`return_flag=False`).
- Can get URL from public GCS/S3 bucket (without presigning)
- For `GCSURI`, `S3URI` only. Formatting for a public URL looks like the following:

_GCS_PUBLIC_URL_FORMAT = 'http://storage.googleapis.com/{bucket}/{path}'
_S3_PUBLIC_URL_FORMAT = 'http://{bucket}.s3.amazonaws.com/{path}'

- This is different from a presigned URL. It's for a file on a bucket with public access. If users use this function on a private bucket file then this function will still return a URL but public access to this URL will be limited (403).

First release