Caper

* bug fix
- caper_uri: typo in URL -> S3 transfer command
* URL to cloud transfer is two-step
- URL to local and then local to cloud (gcs, s3)
* use call-caching by default
* can use both _ and - in a configuration file
* added `-m` to specify metadata JSON file destination
* fix Java heap memory for `run` (1G) and `server` (2G)

Caper can be PIP-installed.

$ pip install caper

Three executables will be installed. `caper`, `run_mysql_server_singularity.sh` and `run_mysql_server_docker.sh`.


$ caper
/home/ec2-user/miniconda3/lib/python3.7/site-packages/caper/caper.py
usage: caper [-h] [-c FILE] {run,server,submit,abort,unhold,list,metadata} ...

positional arguments:
{run,server,submit,abort,unhold,list,metadata}
run Run a single workflow without server
server Run a Cromwell server
submit Submit a workflow to a Cromwell server
abort Abort running/pending workflows on a Cromwell server
unhold Release hold of workflows on a Cromwell server
list List running/pending workflows on a Cromwell server
metadata Retrieve metadata JSON for workflows from a Cromwell
server

optional arguments:
-h, --help show this help message and exit
-c FILE, --conf FILE Specify config file
(base) [ec2-userip-172-30-0-159 ~]$ run_mysql_server_
run_mysql_server_docker.sh run_mysql_server_singularity.sh



$ run_mysql_server_docker.sh
Usage: ./run_mysql_server_docker.sh [DB_DIR] [PORT] [MYSQL_USER] [MYSQL_PASSWORD] [CONTAINER_NAME]

Example: run_mysql_server_docker.sh ~/cromwell_data_dir 3307

[DB_DIR]: This directory will be mapped to '/var/lib/mysql' inside a container
[PORT] (optional): MySQL database port for docker host (default: 3306)
[MYSQL_USER] (optional): MySQL username (default: cromwell)
[MYSQL_PASSWORD] (optional): MySQL password (default: cromwell)
[CONTAINER_NAME] (optional): MySQL container name (default: mysql_cromwell)



$ run_mysql_server_singularity.sh
Usage: ./run_mysql_server_singularity.sh [DB_DIR] [PORT] [MYSQL_USER] [MYSQL_PASSWORD] [CONTAINER_NAME]

Example: run_mysql_server_singularity.sh ~/cromwell_data_dir 3307

[DB_DIR]: This directory will be mapped to '/var/lib/mysql' inside a container
[PORT] (optional): MySQL database port for singularity host (default: 3306)
[MYSQL_USER] (optional): MySQL username (default: cromwell)
[MYSQL_PASSWORD] (optional): MySQL password (default: cromwell)
[CONTAINER_NAME] (optional): MySQL container name (default: mysql_cromwell)

Cromweller changed its name to caper.

* added shell scripts to `/mysql/` to start a MySQL server with Docker or Singularity.
* supports soft linking for local-to-local file transfer.
* explicitly show `symlinking` instead of `copying` if it makes a symlink.
* for `run` mode, build singularity image before running a pipeline (local backend only).
- fix for multiple tasks trying to build on the same image when there is no pre-built image.

First beta release