Trains-agent

Features and Bug Fixes

- Add daemon detached mode (`--detached`, `-d`) that runs the agent as daemon in the background and returns immediately
- Auto mount `~/.git-credentials` into docker container (if file exists)
- Add `TRAINS_AGENT_EXTRA_PYTHON_PATH` environment variable to allow adding additional python path during experiment execution (helpful when using extra un-tracked modules)
- Fix "run as user" feature (using `TRAINS_AGENT_EXEC_USER` environment variable)
- Fix PyTorch support to ignore minor versions when looking for package to install/download
- Fix experiment execution output handling

Features and Bug Fixes

- Add support for `trains-agent execute --id <experiment-id> --docker` that allows executing a specific experiment inside a docker container
- Add support for `trains-agent execute --id <template-experiment-id> --clone` that clones the provided experiment and executes the cloned experiment
- Add support for `APIClient.models.delete()` to allow programmatically deleting a model https://github.com/allegroai/trains-server/issues/32
- Add daemon support for passing storage-related OS environment variables to experiments executed inside a docker container (supported by `trains>=0.13.3`):
* AWS: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_DEFAULT_REGION`
* Azure: `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_KEY`
* Google: `GOOGLE_APPLICATION_CREDENTIALS`
- Fix git checkout with submodules https://github.com/allegroai/trains/issues/112
- Prefer docker image from command line over the one specified in experiment

Features and Bug Fixes

- Allow providing queue names instead of queue IDs in daemon mode
- Docker mode improvements
* Support running as a specific user inside a docker using the `TRAINS_AGENT_EXEC_USER` environment flag
* Pass correct GPU limit when skipping gpus flag
* Add `--force-current-version` daemon command-line flag
- Add K8s/trains glue service example
- Added K8s support in daemon mode
* Running inside a K8s pod
* Mounting dockerized experiment folders to host
* Allow a specific network for the docker
- Add default storage environment vars (for AWS, GS and Azure) to generated agent configuration
- Improve Unicode/UTF stdout handling

Features and Bug Fixes

* Pre-install `numpy` if it exists in the requirements
* Add experiment archiving example
* Add `.bashrc` reloading before running trains-agent in the AWS dynamic cluster management service
* Add support for pulling recursive git modules as as well as main project
* Limit `virtualenv` version to `<20` due to an import issue in v20.0.0
* Fix `pip` install/upgrade with limit in `conda`
* Fix daemon monitor to not stop experiments if network is down

Features
- Add poetry support
- Add `agent.docker_force_pull` config option to force docker pull before running an experiment using a docker image
- Add `agent.extra_docker_arguments` and `agent.extra_docker_shell_script` config options for extra docker parameters and shell script which will always be used when running an experiment using a docker image
- Add `agent.package_manager.pip_version` config option to limit pip version used in virtual env. By default, this value is set to `<20` due to a pip version instability
- Add support for pip "editable" packages syntax in experiment requirements (e.g. `-e git://github.com/my-repo`, see [here](https://pip.readthedocs.io/en/1.1/requirements.htmlrequirements-file-format))
- Add support for git repositories without ".git" suffix (e.g. [Azure Repos](https://azure.microsoft.com/en-us/services/devops/repos))
- Improve conda support
- Fix cases where virtualenv was installed but unavailable as a shell command
- Fix logging monitor to make sure logs are sent even in case an exception occurs inside the monitor itself

Features
- Add support for docker pre-installed pytorch versions that do not exist on PyPI/PyTorch.org
- Add AWS dynamic cluster management service
- Add support for various event query endpoints in APIClient
- Improve the configuration wizard