Clearml-agent

Bug Fixes

- Fix requests requirement lower constraint breaks backwards compatibility for Python 3.6

New Features

- Upgrade requests library (https://github.com/allegroai/clearml-agent/pull/162, thanks jday1!)
- Add support for controlling PyTorch resolving mode using the `CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVE` environment variable and `agent.package_manager.pytorch_resolve` configuration setting with `none` (no resolving), `pip` (sets extra index based on cuda and lets pip resolve) or `direct` (the previous parsing algorithm that does the matching and downloading), default is `pip` (152)
- Add backwards compatibility in standalone mode using the `CLEARML_AGENT_STANDALONE_CONFIG_BC` environment variable
- Add `CLEARML_AGENT_DOCKER_AGENT_REPO` alias for the `FORCE_CLEARML_AGENT_REPO` environment variable
- Show a better message for agent init when an existing `clearml.conf` is found
- Add support for task field injection into container docker name using the `agent.docker_container_name_format_fields` configuration setting
- Add support for adding additional labels to docker containers using the `CLEARML_AGENT_EXTRA_DOCKER_LABELS` environment variable
- Add support for setting file mode in files applied by the agent (using the `files` configuration option) using the `mode` property
- Add support for skipping agent pip upgrade in the default k8s pod container bash script using the `CLEARML_AGENT_NO_UPDATE` environment variable
- Add support for additional pip install flags when installing dependencies using the `CLEARML_EXTRA_PIP_INSTALL_FLAGS` environment variable and `agent.package_manager.extra_pip_install_flags` configuration option
- Add support for extra docker arguments referencing machines environment variables using the `agent.docker_allow_host_environ` configuration option, allowing users to use $ENV in the task docker arguments (e.g. `-e HOST_NAME=$HOST_NAME`)
- Add support for k8s jobs execution (as opposed to only pods)
- Update default docker image versions
- Add Python 3.11 support


Bug Fixes

- Fix `git+ssh://` links inside installed packages not being properly converted to authenticated `https://` and vice versa
- Fix pip version required in the "Installed Packages" is now preserved and reinstalled
- Fix various agent paths not loaded correctly if an empty string or null is used (should be disabled, not converted to `.`)
- Fix docker container backwards compatibility for API<2.13
- Fix default docker match rules resolver (used incorrect field "container" instead of "image")
- Fix task docker argument might be passed twice (might cause an error with flags such as `--network` and `--ipc`)

New Features and Improvements

- Switch services agent entrypoint shell from `sh` to `bash` (141, thanks InCogNiTo124!)
- Improve poetry support
* Add poetry cwd support (142, thanks nielstenboom!)
* Add `agent.package_manager.poetry_install_extra_args` configuration option
- Do not allow request exceptions (keep retrying, throw error only on the initial login call)

Bug Fixes

- Fix agent update version (132, thanks achaiah!)
- Fix login uses GET with payload which breaks when trying to connect a server running in GCP
- Fix `clearml-agent build --docker` stuck on certain containers
- Fix build fails when target is relative path
- Fix pinging running task (change default to once a minute)
- Fix `_` is allowed in k8s label names
- Fix k8s glue does not delete pending pods if the tasks they represent were aborted
- Reintroduce `CLEARML_AGENT_SERVICES_DOCKER_RESTART` accidentally reverted by a previous merge
- Fix `git+ssh://` links inside installed packages not being converted properly to HTTPS-authenticated links

New Features and Improvements
- Upgrade requirements for `attrs`, `jsonschema`, `pyparsing`, `six` and `pyjwt` (129)
- Add default output URI selection to `clearml-agent init`
- Add `agent.disable_task_docker_override` configuration option to disable docker override specified in executing tasks
- Add `CLEARML_AGENT_FORCE_SYSTEM_SITE_PACKAGES` env var (default `true`) to allow overriding default `system_site_packages: true` behavior when running tasks in containers (docker mode and k8s-glue)

Bug Fixes
- Fix using deprecated types validator argument raises an error (deprecated even before `jsonschema` 3.0.0 and unsupported since 4.0.0)
- Fix pip support allowing multiple pip version constraints (by default, one for < Python 3.10 and one for >= Python 3.10)

New Features and Improvements
- Add option to crash agent on exception using `agent.crash_on_exception` configuration setting (123, thanks nielstenboom!)
- Improve venv cache disabled message
- Upgrade packages for better Python 3.10 support
- Remove future package dependency (Python 2 is not supported for `clearml-agent`)
- Change default pip version used to `pip<21` for better Python 3.10 support
- Add support for operator `!=` in package version (mostly for better pytorch resolving)
- Add support for PyTorch new `extra_index_url repo` (find the correct index url based on the cuda version, and let pip do the rest)
- Make venv caching the default behavior
- Add support for `CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV` environment variable (see `agent.hide_docker_command_env_vars` config option)
- Ping executing tasks to make sure the server does not consider them stale (set using the `agent.task_ping_interval_sec` configuration option, defaults to every 120 seconds)

Bug Fixes
- Fix docker extra arguments showing up in configuration printout
- Fix an issue with running on Python 3.10 / 3.11
- Fix cached git token prevents cloning repository (using `agent.enable_git_ask_pass` forcing the agent to use `GIT_ASKPASS` for user/password when cloning/fetching repositories)
- Fix setting `CLEARML_API_DEFAULT_REQ_METHOD` raises an error
- Fix `get_task_session()` may cause an old copy of the `APIClient` to be used containing a reference to the previous session
- K8s Glue
- Fix `agent.system_site_packages` is not turned on by default in k8s glue
- Make sure we git_user/pass is passed to the task pod
- Remove support for `kubectl run`

Improvements
- Add warning if venv cache is disabled
- Add `agent.disable_ssh_mount` configuration option (same as the `CLEARML_AGENT_DISABLE_SSH_MOUNT` environment variable)

Bug Fixes
- Fix docker command for monitoring child agents
- Fix `--gpus all` not reporting GPU stats on worker machine