Clearml

New Features and Bug Fixes

- Add support for IP overriding with `CLEARML_AGENT_HOST_IP` environment variable
- Add port mapping support (requires `clearml-agent` v2.0 and up)
- Fix bug in plotly histogram, single series labels were shown incorrectly
- Fix adding dataset folder with modified files will upload all files instead of just the modified ones
- Fix detecting git branch in detached HEAD state
- Fix issue with A100 GPU monitoring
- Fix syntax warnings with Python 3.12 (1369)

New Features and Bug Fixes

- Fix Windows `PermissionError` (`WinError 5`) while uploading datasets (1349, thanks Octoslav!)
- Fix offline mode clearml import (1363)
- Add missing `Task.mark_stop_request()`, requesting an agent to stop a running task gracefully
- Add support for streaming in router
- Add async callback support to router
- Fix router did not pass timeout

New Features

* Add programmatic pipeline clone using `PipelineController.create()` and `PipelineController.clone()` (1353)
* Add Python 3.13 support
* Add support for local imports in pipeline steps
* Add support for the ClearML HTTP router using `Task.get_http_router()`
* Add TCP protocol support to `Task.request_external_endpoint()`
* Update `pyjwt` version

Bug Fixes

* Fix slow handling of cached files with large cache_file_limit (1352)
* Fix pipeline crash when repository is set to a directory
* Fix token is not renewed when using an external token (`CLEARML_AUTH_TOKEN`) and no credentials
* Fix don't download external files from parent datasets if they have been modified/removed in the child dataset

New Features

- Add `sdk.development.artifacts.auto_pickle` configuration option to support changing the default pickle behavior when uploading artifacts
- Add `silent_on_errors` argument to `Task.delete_artifacts()` (default `False`)
- Add support for last change time in triggers using tags
- Add `Task.request_external_endpoint()` to request external endpoints on supported backends

Bug Fixes

- Fix `clearml-data search` CLI error if dataset version is `None` (1329, thanks d-vignesh!)
- Fix `maxfile` attribute dropped in `psutil` v6.0.0 causing an error to be printed
- Fix `api.auth.req_token_expiration_sec` configuration option to `api.auth.request_token_expiration_sec` (matches agent setting, keep backwards compatibility)
- Bump `six` version due to Python 3.12 issue with `six.moves`
- Fix bar charts with only 1 bar are not reported correctly

New Features
- Add custom task binary support to `clearml-task` and `CreateAndPopulate` (allows bash script execution, requires agent version >=1.9.0)
- Add support for a default extension name when uploading a pandas `dataframe` artifact (see `sdk.development.artifacts.default_pandas_dataframe_extension_name` configuration option)
- Add verify field support for downloadable URL instead of a file path (see `sdk.aws.s3` configuration section)

Bug Fixes
- Fix valid model URL might be overridden by an invalid one in case the upload failed

New Features
- Add `--tags` option to clearml-task (1284)
- Add retries parameter to `StorageManager.upload_folder()` (1305)
- Add `clearml-task` and `CreateAndPopulate` support for bash scripts, ipynb and python modules (requires `clearml-agent` v1.9+)
- Add support for HTTP file upload progress reporting
- Add `CLEARML_MULTI_NODE_SINGLE_TASK` (values -1, 0, 1, 2) for easier multi-node single Task workloads
- Add `Model.original_task` property to models
- Change `Model.task` property to return connected task
- Update docstring on allowing users to pass `packages=False` to revert to `requirements.txt` inside their git repository

Bug Fixes
- Fix Kerastuner framework and examples (1279)
- Fix scalar logging bug with Fire (1301, thanks tvelovraf!)
- Fix support passing folder to `Task.get_script_info()` to get the git info
- Fix `Task.launch_multi_node()` to enforce the parent of sub-tasks to be the master node 0 task
- Fix tensorboard numpy 2.0 incompatibility breaks binding
- Fix `Task.launch_multi_node()` not supported when used via Pytorch Lightning
- Fix Jupyter notebook packages and uncommitted changes are sometimes not fetched
- Fix `"can't create new thread at interpreter shutdown"` errors (known issue with Python v3.12.0 and other versions)
- Fix injected task import in `Task.populate()`
- Fix dataset with external links will not reuse downloaded data from parents
- Fix hierarchy for pipeline nodes without args
- Fix when abort callback is set, set task status to stopped only if running locally, otherwise leave it for the Agent to set it
- Fix `jsonschema` / `referencing` import to include `TypeError` protection
- Fix Dataset offline behavior