Clearml

New Features

- Add `sdk.development.artifacts.auto_pickle` configuration option to support changing the default pickle behavior when uploading artifacts
- Add `silent_on_errors` argument to `Task.delete_artifacts()` (default `False`)
- Add support for last change time in triggers using tags
- Add `Task.request_external_endpoint()` to request external endpoints on supported backends

Bug Fixes

- Fix `clearml-data search` CLI error if dataset version is `None` (1329, thanks d-vignesh!)
- Fix `maxfile` attribute dropped in `psutil` v6.0.0 causing an error to be printed
- Fix `api.auth.req_token_expiration_sec` configuration option to `api.auth.request_token_expiration_sec` (matches agent setting, keep backwards compatibility)
- Bump `six` version due to Python 3.12 issue with `six.moves`
- Fix bar charts with only 1 bar are not reported correctly

New Features
- Add custom task binary support to `clearml-task` and `CreateAndPopulate` (allows bash script execution, requires agent version >=1.9.0)
- Add support for a default extension name when uploading a pandas `dataframe` artifact (see `sdk.development.artifacts.default_pandas_dataframe_extension_name` configuration option)
- Add verify field support for downloadable URL instead of a file path (see `sdk.aws.s3` configuration section)

Bug Fixes
- Fix valid model URL might be overridden by an invalid one in case the upload failed

New Features
- Add `--tags` option to clearml-task (1284)
- Add retries parameter to `StorageManager.upload_folder()` (1305)
- Add `clearml-task` and `CreateAndPopulate` support for bash scripts, ipynb and python modules (requires `clearml-agent` v1.9+)
- Add support for HTTP file upload progress reporting
- Add `CLEARML_MULTI_NODE_SINGLE_TASK` (values -1, 0, 1, 2) for easier multi-node single Task workloads
- Add `Model.original_task` property to models
- Change `Model.task` property to return connected task
- Update docstring on allowing users to pass `packages=False` to revert to `requirements.txt` inside their git repository

Bug Fixes
- Fix Kerastuner framework and examples (1279)
- Fix scalar logging bug with Fire (1301, thanks tvelovraf!)
- Fix support passing folder to `Task.get_script_info()` to get the git info
- Fix `Task.launch_multi_node()` to enforce the parent of sub-tasks to be the master node 0 task
- Fix tensorboard numpy 2.0 incompatibility breaks binding
- Fix `Task.launch_multi_node()` not supported when used via Pytorch Lightning
- Fix Jupyter notebook packages and uncommitted changes are sometimes not fetched
- Fix `"can't create new thread at interpreter shutdown"` errors (known issue with Python v3.12.0 and other versions)
- Fix injected task import in `Task.populate()`
- Fix dataset with external links will not reuse downloaded data from parents
- Fix hierarchy for pipeline nodes without args
- Fix when abort callback is set, set task status to stopped only if running locally, otherwise leave it for the Agent to set it
- Fix `jsonschema` / `referencing` import to include `TypeError` protection
- Fix Dataset offline behavior

New features
- Make dataset preview optional (1270, thanks pedroconceicao!)
- Add `api.public_ip_ping` (default: `8.8.8.8`) and `api.public_ip_service_urls` (default: `["api.ipify.org";, "ident.me";]`) configuration settings to detect public IP and network interface
- Improve `Dataset.get_mutable_local_copy()` docstring

Bug Fixes
- Fix typo in docs and default SDK config (1281, thanks jimdiroffii!)
- Fix python-fire integration (1275, thanks tvelovraf!)
- Fix path substitution for `file://` URIs (1251, thanks nfzd!)
- Fix numpy 2.0 compatibility (`np.NINF` removed)
- Fix no need to recreate reporter if forking and reporting in subprocess
- Fix forked detection mechanism

Bug Fixes

- Fix pipeline breaks when `continue_on_abort` is set to true
- Fix Pycharm Plugin Windows/Linux interoperability

New Features

- Add additional warning instructing on how to install in case we failed detecting a Jupyter notebook with an import error
- Add `Task.get_executed_queue()` to get the name/ID of the queue a task was executed in
- Move `Task.set_resource_monitor_iteration_timeout()` to a class method, add `wait_for_first_iteration_to_start_sec` and `max_wait_for_first_iteration_to_start_sec` arguments (also add `sdk.development.worker.wait_for_first_iteration_to_start_sec` and `sdk.development.worker.max_wait_for_first_iteration_to_start_sec` configuration options)
- Add support for better pipeline continue behavior including control of children using the `continue_behaviour` argument
- Add Python 3.12 support

Bug Fixes

- Fix issue 1249 pytorch-lightning patches (1254, thanks a-gardner1!)
- Fix parameter overrides are converted to string when using HPO (975)
- Fix FastAI performance (1234)
- Fix MIG GPU support
- Fix AMD GPU metrics collection
- Fix Jupyter password might not be used in some protected Jupyterlab instances
- Fix URL substitution was not applied to registered uploaded files when reporting an event