Clearml

New features
- Make dataset preview optional (1270, thanks pedroconceicao!)
- Add `api.public_ip_ping` (default: `8.8.8.8`) and `api.public_ip_service_urls` (default: `["api.ipify.org";, "ident.me";]`) configuration settings to detect public IP and network interface
- Improve `Dataset.get_mutable_local_copy()` docstring

Bug Fixes
- Fix typo in docs and default SDK config (1281, thanks jimdiroffii!)
- Fix python-fire integration (1275, thanks tvelovraf!)
- Fix path substitution for `file://` URIs (1251, thanks nfzd!)
- Fix numpy 2.0 compatibility (`np.NINF` removed)
- Fix no need to recreate reporter if forking and reporting in subprocess
- Fix forked detection mechanism

Bug Fixes

- Fix pipeline breaks when `continue_on_abort` is set to true
- Fix Pycharm Plugin Windows/Linux interoperability

New Features

- Add additional warning instructing on how to install in case we failed detecting a Jupyter notebook with an import error
- Add `Task.get_executed_queue()` to get the name/ID of the queue a task was executed in
- Move `Task.set_resource_monitor_iteration_timeout()` to a class method, add `wait_for_first_iteration_to_start_sec` and `max_wait_for_first_iteration_to_start_sec` arguments (also add `sdk.development.worker.wait_for_first_iteration_to_start_sec` and `sdk.development.worker.max_wait_for_first_iteration_to_start_sec` configuration options)
- Add support for better pipeline continue behavior including control of children using the `continue_behaviour` argument
- Add Python 3.12 support

Bug Fixes

- Fix issue 1249 pytorch-lightning patches (1254, thanks a-gardner1!)
- Fix parameter overrides are converted to string when using HPO (975)
- Fix FastAI performance (1234)
- Fix MIG GPU support
- Fix AMD GPU metrics collection
- Fix Jupyter password might not be used in some protected Jupyterlab instances
- Fix URL substitution was not applied to registered uploaded files when reporting an event

Bug Fixes

- Fix auto-scaler should recheck that the worker is still IDLE before shutting it down (1240, thanks cthorey!)
- Fix resource monitor fails to get GPU stats in some edge cases

New Features
- Add draft option to pipeline steps (1226, thanks CharlesFrankum!)
- Add support for custom working directory for pipelines (1194)
- Add `Task.get_requirements()` method returning the task’s requirements
- Allow controlling the number of threads used by `StorageManager.download_folder()` using the `max_workers` argument
- Update examples dependencies
- Improve auto populate in `Task.init()`
- Documentation:
* Add docstrings for model properties
* Improve docstring for `force_requirements_env_freeze`
* Add `Task.set_packages()` docstring notes


Bug Fixes
- Fix UTF-8 script code encoding issue (1208, thanks ae-ae!)
- Fix Colab docs (1220, thanks tkukurin!)
- Fix metrics reporting with `OutputModel` while in offline mode results in an error (1172)
- Fix task running in Google Colab doesn't properly get the notebook diff (1204)
- Fix hydra binds break in offline mode (1215)
- Fix HPO crashes when optimizing for single value scalars (1221)
- Fix GPU info such as `gpu_memory` and `gpu_type` is not being collected in some cases
- Fix `clearml-data` CLI tool will move non-dataset tasks to a `.dataset` project if the respective task is not a dataset
- Fix pandas `DataFrame` artifacts with hierarchical indices get mangled by the CSV round-trip
- Fix `urllib3` sends deprecation warning when setting `ssl_version`
- Fix `Task.connect(dict)` return value is not dict-compatible
- Fix `jsonargparse` sub-command config parsing
- Fix Lightning integration crashes when a config entry contains `.` in its name
- Fix Python 3.5 compatibility

Bug Fixes

- Fix AWS S3 configuration `secure` property not forwarded in `StorageHelper` (1205, thanks odulcy-mindee!)
- Fix `NVIDIA_VISIBLE_DEVICES=all` env var value will result in no GPU metrics being reported
- Fix setting `output_uri` when an AWS S3 config section with `secure=false` exists raises an error
- Fix `CacheManager` attempts to clear cache when file limit has not been reached resulting in slow unnecessary checks