Clearml

New Features and Improvements

- Added `VCS_ENTRY_POINT` environment variable that overrides ClearML's entrypoint auto-detection

Bug Fixes

- Fix all parameters returned from a pipeline are considered strings
- Fix `Task.set_parameters()` does not add parameter type when parameter exists but does not have a type

New Features and Improvements
- Raise error on failed uploads (820, thanks shpigi!)
- Add hyperdataset examples (823)
- Change `report_event_flush_threshold` default to 100
- Add `ModelInfo.weights_object()` for store callback access to the actual model object being stored (valid for both pre/post save calls, otherwise `None`)
- Support `num_workers` in dataset operations
- Support max connections setting for Azure storage using the `sdk.azure.storage.max_connection` configuration option

Bug Fixes
- Fix `clearml` logger default level cannot be changed (741)
- Fix Hydra does use get overridden information from ClearML (751)
- Fix `StorageManager.list(“s3://..”, with_metadata=True)` doesn't work
- Fix `ModelsList.keys()` is missing
- Fix `CLEARML_DEFERRED_TASK_INIT=1` doesn't work
- Fix default API method does not work when set in configuration

New Features and Improvements
- Add tarfile member sanitization to `extractall()` (803, thanks TrellixVulnTeam!)
- Add `Task.delete_artifacts()` with `raise_on_errors` argument (806, thanks frolovconst!)
- Add CI/CD example (815, thanks thepycoder!)
- Limit number of `_serialize` requests when adding list of links with `add_external_files()` (813)
- Add support for connecting Enum values as parameters
- Improve CoLab integration (store entire colab, not history)
- Add `clearml.browser_login` to authenticate browser online sessions such as CoLab, Jupyter Notebooks etc.
- Remove `import_bind` from stack trace of import errors
- Add `sdk.development.worker.report_event_flush_threshold` configuration option to control the number of events to trigger a report
- Return stub object from `Task.init()` if no `clearml.conf` file is found
- Improve manual model uploading example
- Remove deprecated demo server

Bug Fixes
- Fix passing `compression=ZIP_STORED` (or 0) to `Dataset.upload()` uses `ZIP_DEFLATED` and overrides the user-supplied argument (812, thanks doronser!)
- Fix `unique_selector` is not applied properly on batches after the first batch. Remove default selector value since it does not work for all event types (and we always specify it anyway)
- Fix clearml-init colab detection
- Fix cloning pipelines ran with `start_locally()` doesn't work
- Fix if project has a default `output uri` there is no way to disable it in development mode (manual), allow passing `output_uri=False` to disable it
- Fix git remote repository detection when remote is not "origin"
- Fix reported images might not all be reported when waiting to complete the task
- Fix `Dataset.get_local_copy()` deletes the source archive if it is stored locally
- Fix too many parts will cause preview to inflate Task object beyond its 16MB limit - set a total limit of 320kbs
- Fix media preview is created instead of a table preview
- Fix `task.update_output_model()` should always upload local models to a remote server
- Fix broken pip package might mess up requirements detection

New Features and Improvements
- Support running jupyter notebook inside a git repository (repository will be referenced without uncommitted changes and jupyter notebook will be stored om plain code as uncommitted changes)
- Add jupyter notebook fail warning
- Allow pipeline steps to return string paths without them being treated as a folder artifact and zipped (780)
- Remove `future` from Python 3 requirements

Bug Fixes
- Fix exception raised when using `ThreadPool` (790)
- Fix Pyplot/Matplotlib binding reports incorrect line labels and colors (791)
- Pipelines
- Fix crash when running cloned pipeline that invokes a step twice (770, related to 769, thanks tonyd!)
- Fix pipeline argument becomes None if default value is not set
- Fix `retry_on_failure` callback does nothing when specified on `PipelineController.add_step()`
- Fix pipeline clone logic
- Jupyter Notebook
- Fix support for multiple jupyter servers running on the same machine
- Fix issue with old/new notebook packages installed
- Fix local cache with access rules disabling partial local access
- Fix `Task.upload_artifact()` fails uploading pandas `DataFrame`
- Fix relative paths in examples (787, thanks mendrugory!)

New Features and Improvements
- Add callback option for pipeline step retry

Bug Fixes
- Fix Python Fire binding
- Fix Dataset failing to load helper packages should not crash
- Fix `Dataset.get_local_copy()` is allowed for a non-finalized dataset
- Fix `Task.upload_artifact()` does not upload empty lists/tuples
- Fix pipeline retry mechanism interface
- Fix Python <3.5 compatibility
- Fix local cache warning (should be a debug message)

New Features and Improvements
- ClearML Data: Support providing list of links
- Upload artifacts with a custom serializer (689)
- Allow user to specify extension when using custom serializer functions (for artifacts)
- Skip server URL verification in clearml-init wizard process
- When calling `Dataset.get()` without "alias" field, tell user that he can use alias to log it in the UI
- Add mmcv support for logging models
- Add support for Azure and GCP storage in `Task.setup_upload()`
- Support pipeline retrying tasks which are failing on suspected non-stable failures
- Better storage (AWS, GCP) internal load balancing and configurations
- Add `Task.register_abort_callback`

Bug Fixes
- Allow getting datasets with non-semantic versioning (776)
- Fix interactive plots (instead of a generated png)
- Fix Python 2.7 support
- Fix clearml datasets list functionality
- Fix `Dataset.init()` modifies task (moved to `Dataset.create()`)
- Fix failure with large files upload on HTTPS
- Fix 3d plots with plt shows to show 2d plot on task results page
- Fix uploading files with project's `default_upload_destination` (734)
- Fix broken reporting of Matplotlib - Using logarithmic scale breaks reporting
- Fix supporting of wildcards in clearml-data CLI
- Fix `report_histogram` - does not show "horizontal" orientation (699)
- Fix table reporting 'series' arg does not appear on UI when using `logger.report_table(title, series, iteration...)` (684)
- Fix artifacts (and models) use task original name and not new name
- Fix very long filenames from S3 can't be downloaded (with `get_local_copy()`)
- Fix overwrite of existing output models on pipeline task with monitor_models (758)