Octue

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([440](https://github.com/octue/octue-sdk-python/pull/440))

Enhancements
- Add ability to download datafiles from a signed URL without cloud permissions
- Add `Datafile.generate_signed_url` method
- Raise error if trying to modify a URL-based datafile
- Raise error if trying to generate a signed URL for a local datafile or dataset

Refactoring
- Use URI terminology in cloud storage path module

Testing
- Test that datafiles and datasets can be downloaded from URLs without cloud permissions

<!--- END AUTOGENERATED NOTES --->

Summary

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([439](https://github.com/octue/octue-sdk-python/pull/439))

Enhancements
- Validate output locations given to `Runner`

Dependencies
- Use `twined=^0.4.1`

<!--- END AUTOGENERATED NOTES --->

Summary
This release provides public methods and a context manager for updating datasets' metadata easily. It also standardises the internals of metadata getting, setting, and using across `Datafile` and `Dataset`.

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([436](https://github.com/octue/octue-sdk-python/pull/436))

New features
- Add context manager for updating dataset stored metadata

Enhancements
- Add new `Metadata` mixin to `Datafile`, `Dataset`, and `Manifest`
- Allow kwargs to be provided to `Dataset.from_cloud`

Fixes
- Stop creating local metadata file on instantiation of `Dataset`
- Stop implicitly uploading metadata when calling `Dataset.from_cloud`
- Add missing `name` property setter to `Dataset`
- Use correct metadata path for signed URL datasets

Refactoring
- Factor out `metadata` method into new `Metadata` mixin
- Rename `Dataset._upload_cloud_metadata` to `Dataset.update_cloud_metadata`
- Rename `Dataset._save_local_metadata` to `Dataset.update_local_metadata`

<!--- END AUTOGENERATED NOTES --->

Summary
This release simplifies the `Datafile` class internals and clarifies its metadata resolution order. Stored metadata will now be used in preference to instantiation metadata unless the `hypothetical` parameter is `True`, allowing the removal of some confusing internal logic from the class.

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([433](https://github.com/octue/octue-sdk-python/pull/433))

Enhancements
- If `hypothetical` is not `True` when re-instantiating existing datafiles, always use their stored metadata (from the cloud object or local metadata file)
- Store cloud metadata on `Datafile` instances without the `octue__` namespace prefix in its keys
- Make `Datafile` metadata update methods public so they can be called easily by users
- Make it optional whether to include the SDK version in the output of `Datafile.metadata`
- Return `None` from `GoogleCloudStorageClient.get_metadata` if bucket not found instead of raising an error

Fixes
- Allow instantiation of a cloud datafile with a non-existent or inaccessible cloud path (defer raising errors until attempting to access it)

Refactoring
- Simplify `Datafile` internals by removing the "initialisation parameters" concept
- Rename `Datafile._get_local_metadata` to `Datafile._use_local_metadata`
- Align `Datafile._use_cloud_metadata` and `Datafile._use_local_metadata` methods
- Factor out setting `Datafile` instance metadata from stored metadata into `_set_metadata` method

<!--- END AUTOGENERATED NOTES --->

Summary
Remove the `Pathable` mixin from `Datafile`. The mixin became increasingly outdated as datafiles became able to represent cloud objects, and other classes such as `Manifest` and `Dataset` could include objects from arbitrary cloud locations.

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([432](https://github.com/octue/octue-sdk-python/pull/432))
**IMPORTANT:** There are 4 breaking changes.
Enhancements
- Improve usage of datafile in fractal template app

Refactoring
- **BREAKING CHANGE:** Remove `Pathable` mixin from `Datafile`
- **BREAKING CHANGE:** Remove the `check` and `exists` methods from `Datafile`
- **BREAKING CHANGE:** Use datafile names for less-than/greater-than comparisons
- **BREAKING CHANGE:** Remove `--skip-checks` option from CLI
- Remove`skip_checks` parameter from `Analysis` and `Runner` constructors
- Delete `Pathable` mixin
- Factor cloud and local `Datafile` instantiation out into methods

Fixes
- Match `Datafile.deserialise` signature to `Serialisable.deserialise`

<!--- END AUTOGENERATED NOTES --->

Summary
Add further improvements and fixes to the SDK to facilitate and improve the experience and speed of digital twin development and deployment.

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([427](https://github.com/octue/octue-sdk-python/pull/427))
**IMPORTANT:** There is 1 breaking change.

New features
- Allow specification of cloud output location to upload output datasets to in app configuration

Enhancements
- Add shorter name `--rm` to optional argument `--delete-topic-and-subscription-on-exit` in CLI `start` command
- Log deletion of topic and subscription in `Service.serve`
- Demote datafile upload and download messages to debug level

Fixes
- **BREAKING CHANGE:** Stop storing file paths in dataset metadata
- Ensure generating signed URLs works on Google Cloud Run as well as with a service account key
- Set default signed URL expiration to the maximum of 7 days
- Refresh credentials to get access token for blob signing
- Ensure signed URL datasets can be uploaded and downloaded
- Ensure `Datafile` instances can be copied and pickled
- Exclude query parameters from dataset names where the dataset was instantiated from a signed URL
- Ensure topics and subscriptions are always deleted in `Service.serve` and `Service.wait_for_answer` if requested
- Ensure datasets' `path` parameter doesn't change after instantiation if the default is used
- Unify local and cloud dataset metadata file formats
- Ignore signed metadata directory when calling `Dataset.from_cloud`
- Fix `Subscription.update`
- Minimise resending of questions via Pub/Sub while they are processing
- Ensure consecutive child analysis log message sections are always coloured differently
- Avoid colourising logs in Google Cloud
- Strip ANSI escape sequences from log messages handled by `GooglePubSubHandler`

Dependencies
- Update to `twined=^0.4` to allow validation of the new output location strand
- Increase the allowed range of `click` to `>=7,<9`
- Allow freedom of most dependencies within their major versions
- Update Dataflow `setup.py`

Refactoring
- Move signed URL generator mock so it's importable externally
- Rename `LOCAL_METADATA_FILENAME` to `METADATA_FILENAME`

Testing
- Test getting output dataset files from the example service
- Use `Child` instead of `Service` in cloud run deployment test

<!--- END AUTOGENERATED NOTES --->