Octue

Summary
Add the ability to store crash diagnostics (the configuration, input, and logs) for a question when a child service fails. This requires:
- The permission of the parent on a per-question basis
- For the child to be configured to store this data

The crash diagnostics can be retrieved via the new `octue get-crash-diagnostics` command.

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([507](https://github.com/octue/octue-sdk-python/pull/507))

New features
- Allow inputs and configurations to be saved in the cloud and retrieved via the new `get-crash-diagnostics` CLI command if an app crashes

Enhancements
- Add crash diagnostics cloud path to `ServiceConfiguration`
- Add crash diagnostics attribute to questions
- Add ability to copy files within Google Cloud
- Log if the path of a `CloudPathable` is changed
- Avoid need to download datafiles to move them to a new cloud path
- Add ability to choose whether colour is used by log formatter
- Add `GoogleCloudStorageClient.download_all_files` method

Fixes
- Fix configuration manifest validation in `Runner`
- Ensure existing cloud datafiles and datasets can be uploaded to a new cloud location
- Avoid misleading warning when writing to new cloud datafiles

Refactoring
- Rename `CloudPathable._get_cloud_location` to `_set_cloud_location`
- Simplify datafile upload/download in datasets

<!--- END AUTOGENERATED NOTES --->

Summary
Add a child emulator that can be used to test a service without connecting to its real/production children. The messages the child returns are specified during instantiation in python or via a JSON file. All messages that a real child can produce are supported - log messages, monitor messages, exceptions, and results.

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([510](https://github.com/octue/octue-sdk-python/pull/510))

New features
- Add `ChildEmulator` class

Enhancements
- Improve `OrderedMessageHandler` log messages

Fixes
- Ensure `KeyError`s are properly raised by `OrderedMessageHandler`

Refactoring
- Convert `emulators` module into subpackage
- Move Pub/Sub mocks from tests package into `octue.cloud.emulators` subpackage
- Factor out exceptions mapping into `octue.cloud` subpackage

Testing
- Allow Cloud Run and Dataflow deployment tests to be independently skipped
- Update test structure and imports

<!--- END AUTOGENERATED NOTES --->

Summary
Lazily load Google Cloud credentials for `Service` and `Child` instances, allowing apps that don't always contact their children in an analysis to work without the credentials available when they don't need them (instead of failing as currently happens).

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([509](https://github.com/octue/octue-sdk-python/pull/509))

Fixes
- Allow `Service` to be instantiated without Google Cloud credentials

<!--- END AUTOGENERATED NOTES --->

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([506](https://github.com/octue/octue-sdk-python/pull/506))

Enhancements
- Give specifics in the error raised when a datafile fails dataset file tag template validation

Fixes
- Ignore trailing slashes when splitting/parsing cloud paths
- Ensure the `Dataset.name` property works for datasets with a cloud path ending in a forward slash
- Ensure `GoogleCloudStorageClient.scandir` works on the root directory of a bucket

<!--- END AUTOGENERATED NOTES --->

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([500](https://github.com/octue/octue-sdk-python/pull/500))

Enhancements
- Set default question timeout to 1 day
- Allow dataset upload without metadata change
- Cache dataset cloud metadata privately on instance
- Only update dataset cloud metadata if it's changed

Testing
- Update dataflow deployment test

<!--- END AUTOGENERATED NOTES --->

Summary
Fix the installation of `octue` when including the `dataflow` extra by pre-determining the packages needed for `apache-beam` to work with GCP. This also significantly slims down the dependencies of `octue` by only including those explicitly needed.

<!--- SKIP AUTOGENERATED NOTES --->
Contents ([495](https://github.com/octue/octue-sdk-python/pull/495))

Dependencies
- Widen range of acceptable `packaging` versions to avoid conflict with `poetry`
- Remove `gcp` option from `apache-beam` package
- Explicitly add necessary GCP package ranges for Dataflow

Fixes
- Ensure `octue.services` namespace is used for Dataflow service topics

<!--- END AUTOGENERATED NOTES --->