Octue

Contents

Fixes
- [x] Allow local metadata values to override cloud values if provided at `Datafile` instantiation

<!--- START AUTOGENERATED NOTES --->
<!--- END AUTOGENERATED NOTES --->

Contents

Operations
- [x] Use new pull request description workflow
- [x] Use latest conventional commits package and update workflow

Summary
Add the option to stream the responding service logs to the asker via Google Cloud Pub/Sub. The asker then adds `[REMOTE]` to the start of the log messages and handles them with its own log handlers. The scientist can turn this option on or off each time they ask one of their analysis's children a question.

Contents

Enhancements
- [x] Add `GooglePubSubHandler`, which publishes log records to a Google Pub/Sub topic
- [x] Add option to send log messages created during the running of an analysis in `Service.answer` to the answer topic
- [x] Allow asker to receive and handle any number of non-final-response messages before it deletes the answer topic and subscription
- [x] Allow easy modular addition of new message handlers
- [x] Make the asking service handle log messages forwarded to it by the responding service
- [x] Ensure the asker handles pub/sub messages in the order they were sent by the responder, not the order they were received - this ensures remote log messages are always printed in order and before the final result is logged as received
- [x] Get asker to log receipt of messages of unknown types
- [x] Improve `Service` log messages
- [x] Always send analysis log messages to `stderr` as well as optionally sending them to the given log handler in `Runner.run`

Fixes
- [x] Apply formatter to any handler passed to `apply_log_handler`
- [x] Fix local log notification of logs being streamed to a remote URI
- [x] Add `USE_OCTUE_LOG_HANDLER` environment variable to opt-in to octue log handler (default value opts out) so that importing `octue` doesn't add a potentially unwanted log handler

Refactoring
- [x] Rename `logging_handers` module to `log_handlers`
- [x] Move `log_level` and `log_handler` parameters from `Runner.__init__` to `Runner.run` and update usages
- [x] Factor out pulling of messages from a subscription in `Service`
- [x] Remove thread and process information from logging context

Operations
- [x] Fix `mkver.conf` so new features trigger a minor version increase

Documentation
- [x] Update and condense some docstrings

<!--- START AUTOGENERATED NOTES --->
<!--- END AUTOGENERATED NOTES --->

Quality Checklist
- [x] New features are fully tested (No matter how much Coverage Karma you have)

Contents

Enhancements
- [x] Add shortcut for equals filter

Fixes
- [x] Ensure all defined types of a `Filterable`'s attributes have `equals` and `is` filters
- [x] Treat `KeyError`s and `TypeError`s equally in `getattr_or_subscribe`

Contents

Enhancements
- [x] Allow new datafiles to be created in cloud without local file

Contents

Enhancements
- [x] Improve `Service` log messages
- [x] Keep full module path in logging context when using analysis logger
- [x] Demote storage client log messages to `DEBUG` level

Fixes
- [x] Fix and improve `apply_log_handler` function
- [x] Apply log handler to the root logger in top-level `__init__` module
- [x] Stop duplication of timestamp in Google Cloud Run logs by applying different handler/formatter if `PLATFORM` environment variable is present and equal to `GOOGLE_CLOUD_RUN`
- [x] Add `COMPUTE_PLATFORM=GOOGLE_CLOUD_RUN` environment variable to docker image
- [x] Remove applying of log handler from incorrect places
- [x] Remove extra logger in `runner` module
- [x] Stop analysis logger propagating to the root logger and repeating log messages

Reversions
- [x] Remove `show-twined-logs` option from CLI

Testing
- [x] Silence GCP Storage emulator logs below `ERROR` level
- [x] Ensure unmocked `Subscription` test always deletes topic and subscription
- [x] Remove extra logger from `tests.base`
- [x] Avoid irrelevant exception in Cloud Run tests

Operations
- [x] Remove unnecessary preceding underscores from ARGs in Dockerfile