Prefect

Released on February 2, 2021.

Enhancements

- Add option to provide flow ID to `run flow` CLI command - [4021](https://github.com/PrefectHQ/prefect/pull/4021)
- Flow name and project are no longer required options when calling `run flow` CLI command - [4021](https://github.com/PrefectHQ/prefect/pull/4021)

Task Library

- Add GCSBlobExists which checks for the existence of an object in a given GCS bucket - [4025](https://github.com/PrefectHQ/prefect/pull/4025)
- Use boto3 session in `S3Upload` and `S3Download` tasks, to ensure thread-safe execution - [3981](https://github.com/PrefectHQ/prefect/pull/3981)

Fixes

- Fix issue with fixed duration Paused states not resuming properly - [4031](https://github.com/PrefectHQ/prefect/issues/4031)

Contributors

- [Gregory Roche](https://github.com/gregoryroche)

Released on January 26, 2021.

Enhancements

- S3 storage now logs `ETag`, `LastModified` timestamp, and `VersionId` (if present) when loading a flow - [3995](https://github.com/PrefectHQ/prefect/pull/3995)
- `GitHub` storage now logs the commit sha used when loading a flow - [3998](https://github.com/PrefectHQ/prefect/pull/3998)
- `GitHub` storage now loads from a repo's default branch, allowing default branch names other than 'master' - [3998](https://github.com/PrefectHQ/prefect/pull/3998)
- Improve error message when Secrets are missing with Server - [4003](https://github.com/PrefectHQ/prefect/pull/4003)
- Better error message when passing parameters to `StartFlowRun` constructor - [4008](https://github.com/PrefectHQ/prefect/pull/4008)
- Add warning if user-defined class shadows an attribute used by the base class - [4011](https://github.com/PrefectHQ/prefect/pull/4011)
- Add support for `EXTRA_PIP_PACKAGES` environment variable in `prefecthq/prefect` images, simplifying installation of dependencies during development - [4013](https://github.com/PrefectHQ/prefect/pull/4013)
- Add execution role arn parameter to ecs run config and agent - [4015](https://github.com/PrefectHQ/prefect/pull/4015)

Task Library

- Add `ConnectGetNamespacedPodExec` task which runs an exec command in provided pod container - [3991](https://github.com/PrefectHQ/prefect/pull/3991)
- Ensure connection secrets can be passed to Databricks tasks at runtime - [4001](https://github.com/PrefectHQ/prefect/pull/4001)

Fixes

- Fix Agent registration possibly skipping on server connection issues - [3972](https://github.com/PrefectHQ/prefect/issues/3972)
- `GCSUpload` task now explicitely fails when ran on non-supported types - [3978](https://github.com/PrefectHQ/prefect/pull/3978)
- Make logging to Prefect cloud more robust in the presence of errors or process shutdown - [3989](https://github.com/PrefectHQ/prefect/pull/3989)
- Handle setting state for missing flow runs in Kubernetes agent resource management - [4006](https://github.com/PrefectHQ/prefect/pull/4006)

Contributors

- [Loïc Macherel](https://github.com/LoicEm)
- [Thomas Baldwin](https://github.com/baldwint)

Released on January 19, 2021.

Enhancements

- Retry on additional status codes - [3959](https://github.com/PrefectHQ/prefect/pull/3959)
- Rerun secret tasks on flow-run restart - [3977](https://github.com/PrefectHQ/prefect/pull/3977)

Task Library

- Stream log output from Kubernetes RunNamespacedJob - [3715](https://github.com/PrefectHQ/prefect/pull/3715)
- Add ReadNamespacedPodLogs which reads or streams logs from Kubernetes pod - [3715](https://github.com/PrefectHQ/prefect/pull/3715)
- Add SQL Server task to query SQL Server databases - [3958](https://github.com/PrefectHQ/prefect/pull/3958)
- Add chunking to GCP storage tasks - [3968](https://github.com/PrefectHQ/prefect/pull/3968)

Fixes

- Properly handle `NotImplementedError` exceptions raised by a result's serializer - [3964](https://github.com/PrefectHQ/prefect/pull/3964)
- Fix support for storing multiple flows in a single script in storage - [3969](https://github.com/PrefectHQ/prefect/pull/3969)
- Fix regression in `apply_map` which prevented use in `case`/`resource_manager` blocks - [3975](https://github.com/PrefectHQ/prefect/pull/3975)

Contributors

- [Joël Luijmes](https://github.com/joelluijmes)
- [Peyton Runyan](https://github.com/peytonrunyan/)
- [wangjoshuah](https://github.com/wangjoshuah)

Released on January 13, 2021.

Enhancements

- Better errors/warnings when flow fails to load in execution environment - [3940](https://github.com/PrefectHQ/prefect/pull/3940)

Task Library

- Add an Asana task to add tasks to an asana project - [3935](https://github.com/PrefectHQ/prefect/pull/3935)

Fixes

- Fix `prefect server start` failure when given a custom graphql host port - [3933](https://github.com/PrefectHQ/prefect/pull/3933)
- Fix Kubernetes Agent attempting to report container info for failed pods when no container statuses are found - [3941](https://github.com/PrefectHQ/prefect/pull/3941)
- Avoid race condition when creating task run artifacts for mapped tasks - [3953](https://github.com/PrefectHQ/prefect/pull/3953)
- Propogate agent labels info to k8s flow runs, to match other agent behavior - [3954](https://github.com/PrefectHQ/prefect/pull/3954)

Changelog

Released on January 6, 2021.

Features

- Support for specifying `run_config` for an individual flow run - [3903](https://github.com/PrefectHQ/prefect/pull/3903)

Enhancements

- Allow the usage of a `profile_name` on `get_boto_client` - [3916](https://github.com/PrefectHQ/prefect/pull/3916)
- Support executing Prefect agents/flows without having the `prefect` CLI on path - [3918](https://github.com/PrefectHQ/prefect/pull/3918)

Task Library

- Add support for specifying a `run_config` in `StartFlowRun` - [3903](https://github.com/PrefectHQ/prefect/pull/3903)
- Task to add Trello card for task library - [3910](https://github.com/PrefectHQ/prefect/pull/3910)

Fixes

- Remove unused `description` field on `Task` serializer - [3917](https://github.com/PrefectHQ/prefect/pull/3917)
- Fix edge case in `apply_map` that resulted in cycles in the `Flow` graph - [3920](https://github.com/PrefectHQ/prefect/pull/3920)
- Support storing multiple local flows with the same name when using `Local` storage - [3923](https://github.com/PrefectHQ/prefect/pull/3923)
- Fix bug in `prefect.context` contextmanager that resulted in context fields reverting to their initially configured values - [3924](https://github.com/PrefectHQ/prefect/pull/3924)

Contributors

- [Albert Franzi](https://github.com/afranzi)
- [Allyson Lubimir](https://github.com/whimsicallyson)

Released on December 29, 2020.

Enhancements

- Make `setup` method optional for `resource_manager` tasks - [3869](https://github.com/PrefectHQ/prefect/pull/3869)
- Add labels to all containers managed by the docker agent - [3893](https://github.com/PrefectHQ/prefect/pull/3893)
- Add `prefect server stop` command for stopping the server - [3899](https://github.com/PrefectHQ/prefect/pull/3899)
- Add `--detach` to `prefect server start` for running the server in the background - [3899](https://github.com/PrefectHQ/prefect/pull/3899)

Fixes

- Add support for `google-cloud-storage` < 1.31.0 - [3875](https://github.com/PrefectHQ/prefect/pull/3875)
- Fix use of `imagePullSecrets`/`serviceAccountName` in k8s agent - [3884](https://github.com/PrefectHQ/prefect/pull/3884)
- Fix `read_bytes_from_path` to work properly with S3 - [3885](https://github.com/PrefectHQ/prefect/pull/3885)
- Change default `idempotency_key` in `StartFlowRun` to use `task_run_id` instead of `flow_run_id` - [3892](https://github.com/PrefectHQ/prefect/pull/3892)