Prefect

Released on Jan 30, 2020.

Features

- Allow for parameter defaults to vary based on clock - [1946](https://github.com/PrefectHQ/prefect/pull/1946)

Enhancements

- More graceful handling of Agents competing for work - [1956](https://github.com/PrefectHQ/prefect/issues/1956)

Task Library

- None

Fixes

- Eliminated possible duplicate flow run issue in all agents - [1956](https://github.com/PrefectHQ/prefect/issues/1956)

Deprecations

- None

Breaking Changes

- None

Contributors

- None

Released on Jan 24, 2020.

Features

- None

Enhancements

- Docker daemon reconnect attempts + exit on heartbeat failure -[1918](https://github.com/PrefectHQ/prefect/pull/1918)
- More responsive agent shutdown - [1921](https://github.com/PrefectHQ/prefect/pull/1921)
- Background all agent flow deployment attempts - [1928](https://github.com/PrefectHQ/prefect/pull/1928)
- Add show_flow_logs to Docker agent [1929](https://github.com/PrefectHQ/prefect/issues/1929)
- Add per-task checkpointing opt-out - [1933](https://github.com/PrefectHQ/prefect/pull/1933)
- The Task 'checkpoint' kwarg will no longer be deprecated to allow opt-out - [1933](https://github.com/PrefectHQ/prefect/pull/1933)

Task Library

- None

Fixes

- Fix the Fargate Agent not parsing kwargs as literals - [1926](https://github.com/PrefectHQ/prefect/pull/1926)
- Fix issue with result handler default persisting from initialization - [1936](https://github.com/PrefectHQ/prefect/issues/1936)

Deprecations

- None

Breaking Changes

- None

Contributors

- None

Released on Jan 15, 2020.

Features

- Added the ability to leverage native ECS task definition revisions for flow versions in Fargate agent - [1870](https://github.com/PrefectHQ/prefect/pull/1870)
- Added the ability to pull in kwargs per flow version from S3 on flow submission in Fargate agent - [1870](https://github.com/PrefectHQ/prefect/pull/1870)
- Add sensible default result handlers to non-Docker storage options - [1888](https://github.com/PrefectHQ/prefect/issues/1888)

Enhancements

- Allow for task looping to beyond Python's maximum recursion depth - [1862](https://github.com/PrefectHQ/prefect/pull/1862)
- Prevent duplication of stdout logs from multiple instantiated agents - [1866](https://github.com/PrefectHQ/prefect/pull/1866)
- Allow intervals less than 60 seconds in `IntervalClock`s - [1880](https://github.com/PrefectHQ/prefect/pull/1880)
- Introduce new `Secret.exists` method for checking whether a Secret is available - [1882](https://github.com/PrefectHQ/prefect/pull/1882)
- Introduce new `-e` CLI options on agent start commands to allow passing environment variables to flow runs - [1878](https://github.com/PrefectHQ/prefect/issues/1878)
- Stop persisting `None` when calling result handlers - [1894](https://github.com/PrefectHQ/prefect/pull/1894)
- Change Cancelled state to indicate Finished instead of Failed - [1903](https://github.com/PrefectHQ/prefect/pull/1903)
- All States now store `cached_inputs` for easier recovery from failure - [1898](https://github.com/PrefectHQ/prefect/issues/1898)
- Always checkpoint tasks which have result handlers - [1898](https://github.com/PrefectHQ/prefect/issues/1898)


Task Library

- Remove implicit requirement that Google Tasks use Prefect Cloud Secrets - [1882](https://github.com/PrefectHQ/prefect/pull/1882)

Fixes

- Enforce provision of `max_retries` if specifying `retry_delay` for a `Task` - [1875](https://github.com/PrefectHQ/prefect/pull/1875)
- Fix issue with reduce tasks in `flow.visualize()` - [1793](https://github.com/PrefectHQ/prefect/issues/1793)

Deprecations

- The checkpointing kwarg will be removed from Tasks as it is now a default behavior - [1898](https://github.com/PrefectHQ/prefect/issues/1898)

Breaking Changes

- Remove default value for `aws_credentials_secret` on all S3 hooks - [1886](https://github.com/PrefectHQ/prefect/issues/1886)
- Remove `config.engine.result_handler` section of Prefect config - [1888](https://github.com/PrefectHQ/prefect/issues/1888)
- Remove default value for `credentials_secret` on `GCSResultHandler` - [1888](https://github.com/PrefectHQ/prefect/issues/1888)
- Remove default value for `azure_credentials_secret` on `AzureResultHandler` - [1888](https://github.com/PrefectHQ/prefect/issues/1888)

Contributors

- [Daryll Strauss](daryll.straussgmail.com)
- [Braun Reyes](https://github.com/braunreyes)

Released on Dec 17, 2019

Features

- None

Enhancements

- Enhanced treatment of nested and ordered constant values - [1829](https://github.com/PrefectHQ/prefect/pull/1829)
- Add `on_datetime`, `on_date`, and `at_time` filters - [1837](https://github.com/PrefectHQ/prefect/pull/1837)
- Add `--latest` flag for Kubernetes Agent install CLI command - [1842](https://github.com/PrefectHQ/prefect/pull/1842)
- Add `--no-cloud-logs` flag for all agents to optionally opt-out of logs being sent to Prefect Cloud - [1843](https://github.com/PrefectHQ/prefect/pull/1843)
- Agents mark Flow Runs as `Failed` if a deployment error occurs - [1848](https://github.com/PrefectHQ/prefect/pull/1848)
- `Submitted` states from Agents include deployment identifier information - [1848](https://github.com/PrefectHQ/prefect/pull/1848)
- Update heartbeats to respect Cloud flow settings - [1851](https://github.com/PrefectHQ/prefect/pull/1851)
- Add flow run name to `prefect.context` - [1855](https://github.com/PrefectHQ/prefect/pull/1855)
- Add `--namespace` option for Kubernetes Agent start CLI command - [1859](https://github.com/PrefectHQ/prefect/pull/1859)
- Add Prefect job resource configuration for Kubernetes Agent - [1859](https://github.com/PrefectHQ/prefect/pull/1859)

Task Library

- None

Fixes

- Fix Agent deployment errors interrupting full list of found Flow Runs - [1848](https://github.com/PrefectHQ/prefect/pull/1848)
- Fix issue with a single bad log preventing all logs from being sent to Cloud - [1845](https://github.com/PrefectHQ/prefect/pull/1845)
- Fix Kubernetes Agent passing empty default namespace - [1839](https://github.com/PrefectHQ/prefect/pull/1839)

Deprecations

- None

Breaking Changes

- None

Contributors

- None

Released on Dec 11, 2019

Features

- Universal Deploy: Added new Local Agent to run Flows from Local Storage, Azure Storage, S3 Storage, and GCS Storage - [1819](https://github.com/PrefectHQ/prefect/pull/1819)
- Added Azure Blob Storage for Flows - [1831](https://github.com/PrefectHQ/prefect/pull/1831)
- Added GCS Storage for Flows - [1809](https://github.com/PrefectHQ/prefect/pull/1809)
- Added S3 Storage for Flows - [1753](https://github.com/PrefectHQ/prefect/pull/1753)

Enhancements

- Add convenience `parents()` and `children()` classmethods to all State objects for navigating the hierarchy - [1784](https://github.com/PrefectHQ/prefect/pull/1784)
- Add `--rbac` flag to `prefect agent install` for Kubernetes Agent - [1822](https://github.com/PrefectHQ/prefect/pull/1822)
- Add `flow_run_name` to the context - [1815](https://github.com/PrefectHQ/prefect/pull/1815)
- Add `prefect agent install` option to output `supervisord.conf` file for Local Agent - [1819](https://github.com/PrefectHQ/prefect/pull/1819)
- Add convenience `parents()` and `children()` classmethods to all State objects for navigating the hierarchy - [1784](https://github.com/PrefectHQ/prefect/pull/1784)
- Add new `not_all_skipped` trigger and set it as the default for merge tasks - [1768](https://github.com/PrefectHQ/prefect/issues/1768)


Task Library

- Azure Blob tasks now use newer `BlockBlobService` with connection string authentication - [1831](https://github.com/PrefectHQ/prefect/pull/1831)

Fixes

- Fix issue with `flow.visualize()` for mapped tasks which are skipped - [1765](https://github.com/PrefectHQ/prefect/issues/1765)
- Fix issue with timeouts only being softly enforced - [1145](https://github.com/PrefectHQ/prefect/issues/1145), [#1686](https://github.com/PrefectHQ/prefect/issues/1686)
- Log agent errors using `write_run_logs` instead of the deprecated `write_run_log` - [1791](https://github.com/PrefectHQ/prefect/pull/1791)
- Fix issue with `flow.update()` not transferring constants - [1785](https://github.com/PrefectHQ/prefect/pull/1785)

Deprecations

- `flow.deploy` is deprecated in favor of `flow.register` - [1819](https://github.com/PrefectHQ/prefect/pull/1819)

Breaking Changes

- Default Flow storage is now `Local` instead of `Docker` - [1819](https://github.com/PrefectHQ/prefect/pull/1819)
- Docker based `LocalAgent` is renamed `DockerAgent` - [1819](https://github.com/PrefectHQ/prefect/pull/1819)
- `prefect agent start` now defaults to new `LocalAgent` - [1819](https://github.com/PrefectHQ/prefect/pull/1819)

Changelog

Released on Nov 26, 2019

Features

- Add graceful cancellation hooks to Flow and Task runners - [1758](https://github.com/PrefectHQ/prefect/pull/1758)

Enhancements

- Add option to specify a run name for `cloud run` CLI command - [1756](https://github.com/PrefectHQ/prefect/pull/1756)
- Add `work_stealing` option to `DaskKubernetesEnvironment` - [1760](https://github.com/PrefectHQ/prefect/pull/1760)
- Improve heartbeat thread management - [1770](https://github.com/PrefectHQ/prefect/pull/1770)
- Add unique scheduler Job name to `DaskKubernetesEnvironment` - [1772](https://github.com/PrefectHQ/prefect/pull/1772)
- Add informative error when trying to map with the `LocalDaskExecutor` using processes - [1777](https://github.com/PrefectHQ/prefect/pull/1777)

Task Library

- None

Fixes

- Fix issue with heartbeat thread deadlocking dask execution when using a `worker_client` - [1750](https://github.com/PrefectHQ/prefect/pull/1750)
- Fix issue with Environments not calling `run_flow` on Environment stored on Flow object - [1752](https://github.com/PrefectHQ/prefect/pull/1752)
- Fix issue with Docker build context when providing custom docker files - [1762](https://github.com/PrefectHQ/prefect/pull/1762)

Deprecations

- None

Breaking Changes

- None

Contributors

- None