Prefect

Released on May 5, 2020.

Features

- Add DaskCloudProviderEnvironment to dynamically launch Dask clusters, e.g. on AWS Fargate - [2360](https://github.com/PrefectHQ/prefect/pull/2360)

Enhancements

- Add `botocore_config` option to Fargate agent for setting botocore configuration when interacting with boto3 client - [2170](https://github.com/PrefectHQ/prefect/issues/2170)
- Don't create a `None` task for a null condition when using `ifelse` - [2449](https://github.com/PrefectHQ/prefect/pull/2449)
- Add support for EC2 launch type in Fargate Agent and `FargateTaskEnvironment` - [2421](https://github.com/PrefectHQ/prefect/pull/2421)
- Add `flow_id` to context for Flow runs - [2461](https://github.com/PrefectHQ/prefect/pull/2461)
- Allow users to inject custom context variables into their logger formats - [2462](https://github.com/PrefectHQ/prefect/issues/2462)
- Add option to set backend on `agent install` CLI command - [2478](https://github.com/PrefectHQ/prefect/pull/2478)

Task Library

- None

Fixes

- Fix `start_server.sh` script when an env var is undefined - [2450](https://github.com/PrefectHQ/prefect/pull/2450)
- Fix `server start` CLI command not respecting `version` kwarg on tagged releases - [2435](https://github.com/PrefectHQ/prefect/pull/2435)
- Fix issue with non-JSON serializable args being used to format log messages preventing them from shipping to Cloud - [2407](https://github.com/PrefectHQ/prefect/issues/2407)
- Fix issue where ordered Prefect collections use lexical sorting, not numerical sorting, which can result in unexpected ordering - [2452](https://github.com/PrefectHQ/prefect/pull/2452)
- Fix issue where Resource Manager was failing due to non-JSON timestamp in log writing - [2474](https://github.com/PrefectHQ/prefect/issues/2474)
- Fix periodic error in local agent process management loop - [2419](https://github.com/PrefectHQ/prefect/issues/2419)

Deprecations

- None

Breaking Changes

- None

Contributors

- [Grégory Duchatelet](https://github.com/gregorg)
- [Joe Schmid](https://github.com/joeschmid)

Released on Apr 28, 2020.

Features

- None

Enhancements

- Added serializer for `RemoteDaskEnvironment` - [2369](https://github.com/PrefectHQ/prefect/issues/2369)
- `server start` CLI command now defaults to image build based on current Prefect installation version - [2375](https://github.com/PrefectHQ/prefect/issues/2375)
- Add option to set `executor_kwargs` on `KubernetesJobEnvironment` and `FargateTaskEnvironment` - [2258](https://github.com/PrefectHQ/prefect/issues/2258)
- Add map index to task logs for mapped task runs - [2402](https://github.com/PrefectHQ/prefect/pull/2402)
- Agents can now register themselves with Cloud for better management - [2312](https://github.com/PrefectHQ/prefect/issues/2312)
- Adding support for `environment`, `secrets`, and `mountPoints` via configurable `containerDefinitions` to the Fargate Agent - [2397](https://github.com/PrefectHQ/prefect/pull/2397)
- Add flag for disabling Docker agent interface check on Linux - [2361](https://github.com/PrefectHQ/prefect/issues/2361)

Task Library

- Add Pushbullet notification task to send notifications to mobile - [2366](https://github.com/PrefectHQ/prefect/pull/2366)
- Add support for Docker volumes and filtering in `prefect.tasks.docker` - [2384](https://github.com/PrefectHQ/prefect/pull/2384)

Fixes

- Fix Docker storage path issue when registering flows on Windows machines - [2332](https://github.com/PrefectHQ/prefect/issues/2332)
- Fix issue with refreshing Prefect Cloud tokens - [2409](https://github.com/PrefectHQ/prefect/pull/2409)
- Resolve invalid escape sequence deprecation warnings - [2414](https://github.com/PrefectHQ/prefect/issues/2414)

Deprecations

- None

Breaking Changes

- None

Contributors

- [Nelson Cornet](https://github.com/sk4la)
- [Braun Reyes](https://github.com/braunreyes)
- [Fraznist](https://github.com/Fraznist)
- [sk4la](https://github.com/sk4la)

Released on Apr 21, 2020.

Enhancements

- Agent connection step shows which endpoint it is connected to and checks API connectivity - [2372](https://github.com/PrefectHQ/prefect/pull/2372)

Breaking Changes

- Revert changes to `ifelse` & `switch` (added in [2310](https://github.com/PrefectHQ/prefect/pull/2310)), removing implicit
creation of `merge` tasks - [2379](https://github.com/PrefectHQ/prefect/pull/2379)

Released on Apr 21, 2020.

Features

- None

Enhancements

- Allow GraphQL endpoint configuration via `config.toml` for remote deployments of the UI - [2338](https://github.com/PrefectHQ/prefect/pull/2338)
- Add option to connect containers created by Docker agent to an existing Docker network - [2334](https://github.com/PrefectHQ/prefect/pull/2334)
- Expose `datefmt` as a configurable logging option in Prefect configuration - [2340](https://github.com/PrefectHQ/prefect/pull/2340)
- The Docker agent configures containers to auto-remove on completion - [2347](https://github.com/PrefectHQ/prefect/pull/2347)
- Use YAML's safe load and dump commands for the `server start` CLI command - [2352](https://github.com/PrefectHQ/prefect/pull/2352)
- New `RemoteDaskEnvironment` specifically for running Flows on an existing Dask cluster - [2367](https://github.com/PrefectHQ/prefect/pull/2367)

Task Library

- None

Fixes

- Fix `auth create-token` CLI command specifying deprecated `role` instead of `scope` - [2336](https://github.com/PrefectHQ/prefect/issues/2336)
- Fix local schedules not continuing to schedule on errors outside of runner's control - [2133](https://github.com/PrefectHQ/prefect/issues/2133)
- Fix `get_latest_cached_states` pulling incorrect upstream cached states when using Core server as the backend - [2343](https://github.com/PrefectHQ/prefect/issues/2343)

Deprecations

- None

Breaking Changes

- None

Contributors

- [Brad McElroy](https://github.com/limx0)
- [Joe Schmid](https://github.com/joeschmid)

Released on Apr 14, 2020.

Features

- None

Enhancements

- Task logical operators (e.g. `And`, `Or`, ...) no longer implicitly cast to `bool` - [2303](https://github.com/PrefectHQ/prefect/pull/2303)
- Allow for dynamically changing secret names at runtime - [2302](https://github.com/PrefectHQ/prefect/pull/2302)
- Update `ifelse` and `switch` to return tasks representing the output of the run branch - [2310](https://github.com/PrefectHQ/prefect/pull/2310)

Task Library

- Rename the base secret tasks for clarity - [2302](https://github.com/PrefectHQ/prefect/pull/2302)

Fixes

- Fix possible subprocess deadlocks when sending stdout to `subprocess.PIPE` - [2293](https://github.com/PrefectHQ/prefect/pull/2293), [#2295](https://github.com/PrefectHQ/prefect/pull/2295)
- Fix issue with Flow registration to non-standard Cloud backends - [2292](https://github.com/PrefectHQ/prefect/pull/2292)
- Fix issue with registering Flows with Server that have required scheduled Parameters - [2296](https://github.com/PrefectHQ/prefect/issues/2296)
- Fix interpolation of config for dev services CLI for Apollo - [2299](https://github.com/PrefectHQ/prefect/pull/2299)
- Fix pytest Cloud and Core server backend fixtures - [2319](https://github.com/PrefectHQ/prefect/issues/2319)
- Fix `AzureResultHandler` choosing an empty Secret over provided connection string - [2316](https://github.com/PrefectHQ/prefect/issues/2316)
- Fix containers created by Docker agent on Linux not being able to reach out to host API - [2324](https://github.com/PrefectHQ/prefect/issues/2324)

Deprecations

- None

Breaking Changes

- Remove `env_var` initialization from `EnvVarSecret` in favor of `name` - [2302](https://github.com/PrefectHQ/prefect/pull/2302)

Contributors

- [Brad McElroy](https://github.com/limx0)

Released on Apr 7, 2020.

Features

- CI build for prefect server images - [2229](https://github.com/PrefectHQ/prefect/pull/2229), [#2275](https://github.com/PrefectHQ/prefect/issues/2275)
- Allow kwargs to boto3 in S3ResultHandler - [2240](https://github.com/PrefectHQ/prefect/issues/2240)

Enhancements

- Add flags to `prefect server start` for disabling service port mapping - [2228](https://github.com/PrefectHQ/prefect/pull/2228)
- Add options to `prefect server start` for mapping to host ports - [2228](https://github.com/PrefectHQ/prefect/pull/2228)
- Return `flow_run_id` from CLI `run` methods for programmatic use - [2242](https://github.com/PrefectHQ/prefect/pull/2242)
- Add JSON output option to `describe` CLI commands - [1813](https://github.com/PrefectHQ/prefect/issues/1813)
- Add ConstantResult for eventually replacing ConstantResultHandler - [2145](https://github.com/PrefectHQ/prefect/issues/2145)
- Add new `diagnostics` mode for timing requests made to Cloud - [2283](https://github.com/PrefectHQ/prefect/pull/2283)

Task Library

- Make `project_name` optional for `FlowRunTask` to allow for use with Prefect Core's server - [2266](https://github.com/PrefectHQ/prefect/pull/2266)
- Adds `prefect.tasks.docker.container.RemoveContainer`

Fixes

- Fix `S3ResultHandler` safe retrieval of `_client` attribute - [2232](https://github.com/PrefectHQ/prefect/issues/2232)
- Change default log `timestamp` value in database to be identical to other tables instead of a hard coded value - [2230](https://github.com/PrefectHQ/prefect/pull/2230)

Deprecations

- None

Breaking Changes

- None

Contributors

- [Manuel Aristarán](https://github.com/jazzido)
- [szelenka](https://github.com/szelenka)
- [Aditya Bhumbla](https://github.com/abhumbla)
- [Alex Cano](https://github.com/alexisprince1994)