Prefect

Two items of note in this release:

**A note on dependencies**: Last week, a major version upgrade of one of Prefect's dependencies (`griffe`) caused quite the headache for some users' environments, and we are sorry for the inconvenience this caused; in general, we strive to keep _major version_ upper bounds on all of our dependencies, but an oversight caused this dependency to go without an upper bound. This release fixes that issue and added in a few other upper bounds that weren't present.

**`prefect server start --background`**: By popular request, this release features a new flag on `prefect server start` that runs the server in a background process. When starting a background server, a `server.pid` file is written to the `$PREFECT_HOME` directory with the process ID of the background server. This lets `prefect server stop` know which process to interrupt. Currently, only one background server can be started at a time.

New Features 🎉
* Adds the ability to run a server in the background by desertaxle in https://github.com/PrefectHQ/prefect/pull/15013
* Backport `--overwrite` for `create_work_pool` by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14977
* Add Block Type registration for its nested block types in field annotations by GalLadislav in https://github.com/PrefectHQ/prefect/pull/15032

Bug Fixes 🐞
* Ensure Python executable is quoted in runner command on Windows by desertaxle in https://github.com/PrefectHQ/prefect/pull/15015
* Remove duplicate `griffe` requirement by desertaxle in https://github.com/PrefectHQ/prefect/pull/14983

Integrations & Dependencies 🤝
* Unbound griffe to 2.0 by cicdw in https://github.com/PrefectHQ/prefect/pull/14998
* Upper bound critical requirements by cicdw in https://github.com/PrefectHQ/prefect/pull/15003
* Update websockets requirement from <13.0,>=10.4 to >=10.4,<14.0 by dependabot in https://github.com/PrefectHQ/prefect/pull/15025
* Update importlib-resources requirement from <6.2.0,>=6.1.3 to >=6.1.3,<6.5.0 by dependabot in https://github.com/PrefectHQ/prefect/pull/15041


**Full Changelog**: https://github.com/PrefectHQ/prefect/compare/2.20.2...2.20.3

This release contains many bug fixes and small enhancements; two that I want to highlight are:
- [a fix to `KubernetesJob.publish_as_work_pool`](https://github.com/PrefectHQ/prefect/issues/14918): this method helps users migrate from agents to workers by exposing a method to convert an infrastructure block to a structured work pool. Work Pools allow for better governance of infrastructure, and will be easier to observe and audit than agents.
- [a fix to how environment variables on work pools are handled](https://github.com/PrefectHQ/prefect/issues/11041): specifically, users who provided per-flow-run environment variable overrides found that any other environment variables explicitly set on their work pool or deployment were being removed. This fix ensures that all environment variable configuration is carefully merged, preventing the loss of any variables at runtime.

New Features 🎉
* Flow version in runtime by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14898

Bug Fixes 🐞
* Allow for null command when building k8s job by cicdw in https://github.com/PrefectHQ/prefect/pull/14920
* First order deep merge of flow run overrides with deployment overrides by cicdw in https://github.com/PrefectHQ/prefect/pull/14923
* Avoid raising errors on database passwords that contain a `$` character by desertaxle in https://github.com/PrefectHQ/prefect/pull/14888

Development & Tidiness 🧹
* Smoke test `prefect-client` across all supported Python versions by desertaxle in https://github.com/PrefectHQ/prefect/pull/14872
* Update 2.x codeowners by cicdw in https://github.com/PrefectHQ/prefect/pull/14891
* Docker step caching 2.x by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14894

Integrations & Dependencies 🤝
* allow passing `stream_output` to dbt tasks by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14866

**Full Changelog**: https://github.com/PrefectHQ/prefect/compare/2.20.1...2.20.2

This release is small and includes fixes for various dependency issues, as well as a fix for submodule handling in git-based deployments.

Python 3.11 introduced the concept of ["exception groups"](https://peps.python.org/pep-0654/) for situations in which multiple unrelated exceptions need to be raised - for example, in concurrent execution models or retrying an operation across distinct error modes.
Prefect has begun to leverage this capability, and we rely on a separate package (`exceptiongroup`) which backports
this functionality to lower supported versions of Python. This dependency was missing from client-side requirements in `2.20.0`.

Bug Fixes 🐞

* Persist include-submodules flag by cicdw in https://github.com/PrefectHQ/prefect/pull/14839
* Move `exceptiongroup` to `requirements-client.txt` by desertaxle in https://github.com/PrefectHQ/prefect/pull/14871

Integrations & Dependencies 🤝

* Fix integrations for AnyIO 4 by abrookins in https://github.com/PrefectHQ/prefect/pull/14827

Development & Tidiness 🧹

* Backport nullable `flow_run_id` in `Log` table by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14830

Continuing the 2.0 lineage, Prefect's 2.20 release includes a major compatibility upgrade with `anyio`: previous versions of `anyio` were subject to a critical race condition that many CVE scanners are flagging as [a high severity vulnerability](https://security.snyk.io/vuln/SNYK-PYTHON-ANYIO-7361842). Because of the changes necessary in this upgrade, _please be aware_ that new versions of many Prefect integration packages such as `prefect-dask` and `prefect-ray` will require 2.20.0 to function properly.

(For those who have already upgraded to 3.0, don't worry - [Prefect 3.0 release candidates](https://github.com/PrefectHQ/prefect/milestone/18) are already `anyio>=4.4.0` compatible).

Bug Fixes 🐞
* refactors how cluster_config is implemented by jeanluciano in https://github.com/PrefectHQ/prefect/pull/14801
* Add support for custom flow decorators to `prefect deploy` by desertaxle in https://github.com/PrefectHQ/prefect/pull/14782

Integrations & Dependencies 🤝
* Bump anyio on Prefect 2.x by abrookins in https://github.com/PrefectHQ/prefect/pull/14599
* Fix setting of service account on Vertex AI jobs by desertaxle in https://github.com/PrefectHQ/prefect/pull/14816
* Backport teams notif update for `2.x` by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14774
* fix teams webhook block syntax for 3.8 by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14790

Development & Tidiness 🧹
* Remove unnecessary config for releases by cicdw in https://github.com/PrefectHQ/prefect/pull/14763

**Full Changelog**: https://github.com/PrefectHQ/prefect/compare/2.19.9...2.20

This PR continues Prefect's 2.0 lineage as we prepare for a [major 3.0 release](https://github.com/PrefectHQ/prefect/milestone/18). This release is particularly special with respect to 3.0 as it adds warnings to APIs or interfaces that will need updating in 3.0.

There are many internal APIs in Prefect that can be run both synchronously and asynchronously depending on the runtime context in which they are called. If you've ever encountered the error `AttributeError: 'coroutine' object has no attribute` when loading a `Block` then you know what I'm referring to. In 3.0 we have updated the internal logic for switching between synchronous/asynchronous modes in an attempt to provide clearer guidelines and improved performance.

If you ever encounter such an error, you can now use the special `_sync` keyword argument to explicitly enforce the mode of execution you'd prefer:
python
my_block = Block.load("example/block", _sync=True)

my_block = await Block.load("example/block", _sync=False)

When this kwarg is not provided, Prefect makes an educated guess as to whether a coroutine should be returned to await or whether to run the logic synchronously on your behalf.

New Features 🎉
* Add `container_create_kwargs` to Docker worker by kevingrismore in https://github.com/PrefectHQ/prefect/pull/14693
* Expose schema validation flag on deployment flow run create by cicdw in https://github.com/PrefectHQ/prefect/pull/14715
* Backport _sync kwarg to 2.x by cicdw in https://github.com/PrefectHQ/prefect/pull/14725
* Removal 3 warnings by WillRaphaelson in https://github.com/PrefectHQ/prefect/pull/14675
Bug Fixes 🐞
* fix task key comp for 2.x by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14707
* Migrates to Kubernetes_asyncio for prefect 2.X by jeanluciano in https://github.com/PrefectHQ/prefect/pull/14570
* Fix runtime case for `load_flow_from_entrypoint` in 2.x by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14669
Development & Tidiness 🧹
* Remove spurious UI change in release notes. by bunchesofdonald in https://github.com/PrefectHQ/prefect/pull/14566
* Update github auto-release notes template by cicdw in https://github.com/PrefectHQ/prefect/pull/14762
* silence `PrefectDeprecationWarning` in tests by zzstoatzz in https://github.com/PrefectHQ/prefect/pull/14708

**Full Changelog**: https://github.com/PrefectHQ/prefect/compare/2.19.8...2.19.9

Enhancements
- Allow loading flows from an entrypoint when dependencies are missing — https://github.com/PrefectHQ/prefect/pull/14548

Fixes
- Handle join edge case in `ParameterTypeError.from_validation_error` — https://github.com/PrefectHQ/prefect/pull/14420
- Respect `--ui/--no-ui` flag on `prefect server start` — https://github.com/PrefectHQ/prefect/pull/14520
- Append task run futures only when entering task run engine from flow run context — https://github.com/PrefectHQ/prefect/pull/14439

Integrations
- Depend on Cloud Run Execution state for timeout enforcement 2.x — https://github.com/PrefectHQ/prefect/pull/14554
- Handle case where `AwsClientParameters` is a `dict` — https://github.com/PrefectHQ/prefect/pull/14438
- Added `print_func` for `run_namespaced_job` — https://github.com/PrefectHQ/prefect/pull/14463
- Use correct paths in `GcsBucket.put/get_directory` — https://github.com/PrefectHQ/prefect/pull/14537
- Cache Kubernetes client — https://github.com/PrefectHQ/prefect/pull/14535

New Contributors
* oaustegard made their first contribution in https://github.com/PrefectHQ/prefect/pull/14505