Prefect

Enhancements to `prefect-dbt` for running `dbt-core` commands

Introducing `prefect-dbt` summary artifacts! With summary artifacts, you get a view of all of the runs that succeeded, failed, or were skipped as well as where the failed models live in your dbt project and why they failed. This exposes information about each dbt node in a succinct format that teams can use for development, debugging, insights, and so much more.

These changes are available in `prefect-dbt` version 0.5.0!

<img width="1184" alt="prefect-dbt-summary-artifact-example" src="https://github.com/PrefectHQ/prefect/assets/104510333/3868b961-5aff-4115-b409-f86d3992704d">

See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/12964
- https://github.com/PrefectHQ/prefect/pull/13387
- https://github.com/PrefectHQ/prefect/pull/13406

Enhancements

- Enable flow run infra overrides section in the Prefect UI - https://github.com/PrefectHQ/prefect-ui-library/pull/2417

Experimental
Events and Automations

- Add ability to create and manage automations in the Prefect UI - https://github.com/PrefectHQ/prefect/pull/13342

Documentation

- Update documentation formatting for `prefect-ray` - https://github.com/PrefectHQ/prefect/pull/13385

Integrations

- Update all integrations libraries to pin `prefect<3.0.0` - https://github.com/PrefectHQ/prefect/pull/13408

Integration library releases

This release was accompanied by the following integration library releases:

- `prefect-aws` - 0.4.17
- `prefect-azure` - 0.3.10
- `prefect-bitbucket` - 0.2.5
- `prefect-dask` - 0.2.9
- `prefect-databricks` - 0.2.7
- `prefect-dbt` - 0.5.0
- `prefect-docker` - 0.5.2
- `prefect-email` - 0.3.5
- `prefect-gcp` - 0.5.12
- `prefect-github` - 0.2.5
- `prefect-gitlab` - 0.2.5
- `prefect-kubernetes` - 0.3.10
- `prefect-ray` - 0.3.6
- `prefect-shell` - 0.2.5
- `prefect-slack` - 0.2.6
- `prefect-snowflake` - 0.27.6
- `prefect-sqlalchemy` - 0.4.3

**All changes**: https://github.com/PrefectHQ/prefect/compare/2.19.0...2.19.1

✨ This release includes a number of enhancements and fixes!


Support for major infrastructure and distributed task integrations
As `prefect-dask` and other integrations have been added to the prefect codebase, this release adds these integrations as extra requirements of the prefect package, making it easier to install support for everything in your Prefect stack:

bash
pip install 'prefect[dask]'


We loved this community contribution so much, we did it for all our first-party integrations:

bash
pip install 'prefect[aws,kubernetes,dask,dbt,sqlalchemy,slack]'


You can see the full list of Prefect's extra requirements in [our setup.py](https://github.com/PrefectHQ/prefect/blob/main/setup.py#L43).

Support for timeout seconds in global concurrency context manager
You may want to fail immediately if a global concurrency slot is unavailable. Rather than block and wait, you can now specify a `timeout_seconds` argument in the global concurrency context manager and catch a `TimeoutError` if a slot is not available within the specified time.

python
flow
def fail_immediately_flow():
try:
with concurrency("there-can-be-only-one", occupy=1, timeout_seconds=0.1):
do_something_resource_intensive()
except TimeoutError:
return Cancelled(message="Another flow run is already running")


Manage global concurrency limits via the CLI
Global concurrency limits let you control how many operations can run simultaneously-- now you can create, read, edit, and delete global concurrency limits via the Prefect CLI!

To create a new concurrency limit, use the `prefect gcl create` command. You must specify a `--limit` argument, and can optionally specify a `--slot-decay-per-second` and `--disable` argument.

bash

Experimental
Engine
- Wire up new engine to deployment runs — https://github.com/PrefectHQ/prefect/pull/12914

Fixes
- Fix parameters becoming unresponsive and disappearing in Prefect UI — https://github.com/PrefectHQ/prefect-ui-library/pull/2355

**All changes**: https://github.com/PrefectHQ/prefect/compare/2.18.2...2.18.3

💡 Providing a deployment name to `flow.serve` is now optional

When running `flow.serve`, you can now omit the deployment name. If you do not provide a deployment name, the deployment name will default to the name of the flow. This change makes it easier to run flows without needing to specify a deployment name each time:

python
flow
def etl_flow():
pass

if __name__ == "__main__":
etl_flow.serve()

results in:
bash
Your flow 'etl-flow' is being served and polling for scheduled runs!

To trigger a run for this flow, use the following command:

$ prefect deployment run 'etl-flow/etl-flow'


🛠✨ We've also released a few important fixes to our deployment parameter form when creating a run in the Prefect UI! 🧑‍🎨

🚀 This release also includes a number of other fixes and in-flight feature work. See the [release notes](https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md#release-2182) for more details!

Fixes
- Fix improper context access for nested async task outside of flow — https://github.com/PrefectHQ/prefect/pull/12810
- Fix using default interval schedule in `prefect deploy` — https://github.com/PrefectHQ/prefect/pull/12833
- Handle case in `validationUpdate` schema where definitions are falsy — https://github.com/PrefectHQ/prefect/pull/12880
- Allow `prefect cloud login` to override current workspace — https://github.com/PrefectHQ/prefect/pull/12867
- Remove extra quotes in `prefect deployment run --watch` — https://github.com/PrefectHQ/prefect/pull/12894

Experimental

Events and Automations
- Support filtering by automation name:
- https://github.com/PrefectHQ/prefect/pull/12850
- https://github.com/PrefectHQ/prefect/pull/12884
- https://github.com/PrefectHQ/prefect/pull/12887
- Add support for using the "normal" Trigger classes for `flow.serve` and `.deploy` — https://github.com/PrefectHQ/prefect/pull/12789
- Add an account-level event subscriber — https://github.com/PrefectHQ/prefect/pull/12808
- Emit flow run state change events — https://github.com/PrefectHQ/prefect/pull/12825
- Emit deployment status persistence and events — https://github.com/PrefectHQ/prefect/pull/12853
- Enable event streaming from `PrefectCloudEventSubscriber` via CLI — https://github.com/PrefectHQ/prefect/pull/12796
- Update the `prefect automation delete` CLI — https://github.com/PrefectHQ/prefect/pull/12876

Engine
- Add new experimental engine for tasks and flows with improved readability and extensibility — https://github.com/PrefectHQ/prefect/pull/12856

Documentation
- Improve installation instructions — https://github.com/PrefectHQ/prefect/pull/12783
- Improve quickstart — https://github.com/PrefectHQ/prefect/pull/12798
- Migrate `prefect-azure` docs to Integrations section of the Prefect docs — https://github.com/PrefectHQ/prefect/pull/12794
- Update storage guide credentials blocks — https://github.com/PrefectHQ/prefect/pull/12819
- Remove `server` import recommendations — https://github.com/PrefectHQ/prefect/pull/12823
- Remove link to removed API page — https://github.com/PrefectHQ/prefect/pull/12824
- Add Azure Container Instances worker guide — https://github.com/PrefectHQ/prefect/pull/12846
- Improve wording on integrations index page — https://github.com/PrefectHQ/prefect/pull/12852

Prefect UI Library
- Add `FormattedDate` component to display accessible, long-form timestamps consistently
- Update modal buttons and add auto-close to the parameters and job variable modals — https://github.com/PrefectHQ/prefect-ui-library/pull/2320
- Add flow run list information density — https://github.com/PrefectHQ/prefect-ui-library/pull/2321
- Fix "Run a deployment" action not populating the default parameters from the deployment — https://github.com/PrefectHQ/prefect-ui-library/pull/2322
- Fix schema form properties with no default value from defaulting to `null` (`None`) — https://github.com/PrefectHQ/prefect-ui-library/pull/2323
- Update date-fns and date-fns-tz — https://github.com/PrefectHQ/prefect-ui-library/pull/2319
- Use correct icon colors for non-destructive actions in the UI — https://github.com/PrefectHQ/prefect-ui-library/pull/2328

Integrations
Prefect CGP
- Remove API ref to nonexistent Google Cloud Run V2 page — https://github.com/PrefectHQ/prefect-gcp/pull/260
- Fix VPC access for Cloud v2 worker — https://github.com/PrefectHQ/prefect-gcp/pull/266
- Handle case where `vpc` isn't in job template — https://github.com/PrefectHQ/prefect-gcp/pull/267

New Contributors
* keizobabybear made their first contribution in https://github.com/PrefectHQ/prefect/pull/12852

Breaking Changes
- Remove deprecated ability to use `deployment.yaml` in `prefect deploy` — https://github.com/PrefectHQ/prefect/pull/12731
- Remove deprecated ability to pass `-f/--flow` as option to `prefect deploy` — https://github.com/PrefectHQ/prefect/pull/12732
- Remove deprecated `projects` from `prefect deploy` — https://github.com/PrefectHQ/prefect/pull/12737
- Remove deprecated `--ci` option from `prefect deploy` — https://github.com/PrefectHQ/prefect/pull/12740

Enhancements
- Improve account selection in `prefect cloud login` and `workspace set` — https://github.com/PrefectHQ/prefect/pull/12717


**Full Changelog:** https://github.com/PrefectHQ/prefect/compare/2.17.0...2.18.0

See [the release notes](https://github.com/PrefectHQ/prefect/blob/1006d2d8af397f3bf00218be3783b992144b3da6/RELEASE-NOTES.md#release-2180) for more!