Prefect

🪄 ✨ Customize resource names when provisioning infrastructure for push work pools

In the past few releases, we've added the ability to provision infrastructure for push work pools via the CLI. This release adds the ability to customize the name of the resources created in your cloud environment when provisioning infrastructure for push work pools so you can follow your organization's naming conventions.

To customize your resource names when provisioning infrastructure for a push work pool, follow the interactive prompts:

bash
? Proceed with infrastructure provisioning with default resource names? [Use arrows to move; enter to select]
┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ┃ Options: ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ │ Yes, proceed with infrastructure provisioning with default resource names │
│ > │ Customize resource names │
│ │ Do not proceed with infrastructure provisioning │
└────┴───────────────────────────────────────────────────────────────────────────┘
? Please enter a name for the resource group (prefect-aci-push-pool-rg): new-rg
? Please enter a name for the app registration (prefect-aci-push-pool-app): new-app
? Please enter a prefix for the Azure Container Registry (prefect): newregistry
? Please enter a name for the identity (used for ACR access) (prefect-acr-identity): new-identity
? Please enter a name for the ACI credentials block (new-work-pool-push-pool-credentials): new-aci-block
╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ Provisioning infrastructure for your work pool new-work-pool will require: │
│ │
│ Updates in subscription: Azure subscription 1 │
│ │
│ - Create a resource group in location: eastus │
│ - Create an app registration in Azure AD: new-app │
│ - Create/use a service principal for app registration │
│ - Generate a secret for app registration │
│ - Create an Azure Container Registry with prefix newregistry │
│ - Create an identity new-identity to allow access to the created registry │
│ - Assign Contributor role to service account │
│ - Create an ACR registry for image hosting │
│ - Create an identity for Azure Container Instance to allow access to the registry │
│ │
│ Updates in Prefect workspace │
│ │
│ - Create Azure Container Instance credentials block: new-aci-block │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
Proceed with infrastructure provisioning? [y/n]: y
Creating resource group
Resource group 'new-rg' created successfully
Creating app registration
App registration 'new-app' created successfully
Generating secret for app registration
Secret generated for app registration with client ID '03923189-3151-4acd-8d59-76483752cd39'
Creating ACI credentials block
ACI credentials block 'new-aci-block' created in Prefect Cloud
Assigning Contributor role to service account
Service principal created for app ID '25329389-3151-4acd-8d59-71835252cd39'
Contributor role assigned to service principal with object ID '483h4c85-4a8f-4fdb-0394-bd0f0b1202d0'
Creating Azure Container Registry
Registry created
Logged into registry newregistry1702538242q2z2.azurecr.io
Creating identity
Identity 'new-identity' created
Provisioning infrastructure. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Your default Docker build namespace has been set to 'newregistry1702538242q2z2.azurecr.io'.
Use any image name to build and push to this registry by default:

╭─────────────────────────── example_deploy_script.py ───────────────────────────╮
│ from prefect import flow │
│ from prefect.deployments import DeploymentImage │
│ │
│ │
│ flow(log_prints=True) │
│ def my_flow(name: str = "world"): │
│ print(f"Hello {name}! I'm a flow running on an Azure Container Instance!") │
│ │
│ │
│ if __name__ == "__main__": │
│ my_flow.deploy( │
│ name="my-deployment", │
│ work_pool_name="my-work-pool", │
│ image=DeploymentImage( │
│ name="my-image:latest", │
│ platform="linux/amd64", │
│ ) │
│ ) │
╰────────────────────────────────────────────────────────────────────────────────╯
Infrastructure successfully provisioned for 'new-work-pool' work pool!
Created work pool 'new-work-pool'!


Using a push work pool with automatic infrastructure provisioning is a great way to get started with a production-level Prefect set up in minutes! Check out our [push work pool guide](https://docs.prefect.io/latest/guides/deployment/push-work-pools/) for step-by-step instructions on how to get started!

See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/11407
- https://github.com/PrefectHQ/prefect/pull/11381
- https://github.com/PrefectHQ/prefect/pull/11412

📆 🕐 An updated date time input on the workspace dashboard

We've added a new date and time filter to the workspace dashboard that gives greater control over the dashboard. You can now filter by days, hours, and even minutes. You can also specify a specific date and time range to filter by. You can also go backwards and forwards in time using that time window, for example, you can scroll through by hour.

See it in action!
[
](https://www.loom.com/share/ca099d3792d146d08df6fcd506ff9eb2?sid=70797dda-6dc6-4fe6-bf4a-a9df2a0bf230)

See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect-ui-library/pull/1937
- https://github.com/PrefectHQ/prefect-design/pull/1048


💥 Enhancements
- Add the ability to publish `KubernetesJob` blocks as work pools — https://github.com/PrefectHQ/prefect/pull/11347
- Add setting to configure a default Docker namespace for image builds — https://github.com/PrefectHQ/prefect/pull/11378
- Add the ability to provision an ECR repository for ECS push work pools — https://github.com/PrefectHQ/prefect/pull/11382
- Add ability to provision an Artifact Registry repository for Cloud Run push work pools — https://github.com/PrefectHQ/prefect/pull/11399
- Add ability to provision an Azure Container Registry for Azure Container Instance push work pools — https://github.com/PrefectHQ/prefect/pull/11387
- Add support for `is_schedule_active` to `flow.deploy` and `flow.serve` — https://github.com/PrefectHQ/prefect/pull/11375
- Allow users to select relative and fixed date ranges to filter the dashboard — https://github.com/PrefectHQ/prefect/pull/11406
- Add support for arbitrary sink types to `prefect.utilities.processutils.stream_text` — https://github.com/PrefectHQ/prefect/pull/11298
- Update the Prefect UI deployments page to add run activity and separate out the deployment and flow names — https://github.com/PrefectHQ/prefect/pull/11394
- Update Prefect UI workspace dashboard filters to use new date range - https://github.com/PrefectHQ/prefect-ui-library/pull/1937

🧰 Fixes
- Fix bug where a pause state reused an existing state ID — https://github.com/PrefectHQ/prefect/pull/11405

🧪 Experimental
- Build out API for creating/reading/deleting flow run inputs — https://github.com/PrefectHQ/prefect/pull/11363
- Integrate flow run input and schema/response mechanics into pause/suspend — https://github.com/PrefectHQ/prefect/pull/11376
- Add typing overloads for pause/suspend methods — https://github.com/PrefectHQ/prefect/pull/11403
- Use bytes for `value` in `create_flow_run_input` — https://github.com/PrefectHQ/prefect/pull/11421
- Validate run input when resuming flow runs — https://github.com/PrefectHQ/prefect/pull/11396
- Run existing deployments via the `Runner` webserver — https://github.com/PrefectHQ/prefect/pull/11333

🧠 Documentation
- Add instructions for automatic infrastructure provisioning to the push work pools guide — https://github.com/PrefectHQ/prefect/pull/11316
- Fix broken links in states concept doc and daemonize guide — https://github.com/PrefectHQ/prefect/pull/11374
- Update agent upgrade guide to include `flow.deploy` and examples — https://github.com/PrefectHQ/prefect/pull/11373
- Update block document names in Moving Data guide — https://github.com/PrefectHQ/prefect/pull/11386
- Rename `Guides` to` How-to Guides` — https://github.com/PrefectHQ/prefect/pull/11388
- Add guide to provision infrastructure for existing push work pools — https://github.com/PrefectHQ/prefect/pull/11365
- Add documentation for required permissions for infrastructure provisioning — https://github.com/PrefectHQ/prefect/pull/11417
- Add docs for managed execution open beta — https://github.com/PrefectHQ/prefect/pull/11397, https://github.com/PrefectHQ/prefect/pull/11426, and https://github.com/PrefectHQ/prefect/pull/11425

🪄 ✨ Azure Container Instance push pool infrastructure provisioning via the CLI

We're introducing an enhancement to the Azure Container Instance push pool experience. You can now conveniently provision necessary Azure infrastructure with the `--provision-infra` flag during work pool creation, automating the provisioning of various Azure resources essential for ACI push pools, including resource groups, app registrations, service accounts, and more.

To provision Azure resources when creating an ACI push pool:

bash
❯ prefect work-pool create my-work-pool --provision-infra --type azure-container-instance:push
? Please select which Azure subscription to use: [Use arrows to move; enter to select]
┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ┃ Name ┃ Subscription ID ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ │ Engineering │ 123 │
│ > │ Azure subscription 1 │ 234 │
└────┴──────────────────────┴──────────────────────────────────────┘
╭───────────────────────────────────────────────────────────────────────────────────────╮
│ Provisioning infrastructure for your work pool my-work-pool will require: │
│ │
│ Updates in subscription Azure subscription 1 │
│ │
│ - Create a resource group in location eastus │
│ - Create an app registration in Azure AD │
│ - Create a service principal for app registration │
│ - Generate a secret for app registration │
│ - Assign Contributor role to service account │
│ - Create Azure Container Instance │
│ │
│ Updates in Prefect workspace │
│ │
│ - Create Azure Container Instance credentials block aci-push-pool-credentials │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────╯
Proceed with infrastructure provisioning? [y/n]: y
Creating resource group
Provisioning infrastructure... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:--Resource group 'prefect-aci-push-pool-rg' created in location 'eastus'
Creating app registration
Provisioning infrastructure... ━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20% -:--:--App registration 'prefect-aci-push-pool-app' created successfully
Generating secret for app registration
Provisioning infrastructure... ━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 40% 0:00:06Secret generated for app registration with client ID 'abc'
ACI credentials block 'aci-push-pool-credentials' created
Assigning Contributor role to service account...
Provisioning infrastructure... ━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 60% 0:00:06Contributor role assigned to service principal with object ID 'xyz'
Creating Azure Container Instance
Provisioning infrastructure... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 80% 0:00:04Container instance 'prefect-acipool-container' created successfully
Creating Azure Container Instance credentials block
Provisioning infrastructure... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Infrastructure successfully provisioned for 'my-work-pool' work pool!
Created work pool 'my-work-pool'!


This marks a step forward in Prefect's Azure capabilities, offering you an efficient and streamlined process for leveraging Azure Container Instances to execute their workflows.

See the following pull request for implementation details:
— https://github.com/PrefectHQ/prefect/pull/11275

🪄 🎩 🐰 Introducing the `provision-infra` sub-command for enhanced push work pool management
This enhancement allows you to directly provision infrastructure for existing push work pools. Rather than recreating a work pool, you can provision necessary infrastructure and
update the existing work pool base job template with the following command:

bash
❯ prefect work-pool provision-infra my-work-pool
? Please select which Azure subscription to use: [Use arrows to move; enter to select]
┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ┃ Name ┃ Subscription ID ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ │ Engineering │ 13d │
│ > │ Azure subscription 1 │ 6h4 │
└────┴──────────────────────┴──────────────────────────────────────┘
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Provisioning infrastructure for your work pool my-work-pool will require: │
│ │
│ Updates in subscription Azure subscription 1 │
│ │
│ - Create a resource group in location eastus │
│ - Create an app registration in Azure AD prefect-aci-push-pool-app │
│ - Create/use a service principal for app registration │
│ - Generate a secret for app registration │
│ - Assign Contributor role to service account │
│ - Create Azure Container Instance 'aci-push-pool-container' in resource group prefect-aci-push-pool-rg │
│ │
│ Updates in Prefect workspace │
│ │
│ - Create Azure Container Instance credentials block aci-push-pool-credentials │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Proceed with infrastructure provisioning? [y/n]: y
...


This PR bolsters support for efficient work pool management across diverse cloud environments, delivering a tool for seamless infrastructure setup.

See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/11341
- https://github.com/PrefectHQ/prefect/pull/11355

💥 Enhancements
- Add a `suspend_flow_run` method to suspend a flow run — https://github.com/PrefectHQ/prefect/pull/11291
- Limit the displayed work pool types when `--provision-infra` is used to only show supported work pool types - https://github.com/PrefectHQ/prefect/pull/11350
- Add the ability to publish `Infrastructure` blocks as work pools — https://github.com/PrefectHQ/prefect/pull/11180
- Add the ability to publish `Process` blocks as work pools — https://github.com/PrefectHQ/prefect/pull/11346
- Add a Prefect Cloud event stream subscriber — https://github.com/PrefectHQ/prefect/pull/11332
- Enable storage of key/value information associated with a flow run — https://github.com/PrefectHQ/prefect/pull/11342
- Delete flow run inputs when the corresponding flow run is delete — https://github.com/PrefectHQ/prefect/pull/11352

🧰 Fixes
- Fix the `read_logs` return type to be `List[Log]` — https://github.com/PrefectHQ/prefect/pull/11303
- Fix an issue causing paused flow runs to become stuck in the `Paused` state — https://github.com/PrefectHQ/prefect/pull/11284

🧠 Documentation
- Combine troubleshooting pages — https://github.com/PrefectHQ/prefect/pull/11288
- Add Google Cloud Run V2 option to Serverless guide — https://github.com/PrefectHQ/prefect/pull/11304
- Add `suspend_flow_run` to flows documentation — https://github.com/PrefectHQ/prefect/pull/11300
- Add `work queues` tag to work pools concept page — https://github.com/PrefectHQ/prefect/pull/11320
- Add missing Python SDK CLI items to the docs — https://github.com/PrefectHQ/prefect/pull/11289
- Clarify SCIM + service accounts handling — https://github.com/PrefectHQ/prefect/pull/11343
- Update the work pool concept document — https://github.com/PrefectHQ/prefect/pull/11331

🪄 Automatic infrastructure provisioning for ECS push work pools

Following the introduction of [automatic project configuration for Cloud Run push pools](https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md#automatic-project-configuration-for-cloud-run-push-work-pools) last week, we've added the ability to automatically provision infrastructure in your AWS account and set up your Prefect workspace to support a new ECS push pool!

You can create a new ECS push work pool and provision infrastructure in your AWS account with the following command:

bash
prefect work-pool create --type ecs:push --provision-infra my-pool


Using the `--provision-infra` flag will automatically set up your default AWS account to be ready to execute flows via ECS tasks:


╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Provisioning infrastructure for your work pool my-work-pool will require: │
│ │
│ - Creating an IAM user for managing ECS tasks: prefect-ecs-user │
│ - Creating and attaching an IAM policy for managing ECS tasks: prefect-ecs-policy │
│ - Storing generated AWS credentials in a block │
│ - Creating an ECS cluster for running Prefect flows: prefect-ecs-cluster │
│ - Creating a VPC with CIDR 172.31.0.0/16 for running ECS tasks: prefect-ecs-vpc │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Proceed with infrastructure provisioning? [y/n]: y
Provisioning IAM user
Creating IAM policy
Generating AWS credentials
Creating AWS credentials block
Provisioning ECS cluster
Provisioning VPC
Creating internet gateway
Setting up subnets
Setting up security group
Provisioning Infrastructure ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Infrastructure successfully provisioned!
Created work pool 'my-pool'!


If you have yet to try using an ECS push pool, now is a great time!

If you use Azure, don't fret; we will add support for Azure Container Instances push work pools in a future release!

See the following pull request for implementation details:
— https://github.com/PrefectHQ/prefect/pull/11267


🚀 Enhancements
- Make flows list on flows page a scannable table and standardize list headers throughout the app — https://github.com/PrefectHQ/prefect/pull/11274

🛠️ Fixes
- Fix `.serve` crashes due to process limiter — https://github.com/PrefectHQ/prefect/pull/11264
- Fix URL formatting in `GitRepository` when using provider-specific git credentials blocks — https://github.com/PrefectHQ/prefect/pull/11282
- Prevent excessively escaping the Windows executable — https://github.com/PrefectHQ/prefect/pull/11253

This release is a follow-up to 2.14.7 which never made it to PyPI because of an issue with our Github workflow.

Documentation
- Fix broken docs link in serverless worker documentation — https://github.com/PrefectHQ/prefect/pull/11269

**All changes**: https://github.com/PrefectHQ/prefect/compare/2.14.7...2.14.8

This release fixes a bug introduced in 2.14.6 where deployments with default Docker image builds looked for images tagged `v2.14.6` instead of `2.14.6`. Users of `2.14.6` should upgrade if planning to create deployments with an image other than a custom image.

🌟 Enhancements

- Use a new route to read work pool types when connected to Prefect Cloud — <https://github.com/PrefectHQ/prefect/pull/11236>
- Add `parent_flow_run_id` as a new API filter for flow runs — <https://github.com/PrefectHQ/prefect/pull/11089>

🔧 Fixes

- Allow more than one dependency package in the requirements of a push or pull step — <https://github.com/PrefectHQ/prefect/pull/11254>

📚 Documentation

- Add serverless work pool landing page — <https://github.com/PrefectHQ/prefect/pull/11004>
- Update Azure Container Instance guide to reflect current Azure Portal interface and Prefect UI — <https://github.com/PrefectHQ/prefect/pull/11256>
- Update imports in **Flows** concept page example — <https://github.com/PrefectHQ/prefect/pull/11235>

🧑‍🤝‍🧑 New Contributors

- oakbramble made their first contribution in <https://github.com/PrefectHQ/prefect/pull/11089>

For more details see [the release notes.](https://github.com/PrefectHQ/prefect/compare/2.14.6...2.14.7)

This release includes:

🔜 View the next run for a deployment at a glance
- View the next run for a deployment in the Runs tab of the Deployments page in the Prefect UI! Upcoming runs are now in a dedicated tab, making the most relevant running and completed flow runs more apparent.

🔧 Automatic project configuration for Cloud Run push work pools
- We've enhanced the prefect work-pool create CLI command to automatically configure your GCP project and set up your Prefect workspace to use a new Cloud Run push pool immediately

📚 Updated Guides
- Instructions for turning off the flow run logger to the unit testing guide

🧑‍🤝‍🧑 Contributors
- A big thanks to ConstantinoSchillebeeckx for their continued support and contributions.

See the [the release notes](https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md#release-2146) for more details!