Zenml

This release primarily fixes a number of bugs that were introduced as part of
the 0.20.0 ZenServer release. These significantly improve the stability when
using ZenML with the ZenML Server.

Notable fixes include:

- Improved the synchronization that transfers pipeline run information from
the MLMD database to the ZenML Server. This helps fix a number of issues with
missing steps in the post-execution workflow, model deployment steps and other
issues.
- The Label Studio example is fixed and now works again end-to-end.
- The ZenML Label Studio integration can now be used with non-local (i.e.
deployed) instances. For more information see [the Label Studiodocs](https://docs.zenml.io/component-gallery/annotators/label-studio).

New features and other improvements:

- ZenML now uses [alembic](https://alembic.sqlalchemy.org/en/latest/) for
automated database migrations. The migrations happen automatically after every
ZenML update.
- New `zenml pipeline runs export / import / migrate` CLI commands are now
available to export, import and migrate pipeline runs from older, pre-0.20.0
versions of ZenML. The ZenML server now also automatically picks up older
pipeline runs that have been logged in the metadata store by ZenML prior to
0.20.0.
- An MLMD gRPC service can now be deployed with the ZenML Helm chart to act
as a proxy between clients, orchestrators and the MySQL database. This
significantly reduces the time it takes to run pipelines locally.
- You can now specify affinity and tolerations and node selectors to all
Kubernetes based orchestrators with the new Kubernetes Pod settings feature.


Breaking Changes

The following changes introduces with this release mey require some manual
intervention to update your current installations:

* the zenml server helm chart `values.yaml` file has been restructured to make
it easier to configure and to clearly distinguish between the zenml server
component and the newly introduced gRPC service component. Please update your
`values.yaml` copies accordingly.
* the Azure integration dependency versions have been updated. Please run
`zenml integration install azure` to update your current installation, if
you're using Azure.


What's Changed
* Implement automatic alembic migration by AlexejPenner in https://github.com/zenml-io/zenml/pull/990
* Fix GCP Artifact Store listdir empty path by safoinme in https://github.com/zenml-io/zenml/pull/998
* Add flavors mini-video to docs by strickvl in https://github.com/zenml-io/zenml/pull/999
* Remove the Client() warning when used inside a step by stefannica in https://github.com/zenml-io/zenml/pull/1000
* Fix broken links caused by updated by AlexejPenner in https://github.com/zenml-io/zenml/pull/1002
* Fix `FileNotFoundError` with remote path in HuggingFace Dataset materializer by gabrielmbmb in https://github.com/zenml-io/zenml/pull/995
* Add `zenml pipeline runs export / import / migrate` CLI commands by fa9r in https://github.com/zenml-io/zenml/pull/977
* Log message when activating a stack as part of registration by schustmi in https://github.com/zenml-io/zenml/pull/1005
* Minor fixes in Migration to 0.20.0 documentation by alvarobartt in https://github.com/zenml-io/zenml/pull/1009
* Doc updates by htahir1 in https://github.com/zenml-io/zenml/pull/1006
* Fixing broken links in docs by dnth in https://github.com/zenml-io/zenml/pull/1018
* Label Studio example fix by strickvl in https://github.com/zenml-io/zenml/pull/1021
* Docs for using CUDA-enabled docker images by strickvl in https://github.com/zenml-io/zenml/pull/1010
* Add social media heading on docs page by dnth in https://github.com/zenml-io/zenml/pull/1020
* Add executing custom command for getting requirements by gabrielmbmb in https://github.com/zenml-io/zenml/pull/1012
* Delay user instruction in dockerfile generation by schustmi in https://github.com/zenml-io/zenml/pull/1004
* Update link checker configs for faster, more accurate checks by dnth in https://github.com/zenml-io/zenml/pull/1022
* Add `pip install zenml[server]` to relevant examples by dnth in https://github.com/zenml-io/zenml/pull/1027
* Add Tolerations and NodeAffinity to Kubernetes executor by wefner in https://github.com/zenml-io/zenml/pull/994
* Support pydantic subclasses in BaseParameter attributes by schustmi in https://github.com/zenml-io/zenml/pull/1023
* Unify run names across orchestrators by schustmi in https://github.com/zenml-io/zenml/pull/1025
* Add gRPC metadata service to the ZenML helm chart by stefannica in https://github.com/zenml-io/zenml/pull/1026
* Make the MLMD pipeline run information transfer synchronous by stefannica in https://github.com/zenml-io/zenml/pull/1032
* Add console spinner back by strickvl in https://github.com/zenml-io/zenml/pull/1034
* Fix Azure CLI auth problem by wjayesh in https://github.com/zenml-io/zenml/pull/1035
* Allow non-local Label Studio instances for annotation by strickvl in https://github.com/zenml-io/zenml/pull/1033
* Before deleting the global zen_server files, spin it down by AlexejPenner in https://github.com/zenml-io/zenml/pull/1029
* Adding zenserver integration to stack recipe CLI by wjayesh in https://github.com/zenml-io/zenml/pull/1017
* Add support for Azure ZenServer by wjayesh in https://github.com/zenml-io/zenml/pull/1024
* Kubernetes Pod settings by schustmi in https://github.com/zenml-io/zenml/pull/1008

New Contributors
* alvarobartt made their first contribution in https://github.com/zenml-io/zenml/pull/1009
* wefner made their first contribution in https://github.com/zenml-io/zenml/pull/994

**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.5...0.21.0

ZenML 0.20.5 fixes another series of minor bugs, significantly improves the performance of the CLI, and adds an option to specify APT packages in Docker images.

What's Changed
* Fix accessing local zen store and artifact store in containers by stefannica in https://github.com/zenml-io/zenml/pull/976
* K3d local registry pod spec updated by wjayesh in https://github.com/zenml-io/zenml/pull/972
* Update readme page by dnth in https://github.com/zenml-io/zenml/pull/985
* Remove beam dependency by schustmi in https://github.com/zenml-io/zenml/pull/986
* Fix error message when registering secret without secrets manager by schustmi in https://github.com/zenml-io/zenml/pull/981
* Update cheat sheet up to `zenml==0.20.4` by dnth in https://github.com/zenml-io/zenml/pull/987
* Example fixes (part 2) by strickvl in https://github.com/zenml-io/zenml/pull/971
* Allow duplicate step classes inside a pipeline by schustmi in https://github.com/zenml-io/zenml/pull/989
* Include deployment in azureml docker build by schustmi in https://github.com/zenml-io/zenml/pull/984
* Automatically open browser upon `zenml up` command by dnth in https://github.com/zenml-io/zenml/pull/978
* Add a `just_mine` flag for `zenml stack list` by strickvl in https://github.com/zenml-io/zenml/pull/979
* Add option to specify apt packages by schustmi in https://github.com/zenml-io/zenml/pull/982
* Replace old flavor references, fix the windows local ZenML server and other fixes by stefannica in https://github.com/zenml-io/zenml/pull/988
* Improve docker and k8s detection by schustmi in https://github.com/zenml-io/zenml/pull/991
* Update GH actions example by schustmi in https://github.com/zenml-io/zenml/pull/993
* Update `MissingStepParameterError` exception message by gabrielmbmb in https://github.com/zenml-io/zenml/pull/996
* Seprated code docs into `core` and `integration` docs by AlexejPenner in https://github.com/zenml-io/zenml/pull/983
* Add docs/mkdocstrings_helper.py to format script sources by fa9r in https://github.com/zenml-io/zenml/pull/997
* Further CLI optimization by bcdurak in https://github.com/zenml-io/zenml/pull/992

**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.4...0.20.5

This release fixes another series of minor bugs that were introduced in 0.20.0.

What's Changed
* Detect failed executions by schustmi in https://github.com/zenml-io/zenml/pull/964
* Only build docker images for custom deployments by schustmi in https://github.com/zenml-io/zenml/pull/960
* M1 Mac Installation Tutorial by fa9r in https://github.com/zenml-io/zenml/pull/966
* Update ZenBytes links in docs by fa9r in https://github.com/zenml-io/zenml/pull/968
* Fix the API docs builder by stefannica in https://github.com/zenml-io/zenml/pull/967
* Fix `gpu_limit` condition in `VertexOrchestrator` by gabrielmbmb in https://github.com/zenml-io/zenml/pull/963
* Add simple node affinitiy configurations by schustmi in https://github.com/zenml-io/zenml/pull/973
* First iteration of the CLI optimization by bcdurak in https://github.com/zenml-io/zenml/pull/962


**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.3...0.20.4

This release fixes another series of minor bugs that were introduced in 0.20.0 and add full support docs for [Kubeflow multi-tenancy deployments](https://docs.zenml.io/component-gallery/orchestrators/kubeflow#important-note-for-multi-tenancy-deployments).

What's Changed
* Fixed GitHub/Colab JSON formatting error on quickstart. by fa9r in https://github.com/zenml-io/zenml/pull/947
* Update YAML config template by htahir1 in https://github.com/zenml-io/zenml/pull/952
* correct code from merge and fix import by wjayesh in https://github.com/zenml-io/zenml/pull/950
* Check for active component using id instead of name by schustmi in https://github.com/zenml-io/zenml/pull/956
* Tekton fix by htahir1 in https://github.com/zenml-io/zenml/pull/955
* Improve zenml up/down UX and other fixes by stefannica in https://github.com/zenml-io/zenml/pull/957
* Update kubeflow docs for multi-tenant deployments by htahir1 in https://github.com/zenml-io/zenml/pull/958
* Update kubeflow.md by abohmeed in https://github.com/zenml-io/zenml/pull/959
* Add additional stack validation for step operators by schustmi in https://github.com/zenml-io/zenml/pull/954
* Fix pipeline run dashboard URL for unlisted runs by fa9r in https://github.com/zenml-io/zenml/pull/951
* Support subclasses of registered types in recursive materialization by fa9r in https://github.com/zenml-io/zenml/pull/953

New Contributors
* abohmeed made their first contribution in https://github.com/zenml-io/zenml/pull/959

**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.2...0.20.3

After a successful release of the new ZenML server and dashboard paradigm, we set to ironing out some bugs that slipped through.

What's Changed
* Capitalize all docs page titles. by fa9r in https://github.com/zenml-io/zenml/pull/937
* Increase field sizes for docstrings and step parameters. by fa9r in https://github.com/zenml-io/zenml/pull/940
* Fixing the bug in the registration of custom flavors by bcdurak in https://github.com/zenml-io/zenml/pull/938
* Implemented `docstring` Attribute of StepModel by fa9r in https://github.com/zenml-io/zenml/pull/936
* Fix shared stack emoji by strickvl in https://github.com/zenml-io/zenml/pull/941
* Fix shared stacks not being allowed to be set as active. by fa9r in https://github.com/zenml-io/zenml/pull/943
* Typo fix by strickvl in https://github.com/zenml-io/zenml/pull/944
* Update Kubernetes Orchestrator Example by fa9r in https://github.com/zenml-io/zenml/pull/942
* Add code and instructions to run quickstart on Colab. by fa9r in https://github.com/zenml-io/zenml/pull/939
* Fixing the interaction in getting stacks/components by bcdurak in https://github.com/zenml-io/zenml/pull/945
* Fix Kubeflow run name by safoinme in https://github.com/zenml-io/zenml/pull/946
* `VertexOrchestrator` apply node selector constraint if `gpu_limit > 0` by gabrielmbmb in https://github.com/zenml-io/zenml/pull/935

**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.1...0.20.2

<img src="https://user-images.githubusercontent.com/3963946/194079497-46355e0d-f837-42d5-986e-92210a5e3c1e.gif" width="500" height="500" />

The ZenML 0.20.0 release brings a number of big changes to its architecture and a lot of cool new features, some of which are not backwards compatible with previous versions.

These changes are only covered briefly in the release notes. For a detailed view on what happened and how you can get the most out of the 0.20.0 release, please head over to [our "ZenML 0.20.0: Our Biggest Release Yet" blog post](https://blog.zenml.io/zenml-revamped).

Warning: Breaking Changes

Updating to ZenML 0.20.0 needs to be followed by a migration of your existing ZenML Stacks and you may also need to make changes to your current ZenML pipeline code. Please read [the migration guide](https://docs.zenml.io/guidelines/migration-zero-twenty) carefully and follow the instructions to ensure a smooth transition. The guide walks you through these changes and offers instructions on how to migrate your existing ZenML stacks and pipelines to the new version with minimal effort and disruption to your existing workloads.

If you have updated to ZenML 0.20.0 by mistake or are experiencing issues with the new version, you can always go back to the previous version by using `pip install zenml==0.13.2` instead of `pip install zenml` when installing ZenML manually or in your scripts.

Overview of Changes

* [ZenML takes over the Metadata Store](https://docs.zenml.io/guidelines/migration-zero-twenty#zenml-takes-over-the-metadata-store-role) role. All information about your ZenML Stacks, pipelines, and artifacts is now tracked by ZenML itself directly. If you are currently using remote Metadata Stores (e.g. deployed in cloud) in your stacks, you will probably need to replace them with [ZenML cloud deployments](https://docs.zenml.io/guidelines/migration-zero-twenty/getting-started/deploying-zenml/deploying-zenml.md).
* the [new ZenML Dashboard](https://docs.zenml.io/guidelines/migration-zero-twenty#the-zenml-dashboard-is-now-available) is now available with all ZenML deployments.
* [ZenML Profiles have been removed](https://docs.zenml.io/guidelines/migration-zero-twenty#removal-of-profiles-and-the-local-yaml-database) in favor of ZenML Projects. You need to [manually migrate your existing ZenML Profiles](https://docs.zenml.io/guidelines/migration-zero-twenty#how-to-migrate-your-profiles) after the update.
* the [configuration of Stack Components is now decoupled from their implementation](https://docs.zenml.io/guidelines/migration-zero-twenty#decoupling-stack-component-configuration-from-implementation). If you extended ZenML with custom stack component implementations, you may need to update the way they are registered in ZenML.
* the updated ZenML server provides a new and improved collaborative experience. When connected to a ZenML server, you can now [share your ZenML Stacks and Stack Components](https://docs.zenml.io/guidelines/migration-zero-twenty#shared-zenml-stacks-and-stack-components) with other users. If you were previously using the ZenML Profiles or the ZenML server to share your ZenML Stacks, you should switch to the new ZenML server and Dashboard and update your existing workflows to reflect the new features.

What's Changed
* Fix error in checking Great Expectations results when exit_on_error=True by TimovNiedek in https://github.com/zenml-io/zenml/pull/889
* feat(user-dockerfile): Add user argument to DockerConfiguration by cjidboon94 in https://github.com/zenml-io/zenml/pull/892
* Minor doc updates for backporting by htahir1 in https://github.com/zenml-io/zenml/pull/894
* Removed feature request and replaced with hellonext board by htahir1 in https://github.com/zenml-io/zenml/pull/897
* Unit tests for (some) integrations by strickvl in https://github.com/zenml-io/zenml/pull/880
* Fixed integration installation command by edshee in https://github.com/zenml-io/zenml/pull/900
* Pipeline configuration and intermediate representation by schustmi in https://github.com/zenml-io/zenml/pull/898
* [Bugfix] Fix bug in auto-import of stack after recipe deploy by wjayesh in https://github.com/zenml-io/zenml/pull/901
* Update TOC on CONTRIBUTING.md by strickvl in https://github.com/zenml-io/zenml/pull/907
* ZenServer by fa9r in https://github.com/zenml-io/zenml/pull/879
* Update `kserve` README by strickvl in https://github.com/zenml-io/zenml/pull/912
* Confirmation prompts were not working by htahir1 in https://github.com/zenml-io/zenml/pull/917
* Stacks can be registered in `Click<8.0.0` now by AlexejPenner in https://github.com/zenml-io/zenml/pull/920
* Made Pipeline and Stack optional on the HydratedPipelineRunModel by AlexejPenner in https://github.com/zenml-io/zenml/pull/919
* Renamed all references from ZenServer to ZenML Server in logs and comments by htahir1 in https://github.com/zenml-io/zenml/pull/915
* Prettify pipeline runs list CLI output. by fa9r in https://github.com/zenml-io/zenml/pull/921
* Warn when registering non-local component with local ZenServer by strickvl in https://github.com/zenml-io/zenml/pull/904
* Fix duplicate results in pipeline run lists and unlisted flag. by fa9r in https://github.com/zenml-io/zenml/pull/922
* Fix error log by htahir1 in https://github.com/zenml-io/zenml/pull/916
* Update cli docs by AlexejPenner in https://github.com/zenml-io/zenml/pull/913
* Fix Pipeline Run Status by fa9r in https://github.com/zenml-io/zenml/pull/923
* Change the CLI emoji for whether a stack is shared or not. by fa9r in https://github.com/zenml-io/zenml/pull/926
* Fix running pipelines from different locations. by fa9r in https://github.com/zenml-io/zenml/pull/925
* Fix zenml stack-component describe CLI command. by fa9r in https://github.com/zenml-io/zenml/pull/929
* Update custom deployment to use ArtifactModel by safoinme in https://github.com/zenml-io/zenml/pull/928
* Fix the CI unit test and integration test failures by stefannica in https://github.com/zenml-io/zenml/pull/924
* Add gcp zenserver recipe by wjayesh in https://github.com/zenml-io/zenml/pull/930
* Extend Post Execution Class Properties by fa9r in https://github.com/zenml-io/zenml/pull/931
* Fixes for examples by strickvl in https://github.com/zenml-io/zenml/pull/918
* Update cheat sheet by dnth in https://github.com/zenml-io/zenml/pull/932
* Fix the docstring attribute of pipeline models. by fa9r in https://github.com/zenml-io/zenml/pull/933
* New docs post ZenML Server by htahir1 in https://github.com/zenml-io/zenml/pull/927

New Contributors
* TimovNiedek made their first contribution in https://github.com/zenml-io/zenml/pull/889
* cjidboon94 made their first contribution in https://github.com/zenml-io/zenml/pull/892
* edshee made their first contribution in https://github.com/zenml-io/zenml/pull/900

**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.13.2...0.20.0