Mlrun

Features / Enhancements
* **Logging**: Support setting log level per run. 321, yaronh
* **Monitoring**: Add job name label to k8s resources. 324, hedingber
* **CI**: Log docker build commands output. 319, orkiguazio
* **Testing**: Add more validations in integration tests. 316, hedingber

Bug fixes
* **Schedules**: Fix scheduled job submission bug. 317, hedingber
* **Remote work**: Fix remote detection. 325, hedingber
* **Python SDK**: Align api client's (httpdb) store_function to return hash key. 323, hedingber
* **Cleanup**: Don't fail cleanup on one resource deletion failure. 322, hedingber

Major features:
* Automatic resources cleanup
* New helper function - `set_environment` to simplify setting environment to work remotely
* Simpler process to upgrade mlrun on iguazio systems
* Bunch of bug fixes and docs improvements

Pull requests:
a527d19 Change makefile defaults (315)
7bd1393 Add environment setup helper function (312)
194991a Fix default logs path (314)
ce16f4e Trigger runtimes cleanup periodically (302)
52ca2eb Default Nuclio function min replicas to 1 (311)
a168473 Bump version to 0.4.9 (310)
aa26261 Initial Model Management and Serving doc (305)
c2e0be1 prioritize hash key over tag (308)
0d88eb0 Change SQLDB to be the default DB (309)
261a0a0 fix: mlrun-models-gpu-legacy push correct name (306)

**Major features:**
- new backend based on FastAPI and SQL
- add pipeline watch option and git/slack notifications, gitops
- enhance hyper-params: add list, random, and custom iterations from file or inline
- context.get_child(), context.commit_children() to manage child context and iteration
- load function as an inline python module (function_to_module())
- Enhance log_model/get_model(), add update_model() functions
- simplify and improve remote access and job/workflow submission
- minimal env var settings, load settings from server, simpler auth
- simplify configuration and reduce boilerplate code
- e.g. auto_mount, extend code_to_function
- improve documentation, complete readthedocs guide
- more exception and user error handling
- enhance mlutils lib with pre-baked model evaluation and plots
- add support for mpi (horovod) operator v1
- cli/api for monitoring and delete of function resources (runtimes)
- enhance CI/CD: auto test notebooks, work with remote k8s, jenkins, ..
- a bunch of bug fixes and api enhancements

Pull requests:
e5c55c6 minor-048-fixes (298)
3eedccc MLRun Docs update (294)
f1ae9df fix-env-issues (295)
fceba8a update-mlutils (292)
164184a Fixing mpijob crd resolve from remote (290)
13638b4 load-mod-and-fixes (289)
9116425 Adding runtimes handlers (285)
7da2b2f Run enhancments (288)
0757dda Add jenkinsfile (286)
e27a657 fix get pipelines (return valid dict) (284)
9dd963e Get pipeline changes (not verbose on wait + refactor) (281)
54305ec Support for mpijob v1 (275)
6f1117e fix bug a missing/wrong model (282)
f74e5e1 Makefile improvements (276)
fd47502 Add GET pipelines endpoint (280)
95c95d3 Change path operations with mix os sync and async to be async def and (279)
9301477 Fixing labels query param typo (278)
ef9f2f Fix function tags handling (271)
0d69e4e bring back plot_importance for compatibility (274)
d1a9d36 Cleanup httpdb test leftovers (272)
b2334e4 Improve hyperparam + fixes (273)
ac0dde5 ML images - moving packages to requirements files (269)
4409614 PHONY targets should be separated by space not comma (270)
1150218 add verbose print and fix v3io stream (268)
c015be3 Adding dask notebook to integration test (267)
ab5d9c5 Scripts -> makefile (265)
00cd1df Adding manual integration tests (266)
301862b Refactor httpd to organized app using FastAPI (255)
8d90457 Mlutils 1 (264)
1ca2e37 Gitops and remote access improvements (263)
5f9de0a fix build cli
057ff63 update jupy docker

Project:
* embed workflow in project ymal
* load project from yaml (in addition to repo)
* support params and secrets in project cli

KFP:
* List workflows API
* TTL and cleanup
* pass env, tag, verbose in deploy_op
* use function links vs in YAML (smaller KFP YAML)

Data
* DataItems: add .as_df() and .local() methods
* support dir logging and .listdir() for local/s3/v3io
* refactor datastore as a separate module

Models and serving:
* add base serving class
* add context.log_model()
* extend ModelArtifact class to support many new attributes and options
* get_model(), generic mechanism to read models + its files and metadata
* inference stream support (record all inference inputs, outputs, metrics, and metadata to a stream)

Dockerfiles
* enhance, doc and centralize containers build (see /dockerfiles)

Other
* Handle credentials/identity forwarding for secure UI & data access
* Bug fixes

New features and enhancements:
* Support function marketplace (`hub://`)
* Enhanced projects support
* add mlutils for auto ML + pre-baked ml images
* simple feature/artifact store (use `store://` paths)
* extend context APIs (add log_dataset, etc.)
* improved UI support
* add robustness, HTTP retry, etc.
* various fixes

* add documentation for running on docker and Kubernetes
* improve defaults and add env vars for simpler deployment