Mlrun

Features / Enhancements
* **Unknown**: Revert "[feature store] use timestamp key to get online features", 4998, gtopper
* **DB**: Lock run db row on store run, 4995, alonmr
* **Model Monitoring**: Log windows result in the controller job, 4994, jond01
* **Batch Infer**: Raise an error when using fv as a dataset without applying `get_offline_features`, 4986, Eyal-Danieli
* **Feature-store + Docs**: Update spark examples + reduce fset.api.ingest() deprecation message scope, 4991, alxtkr77
* **LogCollector**: Remedy vulnerabilities, 4993, liranbg
* **Config**: Migrate `pytest.ini` config to `pyproject.toml`, 4992, jond01
* **Feature Store**: Use timestamp key to get online features, 4989, gtopper
* **CI**: Enforce mlrun now on test scheduler, 4990, liranbg
* **API**: Ensure project deletion in mlrun follower, 4984, alonmr
* **SDK**: Remove deprecated fields, 4953, alonmr
* **Iguazio**: Cache project deletion jobs to avoid consecutive requests, 4969, TomerShor
* **Spark**: Patch pyspark to work with numpy>=1.24.2, 4980, gtopper
* **Feature Store**: Change deprecated `fstore.ingest(fset)` calls to `fset.ingest()`, 4983, gtopper
* **Model Monitoring**: Raise `mlruninvalidargumenterror` if the `modelmonitoringapplicationresult` name didn't match [a-za-z_][a-za-z0-9_]* format, 4985, davesh0812
* **Model Monitoring**: Update "last_updated" in `record_results`, 4982, jond01
* **Model Monitoring**: Delete controller job if the schedule submission went wrong, 4968, davesh0812
* **Tests**: Upgrade model to scikit-learn==1.4.0, 4981, davesh0812
* **Datastore**: Return correct path in kafka and stream targets, 4967, gtopper
* **Model Monitoring**: Update deprecated ingest usage, 4974, jond01
* **Requirements**: Remedy vulnerabilities, 4973, liranbg
* **Artifacts**: Overwrite `spec.db_key` when importing artifact with `new_key` arg, 4970, TomerShor
* **Automation**: Nest ce deployment args under object, 4964, alonmr
* **Docs**: Add how to use with_source_archive, 4952, jillnogold
* **API**: Verify project deletion with new db session, 4972, alonmr
* **Functions**: Support different types of sample_set in get_sample_set_statistics., 4911, tomerm-iguazio
* **Project**: Improve workflow path validation, 4965, alonmr
* **Requirements**: Limit pandas to <2.2 because it requires sqlalchemy 2, 4971, gtopper
* **Artifacts**: Datasets should only allow an existing column to be used as label, 4957, laurybueno
* **Projects**: Delete project background task support for iguazio <= 3.5.4, 4954, quaark
* **Docs**: Add datastore profiles details for gcs, dbfs, azure, 4791, jillnogold
* **Dataset**: Document overwriting behavior of `log_dataset`, 4956, TomerShor
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc23#features-and-enhancements)

Bug fixes
* **CI**: Fix some tests, 4997, liranbg
* **API**: Fix background jobs not playing nice with fastapi middlewares, 4988, liranbg
* **Docs**: Fix 'project' command of mlrun cli, 4922, jillnogold
* **Model monitoring**: Fix logging conventions, 4977, alonmr
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc23#bug-fixes)


Pull requests:
d8935ec6 Revert "[Feature Store] Use timestamp key to get online features" (4998)
983d7efc [CI] Fix some tests (4997)
4c7de219 [DB] Lock run db row on store run (4995)
1a160698 [Model Monitoring] Log windows result in the controller job (4994)
38baddcc [Batch Infer] Raise an error when using FV as a dataset without applying `get_offline_features` (4986)
35310889 [Feature-store + Docs] Update Spark examples + reduce fset.api.ingest() deprecation message scope (4991)
5adb05ea [LogCollector] Remedy vulnerabilities (4993)
209e7cbb [Config] Migrate `pytest.ini` config to `pyproject.toml` (4992)
35591363 [Feature Store] Use timestamp key to get online features (4989)
de344af0 [CI] Enforce mlrun now on test scheduler (4990)
de6ab8ea [API] Fix background jobs not playing nice with fastapi middlewares (4988)
5d13b8b2 [API] Ensure project deletion in mlrun follower (4984)
8d677042 [SDK] Remove deprecated fields (4953)
3f581327 [Iguazio] Cache project deletion jobs to avoid consecutive requests (4969)
5b4e15c9 [Spark] Patch pyspark to work with numpy>=1.24.2 (4980)
8860ac7a [Feature Store] Change deprecated `fstore.ingest(fset)` calls to `fset.ingest()` (4983)
aa42ebb3 [Model Monitoring] raise `MLRunInvalidArgumentError` if the `ModelMonitoringApplicationResult` name didn't match [a-zA-Z_][a-zA-Z0-9_]* format (4985)
bf321680 [Model Monitoring] Update "last_updated" in `record_results` (4982)
82a20113 [Model Monitoring] Delete controller job if the schedule submission went wrong (4968)
d67a76ba [Tests] Upgrade model to scikit-learn==1.4.0 (4981)
ce7256e0 [Docs] Fix 'project' command of mlrun cli (4922)
ee4d9e66 [Datastore] Return correct path in kafka and stream targets (4967)
755b2e56 [Model Monitoring] Update deprecated ingest usage (4974)
cf4e7882 [Model monitoring] Fix logging conventions (4977)
70a9ae12 [Requirements] Remedy vulnerabilities (4973)
b8d338e4 [Artifacts] Overwrite `spec.db_key` when importing artifact with `new_key` arg (4970)
768b1f1c [Automation] Nest CE deployment args under object (4964)
c54e402a [Docs] Add how to use with_source_archive (4952)
3288e164 [API] Verify project deletion with new db session (4972)
106cd937 [Functions] support different types of sample_set in get_sample_set_statistics. (4911)
2bae9932 [Project] Improve workflow path validation (4965)
a1b761f2 [Requirements] Limit pandas to <2.2 because it requires sqlalchemy 2 (4971)
3d065703 [Artifacts] Datasets should only allow an existing column to be used as label (4957)
16ee968b [Projects] Delete Project Background Task Support for Iguazio <= 3.5.4 (4954)
9a6c6538 [Docs] Add datastore profiles details for GCS, DBFS, Azure (4791)
b8c6a8b4 [Dataset] Document overwriting behavior of `log_dataset` (4956)

Features / Enhancements
* **Model Monitoring**: Remove `last_in_batch_set` and use "now" in the controller for batch infer, 4962, jond01
* **Docs**: Replace deprecated sql parameter in documentation, 4959, gtopper
* **V3io datastore**: Use requests session when doing put in chunks, 4958, assaf758
* **Data Store**: Delete deprecated `time_fields` param from `sqltarget` & `sqlsource`., 4955, davesh0812
* **Feature Store**: Replace calls to deprecated `deploy_ingestion_service_v2()`, 4950, gtopper
* **Model Monitoring**: Update deprecated `get_offline_features` calls, 4948, jond01
* **SQLDB**: Lock run row on update, 4942, alonmr
* **Model Monitoring**: Improve the controller job's logs, 4860, jond01
* **Run**: Save enriched function to db when submitting job schedule with custom spec, 4943, TomerShor
* **Automation**: Extend mlrun ce deploy script to allow for a forceful kubeflow pipelines installation, 4944, laurybueno
* **Datastore**: Refactor internal get_filesystem() method, 4924, alxtkr77
* **Artifacts**: Dataset metadata generation must be mindful of differences among header, preview and schema, 4918, laurybueno
* **Builder**: Mount aws credentials to /tmp/aws, 4941, alonmr
* **Packaging**: Add python requirement to `mlrun` package metadata, 4940, jond01
* **Datastore**: Remove run id from `streamtarget` path, 4939, gtopper
* **DB**: Support case-insensitive label queries, 4937, theSaarco
* **SDK**: Todos and deprecations chore, 4925, alonmr
* **CI**: Update dev_utilities.py, 4928, eliyahu77
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc22#features-and-enhancements)

Bug fixes
* **Artifacts**: Fix tag deletion to target only specified artifacts, 4961, Yacouby
* **SDK**: Fix warning typo, 4960, alonmr
* **Databricks**: Fix abortion state transitions, 4945, alonmr
* **Model Monitoring**: Fix "divide by zero" warning in kl divergence metric, 4947, jond01
* **Model Monitoring**: Fix bumping last_request without controller runs, 4946, jond01
* **Execution**: Fix non logging worker lables, 4938, alonmr
* **Model Monitoring**: Fix `remove_model_monitoring_function` to remove function from db, 4949, davesh0812
* **Model Monitoring**: Fix potential last window overflow, 4894, jond01
* **Tests**: Fix `attributeerror` error due to typo, 4936, gtopper
* **Model Monitoring**: Fix (remove) redundent convert to `datetime` in stream pod, 4935, davesh0812
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc22#bug-fixes)


Pull requests:
ae3fdb8b [Model Monitoring] Remove `last_in_batch_set` and use "now" in the controller for batch infer (4962)
4e12dfb3 [Artifacts] Fix tag deletion to target only specified artifacts (4961)
ba399e9f [SDK] Fix warning typo (4960)
7d93abd0 [Databricks] Fix abortion state transitions (4945)
13ca4b61 [Model Monitoring] Fix "divide by zero" warning in KL divergence metric (4947)
8acbe3f7 [Docs] Replace deprecated SQL parameter in documentation (4959)
a4f347be [V3io datastore] use requests session when doing put in chunks (4958)
cf06614b [Data Store] Delete deprecated `time_fields` param from `SQLTarget` & `SQLSource`. (4955)
c49c7256 [Feature Store] Replace calls to deprecated `deploy_ingestion_service_v2()` (4950)
bbc0e374 [Model Monitoring] Fix bumping last_request without controller runs (4946)
5c82445a [Execution] Fix non logging worker lables (4938)
e6f37784 [Model Monitoring] Fix `remove_model_monitoring_function` to remove function from db (4949)
d30fab04 [Model Monitoring] Update deprecated `get_offline_features` calls (4948)
d8422c04 [SQLDB] Lock run row on update (4942)
9835fe4f [Model Monitoring] Improve the controller job's logs (4860)
f97a8e45 [Run] Save enriched function to db when submitting job schedule with custom spec (4943)
ef867251 [Automation] Extend MLRun CE deploy script to allow for a forceful Kubeflow Pipelines installation (4944)
8bc79c98 [Datastore] Refactor internal get_filesystem() method (4924)
ecc696c9 [Model Monitoring] Fix potential last window overflow (4894)
67cf9241 [Artifacts] Dataset metadata generation must be mindful of differences among header, preview and schema (4918)
de10b3b2 [Builder] Mount AWS credentials to /tmp/aws (4941)
65c216f0 [Packaging] Add Python requirement to `mlrun` package metadata (4940)
351989a4 [Datastore] Remove run ID from `StreamTarget` path (4939)
154c1e7d [DB] Support case-insensitive label queries (4937)
44d8a37b [Tests] Fix `AttributeError` error due to typo (4936)
a6a09236 [SDK] Todos and deprecations chore (4925)
7f5ba8c5 [Model Monitoring] Fix (remove) redundent convert to `datetime` in stream pod (4935)
d4441513 [CI] Update dev_utilities.py (4928)

Features / Enhancements
* **Datastore**: Rename access_key to access_key_id in s3 datastore-profile private, 4934, tomerm-iguazio
* **Datastore**: Filter out 'none' values when passing storage options for azure, 4923, alxtkr77
* **API**: Non blocking get project before delete, 4932, alonmr
* **Tests**: Remove system tests ini file, 4931, alonmr
* **Execution**: Don't add host label to non logging workers, 4929, alonmr
* **Model Monitoring**: Change default last analyzed to be the latest between first request time or last update time, 4898, davesh0812
* **Kafka**: Force number of replicas of 1 when nuclio is too old, 4920, gtopper
* **Docs**: Add example of build_image using project source, 4916, jillnogold
* **API**: Support list project background tasks, 4914, alonmr
* **Artifacts**: Retain `db_key` when logging artifact, 4917, TomerShor
* **Project**: Raise error on invalid export path, 4903, alxtkr77
* **Functions**: Add more labels to the image builder pods to facilitate resource usage tracking, 4906, laurybueno
* **CI**: Use javascript regex conventions, 4913, liranbg
* **API**: Revert `/api/healthz` to support iguazio <= 3.5.3, 4909, quaark
* **CI**: Pr title validation, 4907, liranbg
* **Build**: Warn ecr and s3 permissions conflict, 4902, alonmr
* **Requirements**: Bump storey to 1.6.16, 4908, gtopper
* **Config**: Add ds to allowed file paths, 4904, alxtkr77
* **Datastore**: Rename access key to access_key_id in s3., 4901, tomerm-iguazio
* **Datastore**: Add validation for empty url, 4899, gtopper
* **Automation**: Support patch with data node registry, 4900, alonmr
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc21#features-and-enhancements)

Bug fixes
* **Tests**: Fix context carryover between model monitoring tests, 4933, gtopper
* **Model Monitoring**: Fix bumping the last request for batch inference, 4926, jond01
* **Core**: Fix deprecated numpy type, 4930, alxtkr77
* **Runtimes**: Fix databricks wrapper backward compatibility, 4927, tomerm-iguazio
* **Serving**: Fix setting of max_workers for v3io-stream-trigger to use maxworkers name, 4910, gtopper
* **Kafka**: Bump v3io-frames to fix numpy compatibility issue, 4921, gtopper
* **Projects**: Fix `delete /api/v2/projects` not returning 403 on permission denied, 4919, quaark
* **CI**: Match whole pr title, not just prefix, 4912, liranbg
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc21#bug-fixes)


Pull requests:
0ac52869 [Datastore] Rename access_key to access_key_id in S3 datastore-profile private (4934)
55836f44 [Datastore] Filter out 'none' values when passing storage options for Azure (4923)
1de88a56 [Tests] Fix context carryover between model monitoring tests (4933)
f1c071f6 [API] Non blocking get project before delete (4932)
4068b2d1 [Model Monitoring] Fix bumping the last request for batch inference (4926)
bb74ac4e [Core] Fix deprecated numpy type (4930)
8ade0f92 [Tests] Remove system tests ini file (4931)
9df71e95 [Execution] Don't add host label to non logging workers (4929)
b862f0e4 [Model Monitoring] Change default last analyzed to be the latest between first request time or last update time (4898)
2e0a8f8b [Kafka] Force number of replicas of 1 when nuclio is too old (4920)
2e56b006 [Runtimes] Fix Databricks wrapper backward compatibility (4927)
b646825a [Serving] Fix setting of max_workers for v3io-stream-trigger to use maxWorkers name (4910)
72e301f4 [Docs] Add example of build_image using project source (4916)
ed832ef2 [Kafka] Bump v3io-frames to fix numpy compatibility issue (4921)
b36b1ed8 [Projects] Fix `DELETE /api/v2/projects` not returning 403 on permission denied (4919)
c7d3d0d1 [API] Support list project background tasks (4914)
b8fc1557 [Artifacts] Retain `db_key` when logging artifact (4917)
0b6be4dc [Project] Raise error on invalid export path (4903)
111dbb16 [Functions] Add more labels to the image builder pods to facilitate resource usage tracking (4906)
a623e4f4 [CI] Use Javascript regex conventions (4913)
fb5642ba [CI] Match whole pr title, not just prefix (4912)
a0439efe [API] Revert `/api/healthz` to support Iguazio <= 3.5.3 (4909)
0c7b98f4 [CI] PR title validation (4907)
e0d0c004 [Build] Warn ECR and S3 permissions conflict (4902)
529af25e [Requirements] Bump storey to 1.6.16 (4908)
b62fc371 [Config] Add ds to allowed file paths (4904)
7ee3b2fc [Datastore] Rename access key to access_key_id in s3. (4901)
6d206542 [Datastore] Add validation for empty URL (4899)
4f488b93 [Automation] Support patch with data node registry (4900)

Features / Enhancements
* **Spark**: Default spark event log dir to `/tmp`, 4896, gtopper
* **Workflows**: When the user tries to run a non-existing workflow, raise a clear error message, 4795, laurybueno
* **Project**: Take project source into account when running `build_image`, 4892, theSaarco
* **Workflow**: Ensure "owner" label exists on runner, 4881, TomerShor
* **Project**: Delete project background task mark ii, 4869, quaark
* **Datastore**: Databricks artifact path - support another type of spark path., 4895, tomerm-iguazio
* **Artifacts**: Don't overwrite `metadata.key` when storing artifact, 4887, TomerShor
* **Artifacts**: Remove updatable fields from uid calc, 4893, alonmr
* **Artifacts**: Calculate artifact size on create artifact, 4890, Yacouby
* **Spark**: Add support for gcs datastore profiles, 4859, alxtkr77
* **Datastore**: Allow passing gcs credentials as a json string, 4874, alxtkr77
* **Model Monitoring**: Improve the model monitoring application python api, 4866, jond01
* **Serving**: Allow for setting `max_workers` in kafka trigger and change default from 1 to 4, 4888, gtopper
* **Docs**: Updates for fs/fv apis are now featureset / featurevector class methods, 4883, jillnogold
* **Requirements**: Relax aiohttp requirement to resolve security vulnerabilities, 4886, gtopper
* **Requirements**: Change numpy version for tensorflow 2.14, 4885, tomerm-iguazio
* **Project**: Enable adding git to a project after project creation, 4878, Yacouby
* **Requirments**: Bump storey to v1.6.15, 4882, assaf758
* **Requirments**: Bump storey to v1.6.15, 4882, assaf758
* **Databricks**: Bypass intermediate completed state on abortion, 4879, alonmr
* **Context**: Verify logging worker before storing run, 4880, alonmr
* **Spark**: Make spark work with azure datastore profile, 4826, alxtkr77
* **CI**: Enable log collector in deployer and os system tests, 4873, alonmr
* **CI**: Timeout pytest cases, 4870, liranbg
* **Requirements**: Bump nuclio-jupyter to 0.9.14, 4868, TomerShor
* **Docs**: Add new_task example; improve: feature store engines descriptions, log_artifact, function handler, 4756, jillnogold
* **Nuclio**: Default runtime to python:3.9, 4857, alonmr
* **Background Tasks**: Lock internal background tasks by kind, 4865, quaark
* **RunDB**: Add get_log_size to rundb interface, 4867, TomerShor
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc20#features-and-enhancements)

Bug fixes
* **System test**: Fix expected artifact uri, 4897, alonmr
* **Model Monitoring**: Fix event processing to use timezone aware timestamps and objects, 4889, jond01
* **Notifications**: Fix list runs with notifications non-deterministic behaviour, 4891, quaark
* **Artifacts**: Fix list artifacts legacy tree as tag, 4884, alonmr
* **Model Monitoring**: Fix saving multiple metrics in the tsdb, 4872, jond01
* **Datastore**: Fix upload to v3io, 4877, liranbg
* **CI**: Fix test artifact keys, 4876, liranbg
* **Featurestore**: Fix wrong preview() deprecation message, 4871, alxtkr77
* **System tests**: Fix override log collector image, 4875, alonmr
* **Artifacts**: Fix artifact tagging, 4864, TomerShor
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc20#bug-fixes)


Pull requests:
e57839f8 [System test] Fix expected artifact URI (4897)
f077f150 [Spark] Default spark event log dir to `/tmp` (4896)
650931e5 [Workflows] When the user tries to run a non-existing workflow, raise a clear error message (4795)
129961c5 [Project] Take project source into account when running `build_image` (4892)
d07de5dc [Workflow] Ensure "owner" label exists on runner (4881)
73e45706 [Project] Delete Project Background Task Mark II (4869)
fbaedb4b [Datastore] Databricks artifact path - support another type of spark path. (4895)
bf13f879 [Artifacts] Don't overwrite `metadata.key` when storing artifact (4887)
e79e0fa9 [Artifacts] Remove updatable fields from UID calc (4893)
956fc4fc [Artifacts] calculate artifact size on create artifact (4890)
c5f131c8 [Spark] Add support for GCS datastore profiles (4859)
221e4a0a [Model Monitoring] Fix event processing to use timezone aware timestamps and objects (4889)
15e423a9 [Datastore] Allow passing GCS credentials as a json string (4874)
4a243702 [Notifications] Fix List Runs With Notifications Non-Deterministic Behaviour (4891)
a880fe5f [Model Monitoring] Improve the model monitoring application Python API (4866)
9bd2f88b [Serving] Allow for setting `max_workers` in Kafka trigger and change default from 1 to 4 (4888)
9773565b [Docs] Updates for FS/FV APIs are now FeatureSet / FeatureVector class methods (4883)
f69e17dd [Requirements] Relax aiohttp requirement to resolve security vulnerabilities (4886)
d60987ae [Requirements] Change numpy version for tensorflow 2.14 (4885)
bbd1c2ab [Artifacts] Fix list artifacts legacy tree as tag (4884)
71ce55d1 [Project] Enable adding git to a project after project creation (4878)
0ea75ae2 [Requirments] Bump storey to v1.6.15 (4882)
c51a1d5c [Requirments] Bump storey to v1.6.15 (4882)
1a7a0427 [Databricks] Bypass Intermediate completed state on abortion (4879)
63329871 [Context] Verify logging worker before storing run (4880)
8283e9b4 [Model Monitoring] Fix saving multiple metrics in the TSDB (4872)
00b122fc [Datastore] Fix upload to v3io (4877)
ff05b69b [Spark] Make spark work with azure datastore profile (4826)
cf20f356 [CI] Fix test artifact keys (4876)
1ecacd63 [Featurestore] Fix wrong preview() deprecation message (4871)
04cb0fc5 [System tests] Fix override log collector image (4875)
a2190204 [CI] Enable log collector in deployer and OS system tests (4873)
f6171d87 [Artifacts] Fix artifact tagging (4864)
3e004c18 [CI] Timeout pytest cases (4870)
ab0bf96d [Requirements] Bump nuclio-jupyter to 0.9.14 (4868)
d4492e08 [Docs] Add new_task example; improve: feature store engines descriptions, log_artifact, function handler (4756)
fdc23c1b [Nuclio] Default runtime to python:3.9 (4857)
b6606e70 [Background Tasks] Lock Internal Background Tasks by Kind (4865)
b8b958aa [RunDB] Add get_log_size to rundb interface (4867)

Features / Enhancements
* **Model Monitoring**: Save only the new fields that generated by the user to modelmonitoringapplicationresult, 4832, davesh0812
* **Docs**: Add mlflow tutorial, 4694, jillnogold
* **HTTPDB**: Keep watching on created and aborting states, 4852, TomerShor
* **Build**: Update build command, 4849, liranbg
* **Automation**: Support log collector image patch, 4850, alonmr
* **API**: Remove deprecated artifacts endpoints, 4810, alonmr
* **K8s**: Map k8s api exceptions to mlrun exceptions, 4843, alonmr
* **Docs**: Add creating and using a custom/private hub, 4710, jillnogold
* **CI**: Updating system test, 4846, liranbg
* **Requirements**: Bump storey to 1.6.14, 4847, gtopper
* **Artifacts**: Add the option to calculate the size of artifact, 4828, Yacouby
* **Logs**: Add get log size endpoint and sdk method, 4836, TomerShor
* **Project**: Set default build base image, 4842, alonmr
* **Model Monitoring**: Save all the metrics of an application in kv storage, 4841, jond01
* **Spark**: Remove logging of spark options, 4839, alxtkr77
* **Datastore**: Log databricks metadata run as an artifact, 4831, tomerm-iguazio
* **Requirements**: Bump storey to 1.6.12, 4838, gtopper
* **CI**: Run system test on self hosted runners, 4837, liranbg
* **API**: Force delete pods after grace period passed, 4822, alonmr
* **Logs**: Validate positive offset when getting logs, 4829, TomerShor
* **Model Monitoring**: Remove the factor from inf edge detection, 4827, jond01
* **API**: Skip monitoring when crd state is not populated, 4811, alonmr
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc19#features-and-enhancements)

Bug fixes
* **CI**: Fix open source system test missing deleted argument, 4862, liranbg
* **CI**: Fix system test existing project, 4861, liranbg
* **Artifacts**: Fix project level artifact path template, 4854, quaark
* **Model Monitoring**: Fix delete old parquet function, 4830, davesh0812
* **API**: Fix delete runs error status code, 4855, alonmr
* **Automation**: Patch image name hotfix, 4856, alonmr
* **Requirements**: Fix databricks requirements, 4851, tomerm-iguazio
* **CI**: Fix system tests using git examples, 4853, liranbg
* **API**: Fix log of runtime attribute, 4848, alonmr
* **DB**: Fix reusing db sessions, 4840, liranbg
* **Doc**: Add `kuberesource` to docs and fix some malformed doctrings, 4816, theSaarco
* **Docs**: Add example of function with git, fix format of set_state_thresholds, 4834, jillnogold
* **BackwardsCompatibility**: Fix upgrading to mlrun 1.6.x from versions < 1.4, 4798, liranbg
* **Artifacts**: Fix timezone in created timestamp, 4825, TomerShor
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc19#bug-fixes)


Pull requests:
c80213fd Revert "[Project] Delete Project Background Task" (4863)
4fe63b1c [CI] Fix open source system test missing deleted argument (4862)
695ca555 [CI] Fix system test existing project (4861)
fd5c1dce [Model Monitoring] Save only the new fields that generated by the user to ModelMonitoringApplicationResult (4832)
33d7bd53 [Artifacts] Fix Project Level Artifact Path Template (4854)
fbee42af [Model Monitoring] Fix delete old parquet function (4830)
18e8603e [API] Fix delete runs error status code (4855)
692f4782 [Automation] Patch image name hotfix (4856)
8fa55f8c [Docs] Add MLflow tutorial (4694)
ffe3cbe5 [Requirements] Fix databricks requirements (4851)
f15aa2a8 [HTTPDB] Keep watching on created and aborting states (4852)
3ba1859d [CI] Fix system tests using git examples (4853)
3f8c35c9 [Build] Update build command (4849)
d004f804 [Automation] Support log collector image patch (4850)
ba071dd0 [API] Remove deprecated artifacts endpoints (4810)
b1e42ee9 [K8s] Map k8s api exceptions to mlrun exceptions (4843)
9b0523f3 [Docs] Add creating and using a custom/private hub (4710)
97969648 [CI] Updating system test (4846)
a9fd32b9 [API] Fix log of runtime attribute (4848)
c5593419 [Requirements] Bump storey to 1.6.14 (4847)
13c06c84 [Artifacts] Add the option to calculate the size of artifact (4828)
76a1f6e6 [Logs] Add get log size endpoint and sdk method (4836)
87a46ece [DB] Fix reusing db sessions (4840)
4de5585a [Project] Set default build base image (4842)
7b4bbf82 [Model Monitoring] Save all the metrics of an application in KV storage (4841)
e007b641 [Spark] Remove logging of spark options (4839)
e4f20bc2 [Datastore] Log databricks metadata run as an artifact (4831)
4021f31b [Requirements] Bump storey to 1.6.12 (4838)
c6148ece [Doc] Add `KubeResource` to docs and fix some malformed doctrings (4816)
b934cefe [CI] Run system test on self hosted runners (4837)
8624ceba [Docs] Add example of function with git, fix format of set_state_thresholds (4834)
24803795 [API] Force delete pods after grace period passed (4822)
1dc10e3f [Logs] Validate positive offset when getting logs (4829)
2fd93dcd [Project] Delete Project Background Task (4784)
23603b88 [Model Monitoring] Remove the factor from inf edge detection (4827)
174407d3 [BackwardsCompatibility] Fix upgrading to mlrun 1.6.x from versions < 1.4 (4798)
024af7d7 [API] Skip monitoring when CRD state is not populated (4811)
a5d0fc10 [Artifacts] Fix timezone in created timestamp (4825)

Features / Enhancements
* **Tests**: Rerun flaky scheduler test, 4817, alonmr
* **API**: Mark background task as failed when abort run fails, 4823, alonmr
* **KubeResource**: Rename state threshold `running` to `executing`, 4820, alonmr
* **Automation**: Support specific tag, 4819, yanburman
* **HTTPDB**: Warn list runs with notification does not include local runs, 4818, alonmr
* **MPI**: Crd state to run state mapping, 4809, alonmr
* **Feature Store**: Avoid catching and re-raising error, 4805, gtopper
* **UI**: [Features & enhancement](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc18#features-and-enhancements)

Bug fixes
* **Model Monitoring**: Fix histogram handling by always adding ±inf bins, 4824, jond01
* **Datastore**: Fix key field propagation from `featureset` to `onlinesource`, 4821, gtopper
* **Model Monitoring**: Fix the e2e sys test wait time, 4815, jond01
* **Config**: Fix nuclio version resolution for explicit ack, 4813, gtopper
* **Datastore**: Fix store manager reading datastore profile, 4814, alxtkr77
* **Notifications**: Fix pipeline notifications having non-deterministic run amount, 4793, quaark
* **Model Monitoring**: Fix an evidently import, 4812, jond01
* **UI**: [Bug fixes](https://github.com/mlrun/ui/releases/tag/v1.6.0-rc18#bug-fixes)


Pull requests:
b322c4b8 [Model Monitoring] Fix histogram handling by always adding ±inf bins(4824)
52cd6110 [Datastore] Fix key field propagation from `FeatureSet` to `OnlineSource` (4821)
ff225f89 [Tests] Rerun flaky scheduler test (4817)
10f420b5 [API] Mark background task as failed when abort run fails (4823)
d1030e4d [KubeResource] Rename state threshold `running` to `executing` (4820)
f8f0cd15 [Automation] Support specific tag (4819)
19ae9d94 [HTTPDB] Warn list runs with notification does not include local runs (4818)
da7cf4b3 [Model Monitoring] Fix the e2e sys test wait time (4815)
11c21ce5 [Config] Fix nuclio version resolution for explicit ack (4813)
c4eae3ac [Datastore] Fix store manager reading datastore profile (4814)
773cbb34 [Notifications] Fix Pipeline Notifications Having Non-Deterministic Run Amount (4793)
f7600253 [Model Monitoring] Fix an evidently import (4812)
41e2d19f [MPI] CRD state to run state mapping (4809)
571bbed5 [Feature Store] Avoid catching and re-raising error (4805)